Example Kubernetes Manifests using Go Templating

Updated 2 months ago by Michael Cretzman

To make your Kubernetes manifest reusable and dynamic, you can use Go templating and Harness built-in variables in combination in your Manifests files.

This topic shows some common examples and best practices.

Basic Values YAML and Manifests for Public Image

This is a simple example using the Artifact reference <+artifact.image>. It can be used whenever the public image is not hardcoded in manifests.

See Add Container Images as Artifacts for Kubernetes Deployments.

We use Go templates with a values.yaml file and manifests for deployment, namespace, and service. The manifests for deployment, namespace, and service are in a templates folder that's a peer of the values.yaml file.

values.yaml

This file uses the image: <+artifact.image> to identify the primary artifact added in the Harness Service Definition Artifacts section.

It also uses name: <+stage.variables.name> to reference a Stage variable name and namespace: <+infra.namespace> to reference the namespace entered in the Stage's Infrastructure Definition. Service type and ports are hardcoded.

The name, image, and namespace values are referenced in the manifests described later.

name: <+stage.variables.name>
replicas: 2

image: <+artifact.image>
# dockercfg: <+artifact.imagePullSecret>

createNamespace: true
namespace: <+infra.namespace>

# Service Type allow you to specify what kind of service you want.
# Possible values for ServiceType are:
# ClusterIP | NodePort | LoadBalancer | ExternalName
serviceType: LoadBalancer

# A Service can map an incoming port to any targetPort.
# targetPort is where application is listening on inside the container.
servicePort: 80
serviceTargetPort: 80

# Specify all environment variables to be added to the container.
# The following two maps, config and secrets, are put into a ConfigMap
# and a Secret, respectively.
# Both are added to the container environment in podSpec as envFrom source.
env:
config:
key1: value10
secrets:
key2: value2

templates/deployment.yaml

The deployments manifest references the name and image values from values.yaml. The manifest also contains the ConfigMap and Secret objects.

{{- if .Values.env.config}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{.Values.name}}
data:
{{.Values.env.config | toYaml | indent 2}}
---
{{- end}}

{{- if .Values.env.secrets}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}
stringData:
{{.Values.env.secrets | toYaml | indent 2}}
---
{{- end}}

{{- if .Values.dockercfg}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}-dockercfg
annotations:
harness.io/skip-versioning: true
data:
.dockercfg: {{.Values.dockercfg}}
type: kubernetes.io/dockercfg
---
{{- end}}

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{.Values.name}}-deployment
spec:
replicas: {{int .Values.replicas}}
selector:
matchLabels:
app: {{.Values.name}}
template:
metadata:
labels:
app: {{.Values.name}}
spec:
{{- if .Values.dockercfg}}
imagePullSecrets:
- name: {{.Values.name}}-dockercfg
{{- end}}
containers:
- name: {{.Values.name}}
image: {{.Values.image}}
{{- if or .Values.env.config .Values.env.secrets}}
envFrom:
{{- if .Values.env.config}}
- configMapRef:
name: {{.Values.name}}
{{- end}}
{{- if .Values.env.secrets}}
- secretRef:
name: {{.Values.name}}
{{- end}}
{{- end}}

templates/namespace.yaml

The namespace manifest references the namespace value from values.yaml.

{{- if .Values.createNamespace}}
apiVersion: v1
kind: Namespace
metadata:
name: {{.Values.namespace}}
{{- end}}

templates/service.yaml

The service manifest references the hardcoded service type and ports from values.yaml.

apiVersion: v1
kind: Service
metadata:
name: {{.Values.name}}-svc
spec:
type: {{.Values.serviceType}}
ports:
- port: {{.Values.servicePort}}
targetPort: {{.Values.serviceTargetPort}}
protocol: TCP
selector:
app: {{.Values.name}}

Private Artifact Example

When the image is in a private repo, you use the expression <+artifact.imagePullSecret> in the Secret and Deployment objects in your manifest.

This key will import the credentials from the Docker credentials file in the artifact.

It's much simpler to simple use the <+artifact.imagePullSecret> expression in the values.yaml file and then reference it in other manifests.

Using the values.yaml file above, we simply remove the comment in front of dockercfg: <+artifact.imagePullSecret>:

name: <+stage.variables.name>
replicas: 2

image: <+artifact.image>
dockercfg: <+artifact.imagePullSecret>

createNamespace: true
namespace: <+infra.namespace>
...

You don't need to make changes to the deployment manifest from earlier. It uses Go templating to check for the dockercfg value in values.yaml and applies it to Secret and Deployment.

If using the condition {{- if .Values.dockercfg}} to check for dockercfg in values.yaml.

...
{{- if .Values.dockercfg}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}-dockercfg
annotations:
harness.io/skip-versioning: true
data:
.dockercfg: {{.Values.dockercfg}}
type: kubernetes.io/dockercfg
---
{{- end}}

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{.Values.name}}-deployment
spec:
replicas: {{int .Values.replicas}}
selector:
matchLabels:
app: {{.Values.name}}
template:
metadata:
labels:
app: {{.Values.name}}
spec:
{{- if .Values.dockercfg}}
imagePullSecrets:
- name: {{.Values.name}}-dockercfg
{{- end}}
containers:
- name: {{.Values.name}}
image: {{.Values.image}}
...

Quotation Marks

The following example puts quotations around whatever string is in the something value. This can handle values that could otherwise be interpreted as numbers, or empty values, which would cause an error.

{{.Values.something | quote}}

You should use single quotes if you are using a value that might contain a YAML-like structure that could cause issues for the YAML parser.

Verbatim

Use indent and toYaml to put something from the values file into the manifest verbatim.

{{.Values.env.config | toYaml | indent 2}}

Indexing Structures in Templates

If the data passed to the template is a map, slice, or array it can be indexed from the template.

You can use {{index x number}} where index is the keyword, x is the data, and number is an integer for the index value.

If we had {{index names 2}} it is equivalent to names[2]. We can add more integers to index deeper into data. {{index names 2 3 4}} is equivalent to names[2][3][4].

Let's look at an example:

{{- if .Values.env.config}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{.Values.name}}-{{.Values.track}}
labels:
app: {{.Values.name}}
track: {{.Values.track}}
annotations:
harness.io/skip-versioning: "true"
data:
{{- if hasKey .Values.env .Values.track}}
{{index .Values.env .Values.track "config" | mergeOverwrite .Values.env.config | toYaml | indent 2}}
{{- else }}
{{.Values.env.config | toYaml | indent 2}}
{{- end }}
---
{{- end}}

{{- if .Values.env.secrets}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}-{{.Values.track}}
labels:
app: {{.Values.name}}
track: {{.Values.track}}
stringData:
{{- if hasKey .Values.env .Values.track}}
{{index .Values.env .Values.track "secrets" | mergeOverwrite .Values.env.secrets | toYaml | indent 2}}
{{- else }}
{{.Values.env.secrets | toYaml | indent 2}}
{{- end }}
---
{{- end}}

Iterate Over Existing Items

Here's an example inserting an element into an existing list in a manifest for Istio VirtualService and the Destination rule.

The critical line is:

{{- range $track := split " " .Values.nonPrimary }}

This line iterates over a list of existing items, where the list was computed with a simple Shell Script step and output to the context before the rollout.

VirtualService:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: {{ .Values.name }}-gateway-vs
labels:
app: {{ .Values.name }}
group: multiservice
spec:
hosts:
- "*"
gateways:
- ingressgateway
http:
{{- if .Values.nonPrimary }}
{{- range $track := split " " .Values.nonPrimary }}
{{- range $uri := $.Values.uri }}
- name: {{ $track }}
match:
- headers:
x-pcln-track:
exact: {{ $track }}
uri:
{{ $uri.matchType }}: {{ $uri.matchString }}
{{- if $.Values.rewrite }}
rewrite:
uri: {{ $.Values.rewrite }}
{{- end }}
route:
- destination:
host: {{ $.Values.name }}
subset: {{ $.Values.name }}-{{ $track }}
{{- end }}
{{- end }}
{{- end }}
{{- if .Values.hasPrimary }}
- name: primary
match:
{{- range $uri := .Values.uri }}
- uri:
{{ $uri.matchType }}: {{ $uri.matchString }}
{{- end }}
{{- if .Values.rewrite }}
rewrite:
uri: {{ .Values.rewrite }}
{{- end }}
route:
- destination:
host: {{ .Values.name }}
subset: {{ .Values.name }}-primary
{{- end }}

DestinationRule:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: {{ .Values.name }}
labels:
app: {{ .Values.name }}
group: multiservice
spec:
host: {{ .Values.name }}
subsets:
{{- if .Values.nonPrimary }}
{{- range $track := split " " .Values.nonPrimary }}
- name: {{ $.Values.name }}-{{ $track }}
labels:
track: {{ $track }}
{{- end }}
{{- end }}
{{- if .Values.hasPrimary }}
- name: {{ .Values.name }}-primary
labels:
track: primary
{{- end }}

For more information, see the Go text template documentation.

Notes

  • Harness Variables and Expressions may be added to values.yaml, not the manifests themselves. This provides more flexibility.
  • The values.yaml file used in a stage Service doesn't support Helm templating, only Go templating. Helm templating is fully supported in the remote Helm charts you add to your Service.
  • Harness uses Go template version 0.4. If you're used to Helm templates, you can download Go template and try it out locally to find out if your manifests will work. This can help you avoid issues when adding your manifests to Harness.
    You can install Go template version 0.4 locally to test your manifests.
    • Mac OS: curl -O https://app.harness.io/public/shared/tools/go-template/release/v0.4/bin/darwin/amd64/go-template
    • Linux: curl -O https://app.harness.io/public/shared/tools/go-template/release/v0.4/bin/linux/amd64/go-template
    • Windows: curl -O https://app.harness.io/public/shared/tools/go-template/release/v0.4/bin/windows/amd64/go-template
    For steps on doing local Go templating, see Harness Local Go-Templating on Harness Community.
  • Harness uses an internal build of Go templating. It cannot be upgraded. Harness uses Spring templates functions, excluding those functions that provide access to the underlying OS (env, expandenv) for security reasons.
    In addition, Harness uses the functions ToYaml, FromYaml, ToJson, FromJson.


Please Provide Feedback