Steering Straight with Helm Charts Best Practices

Kubernetes, the popular orchestration tool for container applications, is named for the Greek word for “pilot,” or the one who steers the ship. But as in any journey, the navigator can only be as successful as the available map.

An application’sHelmchart is that map, a collection of files that can be deployed from ahelm charts repositorythat describe a related set of K8s resources. Crafting your Helm charts in the most effective way will help Kubernetes maneuver through the shoals when it deploys containers into your production environment.

But there are other ways to go adrift too, as I found while developing publicly available K8s charts to deploy products. With every pull request, feedback from the Helm community helped steer me to the Helm charts best practices that offered the strongest results for both operating and updating containers.

Learn more: 10 Helm tutorials to start your Kubernetes journey

Here are some things to consider when writing K8s charts that will be used by the community or customers in production. Among the things you need to think about are:

• What dependencies do you need to define?
• Will your application need a persistent state to operate?
• How will you handle security through secrets and permissions?
• How will you control running kubelet containers?
• How will you assure your applications are running and able to receive calls?
• How will you expose the application’s services to the world?
• How will you test your chart?

This guide offers some best practices to structure and specify your Helm charts that will help K8s deliver your container applications smoothly into dock.

Helm is for everyone: Get a free Guide to Helm

要开始ed

Before you start, make sure you are familiar with the essential procedures fordeveloping Helm charts.

In this guide, we will create a Helm chart that follows the best practices we recommend to deploy a two-tiercreate, read, update, and delete (CRUD) application for the Mongo database using Express.js.

You can find the source code of our example application inexpress-crud in GitHub.

Creating and filling the Helm chart

Let’s create our template helm chart using the helm client’screate command:

$ helm create express-crud

This will create a directory structure for anexpress-crudHelm chart.

To start, update the chart metadata in theChart.yamlfile that was just created. Make sure to add proper information forappVersion(the application version to be used as docker image tag),description,version(aSemVer 2version string),sources,maintainersandicon.

apiVersion: v1 appVersion: "1.0.0" description: A Helm chart for express-crud application name: express-crud version: 0.1.0 sources: - https://github.com/jainishshah17/express-mongo-crud maintainers: - name: myaccount email: myacount@mycompany.com icon: https://github.com/mycompany17/mycompany.com/blob/master/app/public/images/logo.jpg home: https://mycompany.com/

Defining Dependencies

If your application hasdependencies, then you must create arequirements.yamlfile in the Helm chart’s directory structure that specifies them. Since our application needs themongodbdatabase, we must specify it in thedependencieslist of therequirements.yamlfile we create.

Arequirements.yamlfor this example contains:

dependencies: - name: mongodb version: 3.0.4 repository: https://kubernetes-charts.storage.googleapis.com/ condition: mongodb.enabled

Once arequirements.yamlfile is created, you must run thedependency updatecommand in the Helm client:

$ helm dep update

Creating deployment files

The deployment files of your Helm chart reside in the\templatessubdirectory and specify how K8s will deploy the container application.

As you develop your deployment files, there are some key decisions that you will need to make.

Deployment Object vs StatefulSet Object

The deployment file you create will depend on whether the application requires K8s to manage it as a Deployment Object or aStatefulSet Object.

A Deployment Object is a stateless application that is declared in the filenamedeployment.yamland specifies thekindparameter asdeployment.

A Stateful Object is for applications that are stateful and used with distributed systems. They are declared in the filenamestateless.yamland species thekindparameter asstateful.

As this application does not need state to be persisted I am using deployment object.
Thedeployment.yamlfile has already been created by thehelm createcommand.

We will use AppVersion as the Docker image tag for our application. That allows us to upgrade Helm chart with new version of Application by just changing value inChart.yaml

image: "{{ .Values.image.repository }}:{{ default .Chart.AppVersion .Values.image.tag }}"

Secret Versus ConfigMap

You will need to determine which of the credentials or configuration data is appropriate to store assecrets, and which can be in aConfigMap.

Secrets are for sensitive information such as passwords that K8s will store in an encrypted format.

A ConfigMap is a file that contains configuration information that may be shared by applications. The information in a ConfigMap is not encrypted, so should not contain any sensitive information.

In this example, we shall allow Helm to pull docker images from privatedocker registriesusing image pull secrets.

This procedure relies on having a secret available to the Kubernetes cluster that specifies the login credentials for the repository manager. This secret can be created by akubectl commandline such as:

$ kubectl create secret docker-registry regsecret --docker-server=$DOCKER_REGISTRY_RUL --docker-username=$USERNAME --docker-password=$PASSWORD --docker-email=$EMAIL

In thevalues.yamlfile of your Helm chart, you can then pass the secret name to a value:

imagePullSecrets: regsecret

You can then make use of the secret to allow Helm to access the docker registry through these lines indeployment.yaml:

{{- if .Values.imagePullSecrets }} imagePullSecrets: - name: {{ .Values.imagePullSecrets }} {{- end }}

For secrets available to the application, you should add that information directly tovalues.yaml.

For example, to configure our application to access mongodb with a pre-created user and database, add that information invalues.yaml

mongodb: enabled: true mongodbRootPassword: mongodbUsername: admin mongodbPassword: mongodbDatabase: test

Note that here we do not hardcode default credentials in our Helm chart. Instead, we use logic to randomly generate password when it is not provided via –set flag orvalues.yaml

We will use a secret to pass mongodb credentials to our application, through these lines indeployment.yaml.

env: - name: DATABASE_PASSWORD valueFrom: secretKeyRef: name: {{ .Release.Name }}-mongodb key: mongodb-password

You can control the running of kubelet containers either through specializedInit Containersor throughContainer Lifecycle Hooks.

In our example, add theseinitContainersspecifications todeployments.yamlto hold the start of our application until the database is up and running.

initContainers: - name: wait-for-db image: "{{ .Values.initContainerImage }}" command: - 'sh' - '-c' - > until nc -z -w 2 {{ .Release.Name }}-mongodb 27017 && echo mongodb ok; do sleep 2; done

Adding Readiness and Liveness Probes

通常是一个好主意来添加一个准备和李veness probe to check the ongoing health of the application. If you don’t, then the application could fail in a way that it appears to be running, but doesn’t respond to calls or queries.

These lines in thedeployment.yamlfile will add those probes to perform periodic checks:

livenessProbe: httpGet: path: '/health' port: http initialDelaySeconds: 60 periodSeconds: 10 failureThreshold: 10 readinessProbe: httpGet: path: '/health' port: http initialDelaySeconds: 60 periodSeconds: 10 failureThreshold: 10

Adding RBAC Support

These procedures will addrole-based access control (RBAC)support to our chart, when it is required by an application.

Step 1: Create aRole byadding the following content in arole.yamlfile:
A Role can only be used to grant access to resources within a single namespace.

{{- if .Values.rbac.create }} apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: labels: app: {{ template "express-crud.name" . }} chart: {{ template "express-crud.chart" . }} heritage: {{ .Release.Service }} release: {{ .Release.Name }} name: {{ template "express-crud.fullname" . }} rules: {{ toYaml .Values.rbac.role.rules }} {{- end }}

Step 2: CreateRoleBinding byadding the following content in arolebinding.yamlfile:
A ClusterRole can be used to grant the same permissions as a Role, but because they are cluster-scoped, they can also be used to grant access to:

• cluster-scoped resources (like nodes)
• non-resource endpoints (like “/healthz”)
• namespaced resources (like pods) across all namespaces

{{- if .Values.rbac.create }} apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: app: {{ template "express-crud.name" . }} chart: {{ template "express-crud.chart" . }} heritage: {{ .Release.Service }} release: {{ .Release.Name }} name: {{ template "express-crud.fullname" . }} subjects: - kind: ServiceAccount name: {{ template "express-crud.serviceAccountName" . }} roleRef: kind: Role apiGroup: rbac.authorization.k8s.io name: {{ template "express-crud.fullname" . }} {{- end }}

Step 3: Create aServiceAccountbyadding the following content in aserviceaccount.yamlfile:

A service account provides an identity for processes that run in a Pod.

{{- if .Values.serviceAccount.create }} apiVersion: v1 kind: ServiceAccount metadata: labels: app: {{ template "express-crud.name" . }} chart: {{ template "express-crud.chart" . }} heritage: {{ .Release.Service }} release: {{ .Release.Name }} name: {{ template "express-crud.serviceAccountName" . }} {{- end }}

Step 4: Use helper template to set ServiceAccount name.
We will do that by adding following content in_helpers.tplfile

{{/* Create the name of the service account to use */}} {{- define "express-crud.serviceAccountName" -}} {{- if .Values.serviceAccount.create -}} {{ default (include "express-crud.fullname" .) .Values.serviceAccount.name }} {{- else -}} {{ default "default" .Values.serviceAccount.name }} {{- end -}} {{- end -}}

Adding a Service

Now it’s time to expose our application to the worldthrough a service.

服务允许您的应用程序接收traffic through an IP address. Services can be exposed in different ways by specifying atype:

We will do that by adding the following content toservice.yaml:

apiVersion: v1 kind: Service metadata: name: {{ template "express-crud.fullname" . }} labels: app: {{ template "express-crud.name" . }} chart: {{ template "express-crud.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} spec: type: {{ .Values.service.type }} ports: - port: {{ .Values.service.externalPort }} targetPort: http protocol: TCP name: http selector: app: {{ template "express-crud.name" . }} release: {{ .Release.Name }}

Note that in the above, for our servicetypewe reference a setting in ourvalues.yaml:

service: type: LoadBalancer internalPort: 3000 externalPort: 80

Values.yaml Summary

Defining many of our settings in avalues.yamlfile is a good practice to help keep your Helm charts maintainable.

This is how thevalues.yamlfile for our example appears, showing the variety of settings we define for many of the features discussed above:

# express-mongo-crud的默认值。#这是a YAML-formatted file. # Declare variables to be passed into your templates. ## Role Based Access Control ## Ref: https://kubernetes.io/docs/admin/authorization/rbac/ rbac: create: true role: ## Rules to create. It follows the role specification rules: - apiGroups: - '' resources: - services - endpoints - pods verbs: - get - watch - list ## Service Account ## Ref: https://kubernetes.io/docs/admin/service-accounts-admin/ ## serviceAccount: create: true ## The name of the ServiceAccount to use. ## If not set and create is true, a name is generated using the fullname template name: ## Configuration values for the mongodb dependency ## ref: https://github.com/kubernetes/charts/blob/master/stable/mongodb/README.md ## mongodb: enabled: true image: tag: 3.6.3 pullPolicy: IfNotPresent persistence: size: 50Gi # resources: # requests: # memory: "12Gi" # cpu: "200m" # limits: # memory: "12Gi" # cpu: "2" ## Make sure the --wiredTigerCacheSizeGB is no more than half the memory limit! ## This is critical to protect against OOMKill by Kubernetes! mongodbExtraFlags: - "--wiredTigerCacheSizeGB=1" mongodbRootPassword: mongodbUsername: admin mongodbPassword: mongodbDatabase: test # livenessProbe: # initialDelaySeconds: 60 # periodSeconds: 10 # readinessProbe: # initialDelaySeconds: 30 # periodSeconds: 30 ingress: enabled: false annotations: {} # kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" path: / hosts: - chart-example.local tls: [] # - secretName: chart-example-tls # hosts: # - chart-example.local initContainerImage: "alpine:3.6" imagePullSecrets: replicaCount: 1 image: repository: jainishshah17/express-mongo-crud # tag: 1.0.1 pullPolicy: IfNotPresent service: type: LoadBalancer internalPort: 3000 externalPort: 80 resources: {} # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: # cpu: 100m # memory: 128Mi # requests: # cpu: 100m # memory: 128Mi nodeSelector: {} tolerations: [] affinity: {}

Testing and Installing the Helm Chart

It’s very important to test our Helm chart, which we’ll do using thehelm lintcommand.

$ helm lint ./ ## Output ==> Linting ./ Lint OK 1 chart(s) linted, no failures

Use thehelminstallcommand to deploy our application using helm chart on Kubernetes.

$ helm install --name test1 ./ ## Output NAME: test1 LAST DEPLOYED: Sat Sep 15 09:36:23 2018 NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1beta1/Deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE test1-mongodb 1 1 1 0 0s ==> v1beta2/Deployment test1-express-crud 1 1 1 0 0s ==> v1/Secret NAME TYPE DATA AGE test1-mongodb Opaque 2 0s ==> v1/PersistentVolumeClaim NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE test1-mongodb Pending standard 0s ==> v1/ServiceAccount NAME SECRETS AGE test1-express-crud 1 0s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test1-mongodb ClusterIP 10.19.248.205 27017/TCP 0s test1-express-crud LoadBalancer 10.19.254.169 80:31994/TCP 0s ==> v1/Role NAME AGE test1-express-crud 0s ==> v1/RoleBinding NAME AGE test1-express-crud 0s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE test1-mongodb-67b6697449-tppk5 0/1 Pending 0 0s test1-express-crud-dfdbd55dc-rdk2c 0/1 Init:0/1 0 0s NOTES: 1. Get the application URL by running these commands: NOTE: It may take a few minutes for the LoadBalancer IP to be available. You can watch the status of by running 'kubectl get svc -w test1-express-crud' export SERVICE_IP=$(kubectl get svc --namespace default test1-express-crud -o jsonpath='{.status.loadBalancer.ingress[0].ip}') echo https://$SERVICE_IP:80

Running the above helm install command will produce an External_IP for the Load Balancer. You can use this IP address to run the application.

This is how our application appears when run:

Wrapping Up

As you can see from this example, Helm is an extremely versatile system that allows you a great deal of flexibility in how you structure and develop a chart. Doing so in the ways that match the conventions of the Helm community will help ease the process of submitting your Helm charts for public use, as well as making them much easier to maintain as you update your application.

The completed Helm charts for this example project can be found in theexpress-crudrepo on GitHub, and you may review these functioning files to help you to more thoroughly understand how they work.

To explore more examples, you can review my sample repository ofHelmchartsfor deploying products to Kubernetes.

Additional Resources:

• Helm repository best practices
• What is Helm in Kubernetes
• Kubernetes Cheat Sheet