Introduction

If you had to pick one property of a program that makes it easier to maintain, what would it be? Good documentation is certainly a candidate, but if the code itself is bad enough, that probably won’t help you. How about encapsulation? It can be liberating to know that the changes you’re making to a piece of code will only have local effects. However, what if there are other versions of that same code encapsulated elsewhere? Oops. To my mind, the most important property is lack of redundancy. The more places in the code that perform the same function or embody the same implied understanding, the harder it is to maintain without breaking something. This has been well-understood in the database world for decades; normalization is nearly a religious practice in relational database circles. The principle is important enough that it has a well-known acronym: DRY (Don’t Repeat Yourself). How might we apply this principle to Kubernetes manifests?

Kubernetes Redundancy

First of all, is this even a problem for Kubernetes manifests? For a single application, maybe not. The services, deployments, and other Kubernetes constructs will likely all be unique since any common behavior will be supported by replicated pods at runtime, not by replicated controllers in manifests. However… the ease of container-based deployment has made it common practice to run an application in a variety of environments. At Yipee.io, we run our own application in production, staging, and development environments, from public clouds to developer laptops. This ability to easily replicate a significant application is extremely powerful but brings with it the problem of maintaining several closely related but non-identical application configurations.

Existing Approaches

There have already been several attempts to address this problem within the industry. Here are some of the most prominent:

  • docker-compose provides the option to use multiple configuration files (for Swarm applications). Each additional configuration file overrides any previous ones for singular values and merges sets of values with those from earlier files.

  • Helm supports manifest templates that can be used to parameterize deployments. Sections of a YAML manifest can be marked as variables which are instantiated via separately specified values.

  • Ksonnet supports programmatic configuration of Kubernetes apps via code in the extended JSON dialect Jsonnet.

  • Kustomize has overlays that can be applied to base YAML manifests to produce specific configurations.

All of these work (with varying levels of convenience and power) but have issues:

  • docker-compose’s multiple configuration files only allow certain values to be changed and only in specific ways.

  • Helm templates require the base configuration to anticipate what parts might be changed by later overrides so that template variables can be inserted. Also, they require the user to learn an additional templating language in order to construct shared configurations.

  • Ksonnet uses a unique configuration language based on JSON. This forces users to learn a new formalism and makes configurations opaque for new users.

  • Kustomize’s overlays are just YAML which means users don’t have to learn a new language. However, the division of a model into many separate related files makes it hard to figure out just what a model contains and the overlays themselves cannot be read in isolation since they are updates to existing files.

The Yipee Approach

Yipee.io started out as a SaaS-based graphical modeling tool for building Kubernetes manifests and Docker Compose files. It was clear from the start that YAML, though capable of expressing complex configurations, left something to be desired as the sole representation of an application. In particular, though it’s relatively easy to modify an existing configuration for someone who knows how the application works, it’s difficult to see the big picture when looking at a large configuration in YAML. Yipee.io models, which represent Kubernetes objects graphically, make it far easier to understand the structure of an application. We believe the same holds true for managing multiple related configurations. We’re currently building an on-premise version of Yipee.io that runs in a user’s own Kubernetes cluster. Models in the on-premise Yipee.io can be derived from “parent” models and inherit their configurations. The derived models can then be modified but maintain their connections to their parents so that parent-level changes can be automatically propagated. The advantages of this approach include:

  • The model you’re working on is always complete. You don’t have to piece it together from chunks spread around the filesystem.

  • Graphical diffs between models (and versions of a single model) make it easy to see how a configuration has changed, even if large-scale changes have been made.

  • Upstream changes can be accepted/rejected in bulk or they can be addressed on a per-change basis. The system remembers when you change or remove a parent model object and won’t ask you to reprocess a change if you compare the same two models again.

  • Changes to ancestor models don’t accidentally break descendant models. Unless a user explicitly accepts upstream changes, the descendant models remain unchanged. If a user does want the changes, though, it’s trivial to accept them.

  • Subsystems can be included or embedded. If the subsystem is best viewed as a black box, it can be treated as if it were itself a Kubernetes object. If it’s more naturally seen as a set of top-level parts, it can be unpacked directly into the descendant model.

Contrast two representations of the same application. First, the Kubernetes manifests:

# Generated 2018-08-15T21:33:09.964Z by Yipee.io
# Application: Parse-MongoDB
# Last Modified: 2018-08-15T21:33:09.964Z

apiVersion: v1
kind: Service
metadata:
  name: parse-server
  annotations:
    yipee.io.modelId: 7adf4c72-4ca6-11e8-b87b-87cec0af93b2
    yipee.io.modelURL: https://staging2.yipee.io/editor/7adf4c72-4ca6-11e8-b87b-87cec0af93b2/fc55cc2c-4251-11e8-b067-4f2de2893da6
    yipee.io.contextId: fc55cc2c-4251-11e8-b067-4f2de2893da6
    yipee.io.lastModelUpdate: '2018-04-30T18:46:32.106Z'
spec:
  selector:
    run: parse-server
  ports:
  - port: 1337
    targetPort: 1337
    name: parse-server-1337
    protocol: TCP
    nodePort: 31337
  type: NodePort

---
apiVersion: v1
kind: Service
metadata:
  name: mongo
  annotations:
    yipee.io.modelId: 7adf4c72-4ca6-11e8-b87b-87cec0af93b2
    yipee.io.modelURL: https://staging2.yipee.io/editor/7adf4c72-4ca6-11e8-b87b-87cec0af93b2/fc55cc2c-4251-11e8-b067-4f2de2893da6
    yipee.io.contextId: fc55cc2c-4251-11e8-b067-4f2de2893da6
    yipee.io.lastModelUpdate: '2018-04-30T18:46:32.106Z'
spec:
  selector:
    app: mongo
  ports:
  - port: 27017
    targetPort: 27017
    name: peer
    protocol: TCP
  type: ClusterIP

---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
  annotations:
    yipee.io.lastModelUpdate: '2018-08-15T21:33:09.920Z'
    yipee.io.modelId: edb88f9e-641e-11e8-8a4d-f3d7597e94c6
    yipee.io.contextId: 89a94702-1558-11e7-a147-df26953bf272
    yipee.io.modelURL: https://app.yipee.io/editor/edb88f9e-641e-11e8-8a4d-f3d7597e94c6/89a94702-1558-11e7-a147-df26953bf272
spec:
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      partition: 0
  selector:
    matchLabels:
      name: MongoDB
      component: mongo
      app: mongo
  template:
    spec:
      imagePullSecrets: []
      containers:
      - name: init-mongo
        image: mongo:3.4.1
        command:
        - bash
        - /config/init.sh
        volumeMounts:
        - name: config
          mountPath: /config
      - name: mongodb
        ports:
        - containerPort: 27017
          protocol: TCP
          name: web
        image: mongo:3.4.1
        command:
        - mongod
        - --replSet
        - rs0
      volumes:
      - name: config
        configMap:
          name: mongo-init
      terminationGracePeriodSeconds: 10
    metadata:
      labels:
        name: MongoDB
        component: mongo
        app: mongo
  podManagementPolicy: OrderedReady
  replicas: 3
  serviceName: mongo

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: parse-server
  annotations:
    yipee.io.lastModelUpdate: '2018-08-15T21:33:09.920Z'
    yipee.io.modelId: edb88f9e-641e-11e8-8a4d-f3d7597e94c6
    yipee.io.contextId: 89a94702-1558-11e7-a147-df26953bf272
    yipee.io.modelURL: https://app.yipee.io/editor/edb88f9e-641e-11e8-8a4d-f3d7597e94c6/89a94702-1558-11e7-a147-df26953bf272
spec:
  selector:
    matchLabels:
      name: MongoDB
      component: parse-server
      run: parse-server
  rollbackTo:
    revision: 0
  template:
    spec:
      containers:
      - name: parse-server
        ports:
        - containerPort: 1337
          protocol: TCP
          name: parse-server
        image: parseplatform/parse-server
        env:
        - name: PARSE_SERVER_APPLICATION_ID
          value: my-app-id
        - name: PARSE_SERVER_DATABASE_URI
          value: mongodb://mongo-0.mongo:27017,mongo-1.mongo:27017,mongo-2.mongo:27017/dev?replicaSet=rs0
        - name: PARSE_SERVER_MASTER_KEY
          value: my-master-key
      restartPolicy: Always
      imagePullSecrets: []
    metadata:
      labels:
        name: MongoDB
        component: parse-server
        run: parse-server
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  replicas: 1
  revisionHistoryLimit: 2

Here’s the Yipee.io version:

I think it’s clear that the graphical view helps make the big picture evident. You can see instantly that the application is built from three container types, one deployment, one stateful set, and two services. You can also see how they’re related. If the structure of the application in an ancestor model changes, a graphical diff will make it obvious what changed and what depends on the change.

Summary

The rise of container-based deployment has led many companies to construct multiple variants of their application configurations. These variants have significant commonality and generate the same issues around redundancy that software developers and database administrators have dealt with for decades. The issues are serious enough that, even though Kubernetes is a relatively young technology, multiple players in the industry have attempted to address them with varying levels of success. Yipee.io is developing a graphical approach to managing Kubernetes models that supports embedded models, model inheritance, and model versioning with an intuitive visual editor and graphical diff capability. The graphical diff makes it easy to manage derived models and propagate changes from ancestors to descendants.

If anything you’ve read here sounds interesting to you, please visit our DRY Kubernetes page and fill out the form to let us know. We’re looking for early adopters to help ensure that we’re solving everyone’s problems and not just our own. Also, if you’re a fan of GraphQL, you might want to check out our Kubernetes GraphQL interface that we’re building in support of our on-premise product: Kubeiql.

Thanks for reading!