FLAMEK8sBackend (flame_k8s_backend v0.7.0)

Kubernetes Backend implementation.

Usage

Configure the flame backend in our configuration or application setup:

# application.ex
children = [
  {FLAME.Pool,
   name: MyApp.SamplePool,
   backend: FLAMEK8sBackend,
   min: 0,
   max: 10,
   max_concurrency: 5,
   idle_shutdown_after: 30_000}
]

Options

The following backend options are supported:

• :manifest - If given, specifies the runner pod manifest. This can be either of:

  • a manifest map

  • an arity-2 function - it accepts the parent pod manifest and the app container part of the manifest (as determined by the :app_container_name option)

  See the "Manifest configuration" section below for more details.

• :env - A map with environment variables that should be passed to the runner. When specified, these environment variables take precendence over the ones defined in :manifest. This option is a convenience for passing extra values read at runtime. See the "Manifest configuration" section below for more details.

• :secret_env - Same as :env but these will be applied as a Kubernets secret and appended to the Pod via envFrom.

• :app_container_name - If your application pod runs multiple containers (initContainers excluded), use this option to pass the name of the container running this application. If not given, the first container in the list of containers is used to lookup the container image to be used for the runner pods.

• :omit_owner_reference - If true, no ownerReferences are configured on the runner pods. Defaults to false

• :log - The log level to use for verbose logging. Defaults to false.

Prerequisites

In order for this to work, your application needs to meet some requirements.

Env Variables

In order for the backend to be able to get informations from your pod and use them for the runner pods (e.g. env variables), you have to define POD_NAME and POD_NAMESPACE environment variables on your pod.

apiVersion: apps/v1
kind: Deployment
spec:
selector:
  matchLabels:
    app: myapp
template:
  spec:
    containers:
      - env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace

RBAC

Your application needs run as a service account with permissions to manage pods. This is a simple

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: myapp
  namespace: app-namespace
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: app-namespace
  name: pod-mgr
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create", "get", "list", "delete", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: myapp-pod-mgr
  namespace: app-namespace
subjects:
- kind: ServiceAccount
  name: myapp
  namespace: app-namespace
roleRef:
  kind: Role
  name: pod-mgr
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      serviceAccountName: my-app

Clustering

Your application needs to be able to form a cluster with your runners. Define POD_IP, RELEASE_DISTRIBUTION and RELEASE_NODE environment variables on your pods as follows:

apiVersion: apps/v1
kind: Deployment
spec:
template:
  spec:
    containers:
      - env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: RELEASE_DISTRIBUTION
          value: name
        - name: RELEASE_NODE
          value: my_app@$(POD_IP)

Manifest Configuration

You have full control over the runner pod manifest. For example, to set some environment variables and resources, you can do the following:

manifest = %{
  "spec" => %{
    "containers" => [
      %{
        "env" => [
          %{"name" => "FOO", "value" => "bar"}
        ],
        "resources" => %{
          "requests" => %{"memory" => "256Mi", "cpu" => "100m"},
          "limits" => %{"memory" => "256Mi", "cpu" => "400m"}
        }
      }
    ]
  }
}

{FLAME.Pool,
 name: MyApp.SamplePool,
 backend: {FLAMEK8sBackend, manifest: manifest}}

For more complex manifests, it is convenient to write the manifest as YAML. To do that, you can added the :yaml_elixir package, and use the ~y sigil:

manifest = ~y"""
metadata:
  spec:
    containers:
      - resources:
          requests:
            memory: 256Mi
            cpu: 100m
          limits:
            memory: 256Mi
            cpu: 400m
      - env:
          - name: FOO
            value: bar
"""

{FLAME.Pool,
 name: MyApp.SamplePool,
 backend: {FLAMEK8sBackend, manifest: manifest}}

You may want to automatically copy certain configuration from the parent pod. To do that, you can pass a function to build the manifest:

manifest_fun = fn parent_pod_manifest, app_container ->
  %{
    "metadata" => %{
      # ...
    },
    "spec" => %{
      "containers" => [
        %{
          # Copy all env vars and resources from the parent container definition.
          "env" => app_container["env"] || [],
          "envFrom" => app_container["envFrom"] || [],
          "resources" => app_container["resources"] || %{}
        }
      ]
    }
  }
end

{FLAME.Pool,
 name: MyApp.SamplePool,
 backend: {FLAMEK8sBackend, manifest: manifest_fun}}