Enforcing Policies in Loft

Policies allow you to customize admission control and extend Loft. As Loft internally uses Kubernetes resources for anything (including UI requests), you can customize almost everything with policies.

The following use cases are possible with policies:

• Custom validation of users, teams, spaces, clusters and other objects. For example, you can:
  • Deny creation, updating or deletion of resources only for specific users or teams
  • Deny security critical objects such as privileged pods in clusters
  • Ensure unique names, hosts or other properties within a cluster
• Automatic mutation of created or updated resources. For example, you can:
  • Add a label or annotation which user or team created an object
  • Inject sidecar containers into a pod
  • Add groups, cluster account templates etc. automatically to a user
• Automation of certain cluster tasks, such as:
  • Automatic creation, update or deletion of users, accounts, spaces etc. on a certain condition
  • Sync certain resources within a cluster
  • Garbage collection of not needed resources

Installing JsPolicy

Policies are based on jsPolicy and you will need to install the jsPolicy app into the cluster if you want to use policies. JsPolicy is a recommended app in Loft and can be installed via the Loft UI.

Installing jsPolicy via the Loft UI

Alternatively, you can also install jsPolicy with helm.

Creating policies

Policies can be created either through the Loft UI or through directly creating JsPolicy objects in the cluster. For a more extensive documentation how to write policies, please check the jsPolicy docs.

UI

Go to the Clusters > Policies view and click on 'Add Policy'.

Create a new JsPolicy

Metadata

In the metadata section of the policy, you can set the name of the policy, which has to be in domain format (e.g. my-policy.jspolicy.com or any other domain you want), the annotations and labels of the policy object.

In the options section, you can choose from a variety of examples to start from or create your own from scratch. The following options exist:

Policy Type

The type of the policy. For a more extensive explanation check the jsPolicy documentation. The type can be one of:

• Validating: Validating policies are executed as part of Kubernetes requests after the execution of mutating policies. The objective of validating policies is to inspect the request and then to either deny or allow it.
• Mutating: Mutating policies are executed as part of Kubernetes requests right after the API server performs authentication and authorization (RBAC). The objective of mutating policies is to change the payload (Kubernetes object) provided in a request, e.g. add/remove a label in the metadata.labels of a Kubernetes object that is being created.
• Controller: Unlike mutating and validating policies, controller policies are not part of the lifecycle of a request to the Kubernetes API server. Controller policies are triggered by the Events that Kubernetes creates for each change of the cluster state in etcd, e.g. if you create a new Deployment or update an Ingress, Kubernetes creates an Event for this change. The objective of controller policies is to react to changes to your cluster's state.

Policy Scope

The scope of the policy. This can be either *, Cluster or Namespaced and refers to the scope of Kubernetes resources. A Namespaced policy will only match resources that can be created within a namespace, while Cluster will only match cluster-wide resources such as ClusterRole etc.

Matching Resources

The Kubernetes resources this policy should match. You can specify multiple resources here or a wildcard that matches all resources.

Matching Operations

Operations are the operations the policy cares about and can be one or more of: CREATE, UPDATE, DELETE, CONNECT or *.

Policy Javascript

This is the actual JavaScript payload that is executed if a policy matches a resource and operation. Please checkout the jsPolicy documentation and jsPolicy examples for more information about how the payload could look like.

After you have configured the policy, press 'Create' to add it to the cluster.

kubectl

There are multiple ways to create policies for the cluster. For all options, please check the jsPolicy documentation. This guide will cover the easiest option by creating a simple JsPolicy object.

Make sure your kube context targets the cluster where jsPolicy was installed in. To create a new policy, create a file called policy.yaml with:

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny.resource.example"
spec:
operations: ["CREATE"]
resources: ["pods"]
javascript: |
  // deny default namespace
  if (request.namespace === "default") {
    deny("No new pod allowed in the default namespace")
  }

Then do a kubectl create -f policy.yaml to create the new policy.

Deleting policies

You can delete a policy either via the Loft UI or kubectl directly.

UI

Go to the Clusters > Policies view and click on 'delete' in the actions column.

Delete a JsPolicy

kubectl

You can a delete a policy by executing `kubectl delete jspolicy my-policy.domain.tld` in the cluster where the policy was created in.

Policy Audit

If a request was denied by a policy, it will write this violation into its policy log. You can view all policy violations in the Audit > Policy Violations view.

View policy violations

This view shows information about the request that was denied, such as which user was responsible for the violation (if there was any), the action taken, the returned message etc.

Policy Examples

This section holds several examples for useful policies within loft. For more examples and explanations checkout the jsPolicy documentation.

You can apply these examples by creating a policy.yaml and running kubectl apply -f policy.yaml in the target cluster.

Deny Space Owned By Team

This policy denies spaces that are owned by teams.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-teams-spaces.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["spaces"]
  javascript: |
    // Do not allow creation of spaces that have a team as owner
    if (request.object?.spec?.account) {
        const account = get("Account", "config.kiosk.sh/v1alpha1", request.object.spec.account);
        if (account?.metadata?.labels?.["loft.sh/team"]) {
            deny("Teams cannot own spaces");
        }
    }

Deny Pods of User

This policy will deny pod creation only for a single Loft user called test.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-user-pods.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  javascript: |
    // Do not allow the user "test" to create pods anywhere
    if (request.userInfo?.groups?.includes("loft:user:test")) {
        deny("Sorry test, but you are not allowed to create pods in this cluster");
    }

Deny Resources in default Namespace

This policy will deny all new resources that should be created in the default namespace.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-default-namespace.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["*"]
  scope: Namespaced
  javascript: |
    if (request.namespace === "default") {
        deny("Creation of resources within the default namespace is not allowed!");
    }

Deny Privileged Pods

This policy will deny privileged pods.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-privileged-pods.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  javascript: |
    const containers = request.object.spec?.containers || [];
    const initContainers = request.object.spec?.initContainers || [];

    [...containers, ...initContainers].forEach(container => {
        if (container.securityContext?.privileged) {
            deny("spec.containers[*].securityContext.privileged is not allowed")
        }
    })

Deny Ingress Classes

This policy will deny all ingress classes except nginx.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-ingress-classes.resource.example"
spec:
  operations: ["CREATE", "UPDATE"]
  resources: ["ingresses"]
  javascript: |
    // ingress class can be set via annotation "kubernetes.io/ingress.class" or through spec.ingressClassName.
    const allowedIngressClasses = ["nginx"];
    const ingressClasses = [request.object.metadata?.annotations?.["kubernetes.io/ingress.class"], request.object.spec.ingressClassName];
    const notAllowed = ingressClasses.filter(ingressClass => ingressClass && !allowedIngressClasses.includes(ingressClass));
    if (notAllowed.length > 0) {
      deny(`ingress class ${notAllowed[0]} is not allowed`);
    }

Allow Space only if Part of Team

This policy will only allow space creation if the user is part of team 'admins'.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "allow-space-creation.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["spaces"]
  javascript: |
    // Allow space creation only if part of team "admins"
    if (!request.userInfo?.groups?.includes("loft:team:admins")) {
        deny("Sorry you cannot create spaces since you are not part of the admins team");
    }

Set User as Annotation On New Object

This policy will set the user that has created it on a new kubernetes object.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "set-user-as-annotation.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["*"]
  scope: Namespaced
  type: Mutating
  javascript: |
    // Set the current user as an annotation on the object
    const allAccounts = list("Account", "config.kiosk.sh/v1alpha1");
    
    // Find account of the user by subject
    const userAccount = allAccounts.find(account => {
        // flatten account subjects
        const subjects = account.spec?.subjects?.map(subject => subject.name) || [];
        return subjects.includes(request.userInfo.username);
    });
    if (userAccount) {
        if (!request.object.metadata) {
            request.object.metadata = {};
        }
        if (!request.object.metadata.annotations) {
            request.object.metadata.annotations = {};
        }
        request.object.metadata.annotations["loft.sh/created-by-user"] = userAccount.metadata?.labels?.["loft.sh/user"];
        mutate(request.object);
    }

Inject Sidecar Container

This policy will inject a sidecar container into pods that have an annotation "inject-side-car".

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "inject-sidecar.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  type: Mutating
  javascript: |
    // if the annotation inject-side-car: 'true' is set on the pod we inject a sidecar
    if (request.object.metadata?.annotations?.["inject-side-car"] === "true") {
      if (!request.object.spec.containers) {
        request.object.spec.containers = [];
      }
        
      // add sidecar
      request.object.spec.containers.push({
        name: "injected-container",
        image: "busybox",
        command: ["sleep", "3600"]
      });
    
      mutate(request.object);
    }

Unique Ingress Hosts

This policy will ensure that ingress hosts are unique in the cluster.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "unique-ingress-hosts.resource.example"
spec:
  operations: ["CREATE", "UPDATE"]
  resources: ["ingresses"]
  type: Mutating
  javascript: |
    // iterate over all ingresses in cluster and check for same hosts
    list("Ingress", "networking.k8s.io/v1").forEach(ingress => {
      // don't check self on UPDATE
      if (ingress.metadata.name === request.name && ingress.metadata.namespace === request.namespace) {
        return;
      }
    
      ingress.spec.rules?.forEach(rule => {
        request.object.spec.rules?.forEach(objRule => {
          if (rule.host === objRule.host) {
            deny(\`ingress ` + "${ingress.metadata.namespace}/${ingress.metadata.name} already uses host ${rule.host}" + `\`);
          }
        })
      })
    })

Sync Secrets

This policy will sync secrets that have the labels owner-name and owner-namespace with a parent secret that has the label sync. On a change to a parent or child secret, the controller policy will evaluate if an update is necessary and update the child secret if it differs from the parent secret.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "sync-secrets.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["secrets"]
  type: Controller
  javascript: |
    // this policy will sync changes to a secret to all other secrets in the cluster
    // that have labels in the form of: {"owner-name": SECRET, "owner-namespace": NAMESPACE}
    
    // is this a parent or child secret?
    const labels = request.object.metadata?.labels;
    let syncSecret = undefined; 
    if (labels?.["sync"] === "true") {
      syncSecret = request.object;
    } else if (labels?.["owner-name"] && labels?.["owner-namespace"]) {
      syncSecret = get("Secret", "v1", labels?.["owner-namespace"] + "/" + labels?.["owner-name"]);
    }
    
    // this is not a secret we should sync
    if (!syncSecret) {
      allow();
    }
    
    // list all owned secrets
    list("Secret", "v1", {
      labelSelector: \`owner-name=`+"${syncSecret.metadata.name},owner-namespace=${syncSecret.metadata.namespace}"+`\`
    }).forEach(secret => {
      // only update if data has actually changed
      if (JSON.stringify(secret.data) === JSON.stringify(syncSecret.data)) {
        return;
      }
    
      // sync the secret data
      secret.data = syncSecret.data;
      const updateResult = update(secret);
      if (!updateResult.ok) {
        // just requeue if we encounter an error
        requeue(updateResult.message);
      } else {
        print(\`Successfully synced secret `+"${secret.metadata.namespace}/${secret.metadata.name}"+`\`);
      }
    });

Why jsPolicy and not OPA / Kyverno?

Loft does not require you to use jsPolicy for policies, and you can always use your own preferred solution. Every policy engine should work fine with Loft. However, if you want to use the policies view in Loft, this currently only works with jsPolicy.

We decided to integrate jsPolicy instead of another policy engine, because jsPolicy is the most flexible one that can cover the most use cases within Loft. Especially with controller policies and mutating webhooks that are not at all or only partly supported in other policy engines, jsPolicy has a huge advantage in terms of flexibility and expressiveness.