Immutable Delegate Overview

Updated 5 days ago by Michael Cretzman

Currently, this feature is behind the feature flag USE_IMMUTABLE_DELEGATE. Contact Harness Support to enable the feature. Enabling this Feature Flag means that any future Delegates you install will be immutable, but it doesn’t impact existing Legacy Kubernetes Delegates.

Containerized Delegates can be installed as Legacy or Immutable Delegates.

Immutable Delegates differ from Legacy Delegates in a number of ways.

  • Legacy:
  • Immutable:
    • All binaries and dependencies are installed when the image is built.
    • Ephemeral pods for Kubernetes Delegates.
    • Security scanning of Delegates is possible.
    • Faster setup time.
    • Custom Delegate images can be created by simply building a new image from each image/version Harness releases. Installing additional tools like Terraform and others becomes a lot easier.
    • See Install an Immutable Kubernetes Delegate.

The standard installation process in the Harness Manager for both types is the same.

What types of Delegates are supported for Immutable Delegates?

Currently, only Kubernetes Delegates. We will add support for other containerized Delegates soon.

Are there any UI changes?


What Base Image is used by the Immutable Delegate?

The Immutable Delegate is based on redhat/ubi8-minimal:8.4.

Is the Image Signed?

Yes, the image is signed. You can check it by running the command docker trust inspect --pretty harness/delegate-immutable:22.03.74407.

Does the Immutable Delegate emit Prometheus Metrics?

Immutable delegates come with Prometheus annotations and these metrics exposed:

  1. Current tasks running.
  2. Tasks waiting in queue.
  3. Time taken in tasks executions.

Below are the annotations in the YAMK. Prometheus can be configured to scrape metrics on port 3460 and endpoint /api/metrics.

annotations: "true" "3460" "/api/metrics"

Root vs Non-Root

The Immutable Delegate doesn’t have a root image. There are only two images for the same tag and they are both non-root. For example:

  • harness/delegate-immutable:22.03.74411
  • harness/delegate-immutable:22.03.74411.minimal

The first one has the client tools like kubectl, Helm, ChartMuseum, etc included. The minimal image does not have those tools.

If you want to install more tools in the image, we recommend you make your own image.

Do I need to Migrate all my Existing Legacy Delegates?

No, existing (non-immutable) Legacy Delegates can continue working in parallel with any new Immutable Delegates.

You can migrate Legacy Delegates over time.

How do I Migrate my Existing Delegates to be Immutable?

Currently there is no automated process for this.

You will need to add new Immutable Delegates.

What if the Immutable Delegate keeps Restarting?

Immutable Delegates are configured to get probed by Kubernetes for liveness every 10s. If two probes fail, the pod will restart.

Make sure that the liveness/startup probes are accessible by Kubernetes and that they add sufficient delay so that the Delegate has enough time to start up in your environment.

With the default resource configuration, the Delegate should take around 45-60s to startup.

How do I use a Custom Delegate Image?

See Custom Installation in Install an Immutable Kubernetes Delegate

Which Upgrader Image should I Use?

For the upgrader component, we will be releasing versioned images (e.g. 1.0.0), but also a latest tag, which represents the latest released image (highest version number) for the time being.

The upgrader component doesn't yet support automatic upgrading, so we will default to using the latest image tag (which will get pulled every time upgrader runs, effectively upgrading it).

Please Provide Feedback