Trigger Pipelines on a New Artifact

Updated 4 days ago by Michael Cretzman

You can trigger Harness Pipelines in response to a new artifact version being added to a registry.

For example, every time a new Docker image is pushed to your Docker Hub account, it triggers a CD Pipeline that deploys it automatically.

On New Artifact Triggers simply listen to the registry where one or more of the artifacts in your Pipeline are hosted.

You can set conditions on the Triggers, such as matching a Docker tag or label or a traditional artifact build name or number.

This Trigger is a simple way to automate deployments for new builds.

In this topic:

Before You Begin

Limitations

  • You can only use an On New Artifact Trigger if you have added an artifact in the Stage's Service Definition.
    If the artifact is hardcoded in your Kubernetes manifests, an On New Artifact Trigger will not work correctly. Instead, you can use a Harness Git-based or Manifest-based Trigger.
    You reference an artifact in the Stage's Service Definition in your manifests using the expression <+artifact.image>. See Add Container Images as Artifacts for Kubernetes Deployments.
  • If more than one artifact is collected during the polling interval (two minutes), only one deployment will be started and will use the last artifact collected.
  • The Trigger is executed based on file names and not metadata changes.
  • Do not trigger on the latest tag of an artifact, such as a Docker image. With latest, Harness only has metadata, such as the tag name, which has not changed, and so Harness does not know if anything has changed. The Trigger will not be executed.

Visual Summary

This 5min video walks you through building an app from source code and pushing it to Docker Hub using Harness CIE, and then having an On New Artifact Trigger execute a CD Pipeline to deploy the new app release automatically.

Review: Artifact Polling

Once you have created a Trigger to listen for new artifacts, Harness will poll for new artifacts continuously.

Polling is immediate because Harness uses a perpetual task framework that constantly monitors for new builds/tags.

Input Sets and Artifacts

When you create a Trigger, you can select Input Sets to apply to the Pipeline when the Trigger is executed.

If the Input Set you select has a Fixed Value for the artifact Tag (for example, 2), the Trigger will work, but it will not select the new artifact. Instead, Harness will select the artifact with the Tag value in the Input Set (2).

Instead, in the Input Set's Tag setting, use the expression <+trigger.manifest.build>.

The Input Set option is for settings other than the Tag setting of the artifact you want to pull. For example, the Artifact Server or the Tag of a sidecar artifact.

Step 1: Create Trigger

Select a Harness Pipeline that includes an artifact in the Stage's Service Definition.

You reference an artifact in the Stage's Service Definition in your manifests using the expression <+artifact.image>. See Add Container Images as Artifacts for Kubernetes Deployments.

Click Triggers.

Click New Trigger.

The On New Artifact Trigger options are listed under Artifact.

Each of the Artifact options are described below.

Select the artifact registry where your artifact is hosted. If you artifact is hosted on Docker Hub and you select GCR, you won't be able to set up your Trigger.

Option: Docker Registry Artifacts

In Configuration, in Name, enter a name for the Trigger.

In Listen on New Artifact, click Select Artifact. If this setting is not enabled, it is because the artifact Tag setting is not a Runtime Input (<+input>).

In Select an Artifact Reference, select the artifact for this Trigger to listen for and click Select.

In Configure Artifact Runtime Inputs, provide values for any Runtime inputs you have in your artifact settings (such as the Artifact Server), and click Apply.

Click Continue.

The remaining settings are discussed below.

Option: GCR Artifacts

In Configuration, in Name, enter a name for the Trigger.

In Listen on New Artifact, click Select Artifact. If this setting is not enabled, it is because the artifact Tag setting is not a Runtime Input (<+input>).

In Select an Artifact Reference, select the artifact for this Trigger to listen for and click Select.

In Configure Artifact Runtime Inputs, provide values for any Runtime inputs you have in your artifact settings (such as the Artifact Server), and click Apply.

Click Continue.

The remaining settings are discussed below.

Option: ECR Artifacts

In Configuration, in Name, enter a name for the Trigger.

In Listen on New Artifact, click Select Artifact. If this setting is now enabled, it is because the artifact Tag setting is not a Runtime Input (<+input>).

In Select an Artifact Reference, select the artifact for this Trigger to listen for and click Select.

In Configure Artifact Runtime Inputs, provide values for any Runtime inputs you have in your artifact settings (such as the Artifact Server), and click Apply.

Click Continue.

The remaining settings are discussed below.

Step 2: Set Conditions

In Conditions, enter any conditions that must be matched in order for the Trigger to execute.

Regex and Wildcards

You can use wildcards in the condition's value and you can select Regex.

For example, if the build is todolist-v2.0:

  • With Regex not selected, both todolist* or *olist* will match.
  • With Regex select, the regex todolist-v\d.\d will match.

If the regex expression does not result in a match, Harness ignores the value.

Harness supports standard Java regex. For example, if Regex is enabled and the intent is to match any branch, the wildcard should be .* instead of simply a wildcard *. If you wanted to match all of the files that end in -DEV.tar you would enter .*-DEV\.tar.

Step 3: Select Pipeline Inputs

If your Pipeline uses Input Sets, you can select the Input Set to use when the Trigger executes the Pipeline.

For example, here's an Input Set:

The <+trigger.artifact.build> is used for Tag to ensure that the new artifact version that executed the Trigger is used for the deployment.

If you use a Fixed Value for the artifact Tag (for example, 2), the Trigger will work when a new artifact appears, but it will not select the new artifact. Instead, Harness will select the artifact with the Tag value in the Input Set (2).

The Input Set option is for settings other than the Tag setting of the artifact you want to pull. For example, the Artifact Server.

Step 4: Test Trigger

Once your Trigger is set up, click Create Trigger. The new Trigger is listed.

Once the Pipeline is executed using the Trigger, in Deployments, you can see the Trigger and the user who initiated the deployment.

If you look at the Trigger in your Pipeline again you can see its activation records:

And these records are also in the Trigger details:

You can test the Trigger by pushing a new artifact version to your artifact registry.

You can build and push to your registry using Harness CIE.

Here's a simple CIE Pipeline that uses public source code and pushes it to Docker Hub using Harness CIE. You'll have to replace the <+input> for some settings with your own Connectors and credentials.

CIE Pipeline YAML
pipeline:
name: Build Tweet App
identifier: Build_Tweet_App
projectIdentifier: CD_Examples
orgIdentifier: default
tags: {}
properties:
ci:
codebase:
connectorRef: tweetapp
build: <+input>
stages:
- stage:
name: Build
identifier: Build
description: ""
type: CI
spec:
cloneCodebase: true
infrastructure:
type: KubernetesDirect
spec:
connectorRef: <+input>
namespace: default
execution:
steps:
- step:
type: BuildAndPushDockerRegistry
name: Build and Push
identifier: Build_and_Push
spec:
connectorRef: <+input>
repo: <+input>
tags:
- <+pipeline.sequenceId>
optimize: true
variables: []

Here's a simple CD Pipeline that deploys that artifact to a Kubernetes cluster. It uses public values YAML and Kubernetes manifest files. You'll have to replace the <+input> for some settings with your own Connectors and credentials.

CD Pipeline YAML
pipeline:
name: On New Artifact Tweet
identifier: On_New_Artifact_Tweet
projectIdentifier: CD_Examples
orgIdentifier: default
tags: {}
stages:
- stage:
name: Deploy
identifier: Deploy
description: ""
type: Deployment
spec:
serviceConfig:
serviceRef: tweet
serviceDefinition:
type: Kubernetes
spec:
variables: []
manifests:
- manifest:
identifier: values
type: Values
spec:
store:
type: Github
spec:
connectorRef: <+input>
gitFetchType: Branch
paths:
- default-k8s-manifests/Manifests/Files/ng-values.yaml
branch: main
- manifest:
identifier: manifests
type: K8sManifest
spec:
store:
type: Github
spec:
connectorRef: <+input>
gitFetchType: Branch
paths:
- default-k8s-manifests/Manifests/Files/templates
branch: main
skipResourceVersioning: false
artifacts:
primary:
type: DockerRegistry
spec:
connectorRef: <+input>
imagePath: <+input>
tag: <+input>
infrastructure:
environmentRef: tweet
infrastructureDefinition:
type: KubernetesDirect
spec:
connectorRef: <+input>
namespace: default
releaseName: release-<+INFRA_KEY>
allowSimultaneousDeployments: false
execution:
steps:
- step:
name: Rollout Deployment
identifier: rolloutDeployment
type: K8sRollingDeploy
timeout: 10m
spec:
skipDryRun: false
rollbackSteps:
- step:
name: Rollback Rollout Deployment
identifier: rollbackRolloutDeployment
type: K8sRollingRollback
timeout: 10m
spec: {}
tags: {}
failureStrategies:
- onFailure:
errors:
- AllErrors
action:
type: StageRollback
variables:
- name: name
type: String
value: tweetapp

Finally here's the Trigger in the CD Pipeline that executes the Pipeline when a new artifact version is pushed to the registry.

CD Pipeline Trigger YAML
trigger:
name: tweet trigger Docker Hub
identifier: tweet_trigger
enabled: true
orgIdentifier: default
projectIdentifier: CD_Examples
pipelineIdentifier: On_New_Artifact_Tweet
source:
type: Artifact
spec:
stageIdentifier: Deploy
type: DockerRegistry
spec:
connectorRef: <+input>
imagePath: <+input>
tag: <+trigger.artifact.build>
artifactRef: primary
inputYaml: |
pipeline:
identifier: On_New_Artifact_Tweet
stages:
- stage:
identifier: Deploy
type: Deployment
spec:
serviceConfig:
serviceDefinition:
type: Kubernetes
spec:
artifacts:
primary:
type: DockerRegistry
spec:
connectorRef: <+input>
tag: <+trigger.artifact.build>
imagePath: <+input>
identifier:
properties:
ci:
codebase:
build:
type: branch
spec:
branch: <+trigger.branch>

Option: Enable or Disable Trigger

You can enable or disable Triggers using the Enabled toggle:

Option: Reuse Trigger YAML to Create New Triggers

You can reuse Triggers by copying and pasting Trigger YAML. This can be helpful when you have advanced Conditions you don't want to set up each time.

See Also


Please Provide Feedback