Self-Managed Enterprise Edition - Kubernetes Cluster: Infrastructure Requirements

Updated 3 weeks ago by Michael Cretzman

This document lists the infrastructure requirements for installing Self-Managed Enterprise Edition - Kubernetes Cluster option in an existing Kubernetes cluster.

Production Installation

Self-Managed Enterprise Edition NextGen is installed as an application on an existing Self-Managed Enterprise Edition FirstGen installation.

Below are the requirements for each microservice in each Self-Managed Enterprise Edition installation.

Self-Managed Enterprise Edition FirstGen

Microservice

Pods

CPU / Pod

Memory / Pod

Total CPU

Total Memory

Manager

2

2

4

4

8

Verification

2

1

3

2

6

Machine Learning Engine

1

8

2

8

2

UI

2

0.25

0.25

0.5

0.5

MongoDB

3

4

8

12

24

Proxy

1

0.5

0.5

0.5

0.5

Ingress

2

0.25

0.25

0.5

0.5

TimescaleDB

3

2

8

6

24

KOTS Admin Pods

 

 

 

4

8

Total

 

 

 

37.5

73.5

The KOTS Admin Pods requirements are for a full stack. In an existing cluster, they will likely be lower.

Self-Managed Enterprise Edition NextGen

Microservice

Pods

CPU / Pod

Memory / Pod

Total CPU

Total Memory

Log Minio

1

1

4Gi

1

4Gi

Log service

1

1

3Gi

1

3Gi

SCM

1

0.1

0.5Gi

0.1

0.5Gi

Gateway

2

0.5

3Gi

1

6Gi

NextGen UI

2

0.2

0.2Gi

0.4

0.4Gi

Platform service

2

1

3Gi

2

6Gi

Test Intelligence

2

1

3Gi

2

6Gi

Access Control

2

1

3Gi

2

6Gi

CI Manager

2

1

3Gi

2

6Gi

NextGen Manager

2

2

6Gi

4

12Gi

Pipeline

2

1

6Gi

2

12Gi

Total

19

 

17.5

61.9Gi

Dev Installation

Here are the requirements for each microservice.

Microservice

Pods

CPU / Pod

Memory / Pod

Total CPU

Total Memory

Manager

1

2

4

2

4

Verification

1

1

3

1

3

Machine Learning Engine

1

3

2

3

2

UI

1

0.25

0.25

0.25

0.25

MongoDB

3

2

4

6

12

Proxy

1

0.5

0.5

0.5

0.5

Ingress

1

0.25

0.25

0.25

0.25

TimescaleDB

1

2

8

2

8

Kots Admin Pods

 

 

 

4

8

Total

 

 

 

19

38

Node Specifications: 8 cores vCPU, greater than 12 GB memory.

Storage Requirements

You should have a Kubernetes Storage Class attached to the Kubernetes cluster.

You need to provide the StorageClass name during installation.

Harness uses a total 1000 GB of space distributed as:

  • MongoDB - 3 pods * 200 GB each = 600 GB
  • Timescale DB - 3 pods * 120 GB each = 360 GB
  • Redis: 40 GB total
  • Note: For POC installation, total requirement will be: 200 GB total
  • MongoDB - 3 pods * 50 GB each = 150 GB
  • Timescale DB - 1 pods * 20 GB each = 20 GB
  • Redis: 30 GB total

Whitelist and Outbound Access Requirements

Whitelist the following URLs:

  • kots.io — Kots pulls the latest versions of the kubectl plugin and Kots admin console.
  • app.replicated.com — Kots admin console connects to check for the availability of releases according to your license
  • proxy.replicated.com — Proxy your registry to pull your private images.

Outbound access to the following URLs:

  • proxy.replicated.com​
  • replicated.app
  • k8s.kurl.sh​
  • app.replicated.com
The outbound access is required for a connected install only. If you have opted for Airgap mode, this is not required.

If your cluster does not have direct outbound connectivity and needs a proxy for outbound connections, use these instructions: https://docs.docker.com/network/proxy to set up a proxy on the node machines.

Cluster and Network Architecture

The following diagram illustrates the simple cluster and networking architecture for a Self-Managed Enterprise Edition Kubernetes Cluster setup.

The following sections go into greater detail.

Namespace Requirements

In the examples in all Harness Self-Managed Enterprise Edition documentation, we use namespace named harness.

If you can use a different namespace, make sure that you update any spec samples provided by Harness.

Load Balancer

You need to set up a Load Balancer before installing Harness Self-Managed Enterprise Edition.

During installation, you will provide the Load Balancer URL in the KOTS admin console.

After Harness Self-Managed Enterprise Edition is installed, the load balancer is used to access the Harness Manager UI using a web browser.

Follow the steps on creating the load balancer as part of the process described in Self-Managed Enterprise Edition - Kubernetes Cluster: Setup Guide.

gRPC and Load Balancer Settings

If your load balancer does support HTTP2 over port 443, when you install Harness Self-Managed Enterprise Edition NextGen you will set up gRPC settings:

Enter the following:

  • GRPC Target: enter the load balancer hostname (hostname from the load balancer URL)
  • GRPC Authority: enter manager-grpc-<hostname>. For example: manager-grpc-35.202.197.230.

If your load balancer does not support HTTP2 over port 443 you have two options:

  • If your load balancer supports multiple ports for SSL then add port 9879 in the application load balancer and target port 9879 or node port 32510 on the Ingress controller.
    • GRPC Target: enter the load balancer hostname
    • GRPC Authority: enter the load balancer hostname
  • If your load balancer does not support multiple ports for SSL then create a new load balancer and target port 9879 or node port 32510 on the Ingress controller:
    • GRPC Target: enter the new load balancer hostname
    • GRPC Authority: enter the new load balancer hostname

Trusted Certificate Requirement for Harness Self-Managed Enterprise Edition

All connections to the Harness Manager can be secure or unencrypted according to the URL scheme you use when you configure the Load Balancer URL during installation (https:// or http://):

For secure connections from any integration into the Harness Manager (Github Webhooks, etc), including the Harness Delegate, you must use a publicly trusted certificate.

Harness does not support self-signed certificates for connections to the Harness Manager.

For connections from the Harness Manager outbound to an integration, you can use a self-signed certificate. In this case, you must import the self-signed certificate into Harness Delegate's JRE keystore manually or using a Harness Delegate Profile.

Install Harness Self-Managed Enterprise Edition

Now that you have set up the requirements, proceed with installation in Self-Managed Enterprise Edition - Kubernetes Cluster: Setup Guide.


Please Provide Feedback