KubeOpsDoku – kubeopsctl

Kubeopsctl-2.0: KubeOpsCtl CLI Commands

Mon, 01 Jan 0001 00:00:00 +0000

This documentation shows all commands of the kubeopsctl and how to use them.

⚠ Warning The kosi software must be installed from the start.

General commands

Overview of all KUBEOPSCTL commands

Description:
  kubeopsctl is a kubernetes cluster manager

Usage:
  kubeopsctl [command] [options]

Options:
  --version       Show version information
  -?, -h, --help  Show help and usage information

Commands:
  version  kubeopsctl version information
  login    Login to kubeops hub and registry
  logout   Logout from kubeops hub
  pull     Pull kosi packages for kubernetes cluster setup and plattform tools
  apply    Apply values on kubernetes cluster

Command ‘kubeopsctl –version’

The kubeopsctl --version command shows you the current version of kubeopsctl.

kubeopsctl --version

The output should be:

2.0.3

Command ‘kubeopsctl –help’

The command kubeopsctl --help gives you an overview of all available commands:

kubeopsctl --help

Alternatively, you can also enter kubeopsctl or kubeopsctl -? in the command line.

The command kubeopsctl login performs a login against the KOSI HUB. A valid login session is neccessary to pull the packages.

Description:
  Login to kubeops hub and registry

Usage:
  kubeopsctl login [options]

Options:
  -u, --username <username> (REQUIRED)  Username
  -p, --password <password>             Password
  -?, -h, --help                        Show help and usage information

Example:

kubeopsctl login -u <username> -p <password>

Command ‘kubeopsctl logout’

The command kubeopsctl logout performs a logout from the KOSI HUB.

Description:
  Logout from kubeops hub

Usage:
  kubeopsctl logout [options]

Options:
  -?, -h, --help  Show help and usage information

Example:

kubeopsctl logout

Command ‘kubeopsctl pull’

The command kubeopsctl pull downloads all necessary KOSI packages to the admin node:

Description:
  Pull kosi packages for kubernetes cluster setup and plattform tools

Usage:
  kubeopsctl pull [options]

Options:
  -k, --kubernetesVersion <kubernetesVersion>  Kubernetes version
  -f, --tools <tools>                          Tools values file
  -?, -h, --help                               Show help and usage information

Example:

kubeopsctl pull

If you do not specify a parameter, the latest from kubeopsctl supported Kubernetes version will be pulled.
With parameter --kubernetesVersion 1.30.8 you can pull an older Kubernetes version.

Example:

kubeopsctl pull --kubernetesVersion 1.30.8

⚠ Warning In order to download the KOSI packages, the ADMIN node must have an internet connection to the KubeOps hub
https://dispatcher.kubeops.net/v4/dispatcher/

Command ‘kubeopsctl apply’

The command kubeopsctl apply is used to set up the kubeops platform with a configuration file.

Description:
  Apply values on kubernetes cluster

Usage:
  kubeopsctl apply [options]

Options:
  -f, --file <file> (REQUIRED)  Values files for cluster, tools and user
  -?, -h, --help                Show help and usage information

-f flag

The -f parameter is used to use value parameter yaml-files

Example:

kubeopsctl apply -f cluster-values.yaml

To install your KubeOps Compliance Applications, you have to use a second value file. It is called enterprise-value.yaml

Note Value-parameter will be explained in the references and can be found here.

Example:

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

–delete flag

The -- delete parameter is used to perform a delete action.

This flag delete all nodes which are not present in the cluster-values.yaml - file Example:

kubeopsctl apply --delete -f cluster-values.yaml

Note For a detailed use case click here.

Kubeopsctl-2.0: Joining a Node in a Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

To increase performance or add additional resource capacity to your cluster, adding a node to the cluster is the correct process. This process with kubeopsctl is very easy.
You can use the following steps to join control-plane nodes or worker-nodes to a Kubernetes cluster.

Join Node Process:

Prerequisits

Before performing any action with kubeopsctl, it is recommended to do a login with kosi. Refer to the official KOSI documentation for details here.

ETCD Backup Recommendation

Before performing changes on the control planes, it is recommended to create an ETCD backup. Refer to the official Kubernetes documentation for details here

Example 1: Joining a Control-Plane Node to a Kubernetes Cluster

1. Pull required KOSI packages on your ADMIN

If you do not specify a parameter, the current Kubernetes version 1.32.2 will be pulled.
With parameter --kubernetesVersion 1.34.1 you can pull a specific Kubernetes version.
Available Kubernetes versions are

kubeopsctl pull

2. Add your node definition/specifications in the cluster-values

Note If you want to join a node to your cluster the parameter: changeCluster has set to true

  - name: demo-controlplaneXX
    iPAddress: 10.2.10.XXX
    type: controlplane
    kubeVersion: 1.31.6

Note The parameter kubeVersion defines the kubernetes version in your cluster for the node. This must set to the same version which exists in your cluster.

3. Adjust your cluster-values in zone1
Adjust your cluster-values in comparison to the example below. Be sure to set the actual version in your values, as well as the target version in the nodes. In the snippet below it is just the zone1.

Note Take care of your format an syntax in the cluster-values.yaml

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: false
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.31.6       # -> actual version
kubeVipEnabled: false           
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: https://packagerepo.kubeops.net/
changeCluster: true             # -> has to be set
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-controlplaneXX   # -> has to be changed
    iPAddress: 10.2.10.XXX      # -> has to be changed
    type: controlplane
    kubeVersion: 1.31.6         # -> check with actual version
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.31.6       
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.31.6       
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.31.6      
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.31.6

3. Validate your values and join the node to the cluster

Once the cluster-values.yaml is created, check the values once again. If you are ready just start the join node process with the command:

kubeopsctl apply -f cluster-values.yaml

Example 1: Joining a Worker Node to a Kubernetes Cluster

1. Pull required KOSI packages on your ADMIN

If you do not specify a parameter, the current Kubernetes version 1.32.2 will be pulled.
With parameter --kubernetesVersion x.xx.x you can pull other Kubernetes versions.
Available Kubernetes versions are

kubeopsctl pull

2. Add your node definition/specifications in the cluster-values

Note If you want to join a node to your cluster the parameter: changeCluster has set to true

  - name: demo-workerXX
    iPAddress: 10.2.10.XX
    type: worker
    kubeVersion: 1.31.6

Note The parameter kubeVersion defines the kubernetes version in your cluster for the node. This must set to the same version which exists in your cluster.

3. Adjust your cluster-values in zone2
Adjust your cluster-values in comparison to the example below. Be sure to set the actual version in your values, as well as the target version in the nodes. In the snippet below it is just the zone2.

Note Take care of your format an syntax in the cluster-values.yaml

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: false
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.31.6        # -> actual version
kubeVipEnabled: false           
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: https://packagerepo.kubeops.net/
changeCluster: true              # -> has to be set
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.31.6       
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.31.6       
  - name: demo-workerXX          # -> has to be changed
    iPAddress: 10.2.10.XX        # -> has to be changed
    type: worker
    kubeVersion: 1.31.6          # -> check with actual version     
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.31.6      
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.31.6

3. Validate your values and join the node to the cluster

Once the cluster-values.yaml is created, check the values once again. Before joining the node, note the following:

Note

It is recommended if you join worker nodes to a cluster and rook-ceph is installed, to adjust your rook-ceph CRUSH map. Refer to the official rook-ceph documentation for details here. for updating the crush map, you should set in your enterprise-values.yaml:

...
packages:
- name: rook-ceph
  enabled: true
...

If you are ready, start the join node process with the command:

kubeopsctl apply -f cluster-values.yaml

If you need to apply the CRUSH map adjustment from above, use the following command instead:

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

Kubeopsctl-2.0: KubeOps Versions

Mon, 01 Jan 0001 00:00:00 +0000

Here is the KubeOps and it’s supported tools versions list. Make sure to install or upgrade according to supported versions only.

KubeOps	Supported KOSI Version	Supported kubeopsctl Version	Supported KOSI plugins	Deprecation Date
KubeOps 2.2.X	KOSI 2.15.X	kubeopsctl 2.2.X	enterprise-plugins:2.2.X	01.07.2027
KubeOps 2.1.X	KOSI 2.13.X	kubeopsctl 2.1.X	enterprise-plugins:2.0.X	13.04.2027
KubeOps 2.0.X	KOSI 2.13.X	kubeopsctl 2.0.X	enterprise-plugins:2.0.X	22.12.2026
KubeOps 2.0.X	KOSI 2.13.X	kubeopsctl 2.0.X	enterprise-plugins:2.0.X	22.12.2026

Kubeopsctl: Commands

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps kubeopsctl

This documentation shows all features of kubeopsctl and how to use these features.

The kosi software must be installed from the start.

General commands

Overview of all KUBEOPSCTL commands

Usage:
  kubeopsctl [command] [options]

Options:
  --version       Show version information
  -?, -h, --help  Show help and usage information

Commands:
  apply             Use the apply command to apply a specific config to create or modify the cluster.
  change            change
  drain <argument>  Drain Command.
  uncordon <name>   Uncordon Command.
  upgrade <name>    upgrade Command.
  status <name>     Status Command.

Command ‘kubeopsctl –version’

The kubeopsctl --version command shows you the current version of kubeopsctl.

kubeopsctl --version

The output should be:

0.2.0-Alpha0

Command ‘kubeopsctl –help’

The command kubeopsctl --help gives you an overview of all available commands:

kubeopsctl --help

Alternatively, you can also enter kubeopsctl or kubeopsctl -? in the command line.

Command ‘kubeopsctl pull’

The command kubeopsctl pull downloads all necessary KOSI packages to the admin node:

kubeopsctl pull

If you do not specify a parameter, the current Kubernetes version 1.32.2 will be pulled.
With parameter --kubernetesVersion 1.30.8 you can pull an older Kubernetes version.

kubeopsctl pull --kubernetesVersion 1.30.8

Available Kubernetes versions are 1.30.8, 1.31.6 and 1.32.2.
In order to download the KOSI packages, the ADMIN node must have an internet connection to the KubeOps hub
https://dispatcher.kubeops.net/v4/dispatcher/

Command ‘kubeopsctl apply’

The command kubeopsctl apply is used to set up the kubeops platform with a configuration file.

Example:

kubeopsctl apply -f kubeopsctl.yaml

-f flag

The -f parameter is used to use yaml parameter file.

-l flag

The -l parameter is used to set the log level to a specific value. The default log level is Info. Available log levels are Error, Warning, Info, Debug1, Debug2, Debug3.

Example:

kubeopsctl apply -f kubeopsctl.yaml -l Debug3

Command ‘kubeopsctl change registry’

The command kubeopsctl change registry is used to change the currently used registry to a different one.

Example:

kubeopsctl change registry -f kubeopsctl.yaml -r 10.2.10.11/library -t localhost/library

-f flag

The -f parameter is used to use yaml parameter file.

-r flag

The parameter r is used to pull the docker images which are included in the package to a given local docker registry.

-t flag

The -t parameter is used to tag the images with localhost. For the szenario that the registry of the cluster is exposed to the admin via a network internal domain name, but this name can’t be resolved by the nodes, the flag -t can be used, to use the cluster internal hostname of the registry.

Command ‘kubeopsctl drain’

The command kubeopsctl drain is used to drain a cluster, zone or node.

In this example we are draining a cluster:

kubeopsctl drain cluster/example

In this example we are draining a zone:

kubeopsctl drain zone/zone1

In this example we are draining a node:

kubeopsctl drain node/master1

there can be issues with the draining of nodes if there is rook installed, because rook has pod disruption budgets, so there must be enough nodes ready for rook.

Command ‘kubeopsctl uncordon’

The command kubeopsctl uncordon is used to drain a cluster, zone or node.

In this example we are uncordoning a cluster:

kubeopsctl uncordon cluster/example

In this example we are uncordoning a zone:

kubeopsctl uncordon zone/zone1

In this example we are uncordoning a node:

kubeopsctl uncordon node/master1

Command ‘kubeopsctl upgrade’

The command kubeopsctl upgrade is used to upgrade the kubernetes version of a cluster, zone or node.

In this example we are uncordoning a cluster:

kubeopsctl upgrade cluster/example -v 1.26.6

In this example we are uncordoning a zone:

kubeopsctl upgrade zone/zone1 -v 1.26.6

In this example we are uncordoning a node:

kubeopsctl upgrade node/master1 -v 1.26.6

-v flag

The parameter v is used to set a higher kubernetes version.

Command ‘kubeopsctl status’

The command kubeopsctl status is used to get the status of a cluster.

Example:

kubeopsctl status cluster/cluster1 -v 1.26.6

Kubeopsctl-2.0: Deleting a Node from a Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Deleting a Node from a Kubernetes cluster

Note This guide requires a Compliance enterprise license.

In rare cases, it may be necessary to remove nodes from a Kubernetes cluster. This how-to guide explains the prerequisites and the key considerations to keep in mind before starting the node removal process.

You can use the following steps to delete nodes from a Kubernetes cluster.

Prerequisites

In order to run rook-ceph stable for a longer period your cluster needs at least 3 zones with each zone containing at least 1 worker-node
To check which mon and osd is running on the node you want to delete you can use the command kubectl get pods -n rook-ceph -o wide | grep <node-name> | grep "mon\|osd" | grep -v "osd-prepare" | awk '{print $1}'. As an output you get the mon and the osd running on that node. If you don’t get an output, you don’t have to delete the ressource and can skip to the “delete the node”-section

Worker

Important: Due to rook-ceph, a worker node must not be removed without following the steps below. In this example, worker01 (zone1) is removed from the cluster. Worker01 contains osd.0 and mon-c.

Scale down the rook-ceph-operator deployment to 0

This prevents new MONs or OSDs from being created.

kubectl scale deploy rook-ceph-operator -n rook-ceph --replicas=0

Check which hosts and OSDs belong to each zone

kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph osd tree

ID   CLASS  WEIGHT   TYPE NAME              STATUS  REWEIGHT  PRI-AFF
 -1         0.21478  root default
 -9         0.04880      zone zone1
 -7         0.04880          host worker01                              # worker01 is being removed
  0    ssd  0.04880              osd.0          up   1.00000  1.00000   # osd.0 is being removed
-15         0.04880          host worker04
  3    ssd  0.04880              osd.3          up   1.00000  1.00000
-11         0.10739      zone zone2
 -3         0.05859          host worker02
  1    ssd  0.05859              osd.1          up   0.95001  1.00000
-13         0.05859      zone zone3
 -5         0.05859          host worker03
  2    ssd  0.05859              osd.2          up   0.95001  1.00000

From this output you can see that osd.0 is part of worker01.

Scale down the OSD deployment

kubectl scale deploy -n rook-ceph rook-ceph-osd-<x> --replicas=0
# Example: kubectl scale deploy -n rook-ceph rook-ceph-osd-0 --replicas=0

Remove the OSD via ceph-tools

kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- bash
# show OSD tree
ceph osd tree
# mark OSD out
ceph osd out <x>
# Example: ceph osd out 0
ceph osd purge <x> --yes-i-really-mean-it
# Example: ceph osd purge 0 --yes-i-really-mean-it
ceph auth del osd.<x>
# adjust CRUSH map
ceph osd crush remove <nodename>
# exit from ceph-tools
exit
# show OSD tree (now without the deleted node)
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph osd tree

Check the MON running on the node

Before deleting a MON deployment, check which MON is running on the node you want to remove.

kubectl -n rook-ceph get pods -l app=rook-ceph-mon -o wide

Delete OSD and MON deployments

After checking the OSD and MON running on the node, delete the corresponding deployments.

kubectl delete deploy -n rook-ceph rook-ceph-osd-<x> rook-ceph-mon-<y>

Example

kubectl delete deploy -n rook-ceph rook-ceph-osd-0 rook-ceph-mon-c

Remove the deleted mon from the ceph tools

kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon dump
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon rm <y>
# verify
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon dump

Example

kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon dump
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon rm c
# verfify
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph mon dump

This is the dump before executing the remove:

0: [v2:192.168.231.184:3300/0,v1:192.168.231.184:6789/0] mon.a
1: [v2:192.168.185.9:3300/0,v1:192.168.185.9:6789/0] mon.b
2: [v2:192.168.196.110:3300/0,v1:192.168.196.110:6789/0] mon.c

This is the dump after executing the remove:

0: [v2:192.168.231.184:3300/0,v1:192.168.231.184:6789/0] mon.a
1: [v2:192.168.185.9:3300/0,v1:192.168.185.9:6789/0] mon.b

Delete the node from the kubernetes cluster

Remove the node you want to delete from the cluster-values.yaml file.
After updating the file, apply the deletion with the following command:

kubeopsctl apply --delete -f cluster-values.yaml

Example

The cluster-values.yaml without node1 but with node4

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.31.6      
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true
zones:
- name: zone1
  nodes:
  - name: controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.31.6       
  - name: worker04
    iPAddress: 10.2.10.214
    type: worker
    kubeVersion: 1.31.6       
- name: zone2
  nodes:
  - name: controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.31.6       
  - name: worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.31.6       
- name: zone3
  nodes:
  - name: controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.31.6  
  - name: worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.31.6

After updating the cluster-values.yaml, execute the delete command:

kubeopsctl apply --delete -f cluster-values.yaml

Scale the rook-ceph-operator deployment back to 1

This allows a new MON to be created automatically in zone2.

kubectl scale deploy rook-ceph-operator -n rook-ceph --replicas=1

Timing and health checks

The total duration depends on cluster size and node performance. Before proceeding, verify Ceph health and placement groups are clean.

kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph status
kubectl exec -it deploy/rook-ceph-tools -n rook-ceph -- ceph pg stat

Typical duration ranges from 15 to 120 minutes.

If you want to rejoin the same node, reset it to a time prior to joining the cluster. Only this way you can be sure, that no leftovers from the deletion process remain!

Kubeopsctl: Documentation-kubeopsctl

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps kubeopsctl

this documentation shows all feature of kubeopsctl and how to use these features.

the kosi software must be installed from the start.

General commands

Overview of all KUBEOPSCTL commands

Usage:
  kubeopsctl [command] [options]

Options:
  --version       Show version information
  -?, -h, --help  Show help and usage information

Commands:
  apply             Use the apply command to apply a specific config to create or modify the cluster.
  change            change
  drain <argument>  Drain Command.
  uncordon <name>   Uncordon Command.
  upgrade <name>    upgrade Command.
  status <name>     Status Command.

Command ‘kubeopsctl –version’

The kubeopsctl --version command shows you the current version of kubeopsctl.

kubeopsctl --version

The output should be:

0.2.0-Alpha0

Command ‘kubeopsctl –help’

The command kubeopsctl --help gives you an overview of all available commands:

kubeopsctl --help

Alternatively, you can also enter kubeopsctl or kubeopsctl -? in the command line.

Command ‘kubeopsctl apply’

The command kubeopsctl apply is used to set up the kubeops platform with a configuration file.

Example:

kubeopsctl apply -f kubeopsctl.yaml

-f flag

The -f parameter is used to use yaml parameter file.

-l flag

The -l parameter is used to set the log level to a specific value. The default log level is Info. Available log levels are Error, Warning, Info, Debug1, Debug2, Debug3.

Example:

kubeopsctl apply -f kubeopsctl.yaml -l Debug3

Command ‘kubeopsctl change registry’

The command kubeopsctl change registry is used to change the currently used registry to a different one.

Example:

kubeopsctl change registry -f kubeopsctl.yaml -r 10.2.10.11/library -t localhost/library

-f flag

The -f parameter is used to use yaml parameter file.

-r flag

The parameter r is used to pull the docker images which are included in the package to a given local docker registry.

-t flag

Command ‘kubeopsctl drain’

The command kubeopsctl drain is used to drain a cluster, zone or node.

In this example we are draining a cluster:

kubeopsctl drain cluster/example

In this example we are draining a zone:

kubeopsctl drain zone/zone1

In this example we are draining a node:

kubeopsctl drain node/master1

there can be issues with the draining of nodes if there is rook installed, because rook has pod disruption budgets, so there must be enough nodes ready for rook.

Command ‘kubeopsctl uncordon’

The command kubeopsctl uncordon is used to drain a cluster, zone or node.

In this example we are uncordoning a cluster:

kubeopsctl uncordon cluster/example

In this example we are uncordoning a zone:

kubeopsctl uncordon zone/zone1

In this example we are uncordoning a node:

kubeopsctl uncordon node/master1

Command ‘kubeopsctl upgrade’

The command kubeopsctl upgrade is used to upgrade the kubernetes version of a cluster, zone or node.

In this example we are uncordoning a cluster:

kubeopsctl upgrade cluster/example -v 1.26.6

In this example we are uncordoning a zone:

kubeopsctl upgrade zone/zone1 -v 1.26.6

In this example we are uncordoning a node:

kubeopsctl upgrade node/master1 -v 1.26.6

-v flag

The parameter v is used to set a higher kubernetes version.

Command ‘kubeopsctl status’

The command kubeopsctl status is used to get the status of a cluster.

Example:

kubeopsctl status cluster/cluster1 -v 1.26.6

Prerequisites

Minimum hardware and OS requirments for a linux machine are

OS	Minimum Requirements
Red Hat Enterprise Linux 8	8 CPU cores, 16 GB memory
OpenSUSE 15	8 CPU cores, 16 GB memory

At least one machine should be used as an admin machine for cluster lifecycle management.

Requirements on admin

The following requirements must be fulfilled on the admin machine.

All the utilized users require sudo privileges. If you are using Kubeops as a user, you need a user with sudo rights, so for Opensuse and RHEL 8 Environments it is the wheel group the user should be added to. Make sure that you change your user with:

su -l <user>

Admin machine must be synchronized with the current time.
You need an internet connection to use the default KubeOps registry registry.kubernative.net/kubeops.

A local registry can be used in the Airgap environment. KubeOps only supports secure registries.
It is important to list your registry as an insecure registry in registry.conf (/etc/containers/registries.conf for podman, /etc/docker/deamon.json for docker), in case of insecure registry usage.

Now you can create your own registry instead of using the default. Checkout how to Guide Create a new Repository. for more info.

kosi 2.8.0 must be installed on your machine. Click here to view how it is done in the Quick Start Guide.
it is recommended that runc is uninstalled To uninstall runc on your OS use the following command:
```
dnf remove -y runc
```
```
zypper remove -y runc
```
tc should be installed. To install tc on your OS use the following command:
```
dnf install -y tc
```
```
zypper install -y iproute2
```
for opensearch, the /etc/sysctl.conf should be configured, the line

vm.max_map_count=262144

should be added. also the command

 sysctl -p

should be executed after that.

Podman must be installed on your machine. To install podman on RHEL8 use command.
```
dnf install podman
```
```
zypper install podman
```

Warning

There can be an issue with conflicts with containerd, so it is recommended that containerd.io is removed before installing the podman package.

You must install the kubeops-basic-plugins:0.4.0 .

Simply type in the following command to install the Basic-Plugins.
```
kosi install --hub=public pia/kubeops-basic-plugins:0.4.0
```
Noteable is that you must have to install it on a Root-User Machine.
You must install the kubeops-kubernetes-plugins:0.5.0.

Simply type in the following command to install the Kubernetes-Plugins.
```
kosi install --hub public pia/kubeops-kubernetes-plugins:0.5.0
```

Requirements for each node

The following requirements must be fulfilled on each node.

All the utilized users require sudo privileges. If you are using Kubeops as a user, you need a user with sudo rights, so for Opensuse and RHEL 8 Environments it is the wheel group the user should be added to.
Every machine must be synchronized with the current time.
You have to assign lowercase unique hostnames for every machine you are using.

We recommended using self-explanatory hostnames.

To set the hostname on your machine use the following command:
```
hostnamectl set-hostname <name of node>
```
- Example
  Use the commands below to set the hostnames on each machine as admin, master, node1 node2.
```
hostnamectl set-hostname admin
hostnamectl set-hostname master 
hostnamectl set-hostname node1
hostnamectl set-hostname node2
```
Requires sudo privileges
It is recommended that a dns service is running, or if you don’t have a dns service, you can change the /etc/hosts file. an example for a entry in the /etc/hosts file could be:
```
10.2.10.12 admin
10.2.10.13 master1
10.2.10.14 master2
10.2.10.15 master3
10.2.10.16 node1
10.2.10.17 node2
10.2.10.18 node3
```
To establish an SSH connection between your machines, you either need an SSH key or you need to install sshpass.
1. Generate an SSH key on admin machine using following command
```
ssh-keygen
```
  There will be two keys generated in ~/.ssh directory.
  The first key is the id_rsa(private) and the second key is the id_rsa.pub(public).
2. Copy the ssh key from admin machine to your node machine/s with following command
```
ssh-copy-id <ip address or hostname of your node machine>
```
3. Now try establishing a connection to your node machine/s
```
ssh <ip address or hostname of your node machine>
```

How to Configure Cluster/Nodes/Software using yaml file

you need to have a cluster definition file which describes the different aspects of your cluster. this files describes only one cluster.

Full yaml syntax

Choose the appropriate imagePullRegistry based on your kubeops version.

### General values for registry access ###
imagePullRegistry: "registry.kubernative.net/lima" # mandatory, the registry from which the images for the cluster are pulled
localRegistry: false # mandatory, set to true if you use a local registry, if you want to set the value to true then you also have to set harbor to true

### General values for registry access ###
imagePullRegistry: "registry.kubernative.net/kubeops" # mandatory, the registry from which the images for the cluster are pulled
localRegistry: false # mandatory, set to true if you use a local registry, if you want to set the value to true then you also have to set harbor to true

apiVersion: kubeops/kubeopsctl/alpha/v5 # mandatory
clusterName: "example" # mandatory
clusterUser: "myuser" # mandatory
kubernetesVersion: "1.28.2" # mandatory
#masterHost: optional if you have an hostname, default value in "masterIP"
masterIP: 10.2.10.12 # mandatory
firewall: "nftables" # mandatory, default "nftables"
pluginNetwork: "calico" # mandatory, default "calico"
containerRuntime: "containerd" # mandatory
clusterOS: "Red Hat Enterprise Linux" # mandatory, can be "Red Hat Enterprise Linux" or "openSUSE Leap"
### Additional values for cluster configuration
useInsecureRegistry: false # optional, default is false
ignoreFirewallError: false # optional, default is false
serviceSubnet: 192.168.128.0/17 # optional, default "192.168.128.0/17"
podSubnet: 192.168.0.0/17 # optional, default "192.168.0.0/17"
debug: false # optional, default is false
logLevel: vvvvv # optional, default "vvvvv"
systemCpu: "1" # optional, default "1"
systemMemory: "2G" # optional, default "2G"
sudo: true # optional, default is true
tmpCopyDir: "/tmp" # optional, default is /tmp
createCluster: true # optional, default is true
updateRegistry: true # optional, default is true

zones:
  - name: zone1
    nodes:
      master:
        - name: cluster1master1
          ipAdress: 10.2.10.11
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1master2
          ipAdress: 10.2.10.12
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.14
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1worker2
          ipAdress: 10.2.10.15
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
  - name: zone2
    nodes:
      master:
        - name: cluster1master3
          ipAdress: 10.2.10.13
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: drained
          kubeversion: 1.28.2  
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.16
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2

# set to true if you want to install it into your cluster
rook-ceph: true # mandatory
harbor: true # mandatory
opensearch: true # mandatory
opensearch-dashboards: true # mandatory
logstash: true # mandatory
filebeat: true # mandatory
prometheus: true # mandatory
opa: true # mandatory
kubeops-dashboard: true # mandatory
certman: true # mandatory
ingress: true # mandatory
keycloak: true # mandatory
velero: true # mandatory

nameSpace: "kubeops" #optional, the default value is different for each application
storageClass: "rook-cephfs" # optional, default value is "rook-cephfs"

###Values for Rook-Ceph###
rookValues:
  namespace: kubeops
  cluster:
    spec:
      dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
    storage:
      useAllNodes: true # optional, default value: true
      useAllDevices: true # optional, default value: true
      deviceFilter: "^sd[a-b]" # optional, will only be used if useAllDevices is set to false
      config:
        metadataDevice: "sda" # optional, only set this value, if there is a device available
      nodes: # optional if useAllNodes is set to true, otherwise mandatory
        - name: "<ip-adress of node_1>"
          devices:
            - name: "sdb" 
        - name: "<ip-adress of node_2>"
          deviceFilter: "^sd[a-b]"
          config:
            metadataDevice: "sda" # optional
    resources:
      mgr:
        requests:
          cpu: "500m" # optional, default is 500m, limit: 1000m
          memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
      mon:
        requests:
          cpu: "1" # optional, default is 1, limit: 2000m
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
      osd:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
  operator:
    data:
      rookLogLevel: "DEBUG" # optional, default is DEBUG
#-------------------------------------------------------------------------------------------------------------------------------
### Values for Harbor deployment ###
## For detailed explaination for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ##
harborValues: 
  namespace: kubeops # optional, default is kubeops
  harborpass: "password" # mandatory: set password for harbor access
  databasePassword: "Postgres_Password" # mandatory: set password for database access
  redisPassword: "Redis_Password" # mandatory: set password for redis access
  externalURL: http://10.2.10.11:30002 # mandatory, the ip address and port, from which harbor is accessable outside of the cluster
  nodePort: 30002 # mandatory
  hostname: harbor.local # mandatory
  harborPersistence:
    persistentVolumeClaim:
      registry:
        size: 40Gi # optional, default is 40Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      jobservice:
        jobLog:
          size: 1Gi # optional, default is 1Gi
          storageClass: "rook-cephfs" #optional, default is rook-cephfs
      database:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      redis:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      trivy: 
        size: 5Gi # optional, default is 5Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for filebeat deployment###
filebeatValues:
  namespace: kubeops # optional, default is kubeops   
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Logstash deployment###
##For detailed explaination for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3###
logstashValues:
  namespace: kubeops
  volumeClaimTemplate:
    accessModes: 
      - ReadWriteMany #optional, default is [ReadWriteMany]
    resources:
      requests:
        storage: 1Gi # mandatory, depending on storage capacity
    storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch-Dashboards deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards###
openSearchDashboardValues:
  namespace: kubeops
  nodePort: 30050
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch###
openSearchValues:
  namespace: kubeops
  opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
  resources:
    requests:
      cpu: "250m" # optional, default is 250m
      memory: "1024Mi" # optional, default is 1024Mi
    limits:
      cpu: "300m" # optional, default is 300m
      memory: "3072Mi" # optional, default is 3072Mi
  persistence:
    size: 4Gi # mandatory
    enabled: "true" # optional, default is true
    enableInitChown: "false" # optional, default is false
    labels:
      enabled: "false" # optional, default is false
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    accessModes:
      - "ReadWriteMany" # optional, default is {ReadWriteMany}
  securityConfig:
    enabled: false # optional, default value: false
    ### Additional values can be set, if securityConfig is enabled:
    # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
    # actionGroupsSecret:
    # configSecret:
    # internalUsersSecret: internal-users-config-secret
    # rolesSecret:
    # rolesMappingSecret:
    # tenantsSecret:
    # config:
    #   securityConfigSecret: ""
    #   dataComplete: true
    #   data: {}
  replicas: "3" # optional, default is 3
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Prometheus deployment###
prometheusValues:
  namespace: kubeops # optional, default is kubeops
  privateRegistry: false # optional, default is false
  grafanaUsername: "user" # optional, default is user
  grafanaPassword: "password" # optional, default is password
  grafanaResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 5Gi # optional, default is 5Gi
    nodePort: 30211 # optional, default is 30211

  prometheusResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 25Gi # optional, default is 25Gi
    retention: 10d # optional, default is 10d
    retentionSize: "24GB" # optional, default is 24GB
    nodePort: 32090
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OPA deployment###
opaValues:
  namespace: kubeops

#--------------------------------------------------------------------------------------------------------------------------------
###Values for KubeOps-Dashboard (Headlamp) deployment###
kubeOpsDashboardValues:
  service:
    nodePort: 30007
#--------------------------------------------------------------------------------------------------------------------------------
###Values for cert-manager deployment###
certmanValues:
  namespace: kubeops
  replicaCount: 3
  logLevel: 2
  secretName: root-secret 
#--------------------------------------------------------------------------------------------------------------------------------
###Values for ingress-nginx deployment###
ingressValues:
  namespace: kubeops
keycloakValues:
  namespace: "kubeops" # Optional, default is "keycloak"
  storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
  keycloak:
    auth:
      adminUser: admin # Optional, default is admin
      adminPassword: admin # Optional, default is admin
  postgresql:
    auth:
      postgresUserPassword: "" # Optional, default is ""
      username: bn_keycloak # Optional, default is "bn_keycloak"
      password: "" # Optional, default is ""
      database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
    volumeSize: 8Gi

veleroValues:
  namespace: "velero"
  accessKeyId: "your_s3_storage_username"
  secretAccessKey: "your_s3_storage_password"
  useNodeAgent: false
  defaultVolumesToFsBackup: false
  provider: "aws"
  bucket: "velero"
  useVolumeSnapshots: false
  backupLocationConfig:
    region: "minio"
    s3ForcePathStyle: true
    s3Url: "http://minio.velero.svc:9000"

how to use kubeopsctl

apply changes to cluster

kubeopsctl apply -f kubeopsctl.yaml

Kubeopsctl-2.0: FAQs

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps COMPLIANCE Security Hotfix

What is the issue?

Critical security vulnerabilities in NGINX (Rift and PoolSlip) affect the version used in KubeOps COMPLIANCE. These vulnerabilities may allow remote code execution or denial-of-service attacks.

Who is affected?

KubeOps COMPLIANCE users running affected NGINX versions, including:

KubeOps 2.1.0 KubeOps 2.0.3–2.0.6 KubeOps 1.7.6–1.7.8

What action is required?

Install the provided hotfix as soon as possible.

Note NGINX has reached end of support. KubeOps COMPLIANCE will move to a different ingress controller in a future update. Until then, install this hotfix without delay.

Script Details for KubeOps V2.x.x

Step 1: Load the shell environment & define the KOSI Package.

Load the shell environment from the current user’s bashrc file &

source ~/.bashrc

Set the package name for the Nginx hotfix:

kosinginx="kubeops/ingress-nginx:2.1.0_Beta1"

• For KubeOps 2.1.0 the package name is:
kubeops/ingress-nginx:2.1.0_Beta1

• For KubeOps 2.0.3 to 2.0.6 package name is:
kubeops/ingress-nginx:2.0.6

Create the filename that will be used when the package is pulled:

kosinginxfile="ingress-nginx-$(echo $kosinginx | awk -F: '{print $NF}').tgz"

This creates a file name based on the package version, in this example for KubeOps 2.1.0:

ingress-nginx-2.1.0_Beta1.tgz

Step 2: Prepare the KOSI Directory

Temporarily enable housekeeping in the KOSI configuration, run kosi version, and then disable housekeeping again:
```
sed -i "s/  housekeeping: false/  housekeeping: true/g" $KUBEOPSROOT/kosi/config.yaml
kosi version
sed -i "s/  housekeeping: true/  housekeeping: false/g" $KUBEOPSROOT/kosi/config.yaml
```
This prepares the KOSI directory before the package update.

Step 3: Log In to KOSI

Log in to KOSI with a valid user account. The following loop prompts for a username and retries until the login is successful:

while true; do
    read -rp "$(printf "Username for kosi login: ")" KOSI_USER

    echo "Please enter the password for user '$KOSI_USER'"

    if kosi login -u "$KOSI_USER"; then
        echo "Login successful"
        break
    else
        echo "Login failed. Please try again."
        sleep 1
    fi
done

Step 4: Determine the Harbor Configuration

Retrieve the Harbor namespace, endpoint, port, and password from the KubeOps configuration.

  harborns=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep -m 1 "namespace:" | awk '{print $2}')
  harborep=$(kubectl get cm -n"$harborns" harbor-core -oyaml | grep "EXT_ENDPOINT" | awk '{print $2}')
  harborepohnehttp=$(echo "$harborep" | awk -F"//" '{print $2}')
  harborport=$(echo "$harborepohnehttp" | awk -F: '{print $NF}')
  harborpw=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep "harborpass" | awk '{print $NF}' | sed 's/"//g')

Display the Harbor URL and port:

  echo "Harbor URL : $harborepohnehttp"
  echo "Harbor PORT: $harborport"

Step 5: Log In to Harbor with Podman

if podman login "$harborepohnehttp" -u admin -p "$harborpw" --tls-verify=false; then
    echo "Podman login successful"
else
    echo "Podman login to $harborepohnehttp as admin with password $harborpw was not successful"
    exit 1
fi

Step 6: Pull the Nginx Hotfix Package

Pull the ingress-nginx package from the KOSI hub and save it as the local package archive:

kosi pull --hub kubeops "$kosinginx" -o "$kosinginxfile" -r "$harborepohnehttp/kubeops" -t "localhost:$harborport/kubeops"

Step 7: Determine the Deployment Name for ingress-nginx

Retrieve the deployment name for the existing ingress-nginx package:
```
dname=$(kosi list | grep ingress-nginx | awk '{print $2}')
```
Step 8: Update ingress-nginx

Run the KOSI update command by using the deployment name, the pulled package file, and the KubeOps values file:
```
kosi update --dname="$dname" -p "$kosinginxfile" -f $KUBEOPSROOT/kubeopsctl/tools-values.yaml
```
This applies the hotfix package to the existing ingress-nginx deployment.
Step 9: Remove Local Podman Images

After the update completes, remove the local Podman images related to ingress-nginx and defaultbackend:
```
for i in $(podman images | grep "ingress-nginx\|defaultbackend" | awk '{print $3}'); do
    podman rmi -f $i
done
```
This cleans up the local images from the administration host.

Complete Script for 2.1.0

#!/bin/bash
source ~/.bashrc

# kosi package
kosinginx="kubeops/ingress-nginx:2.1.0_Beta1"

# Filename for kosi pull
kosinginxfile="ingress-nginx-$(echo $kosinginx | awk -F: '{print $NF}').tgz"

# Clean up kosi directory
sed -i "s/  housekeeping: false/  housekeeping: true/g" $KUBEOPSROOT/kosi/config.yaml
kosi version
sed -i "s/  housekeeping: true/  housekeeping: false/g" $KUBEOPSROOT/kosi/config.yaml

# kosi login

while true; do
    read -rp "$(printf "Username for kosi login: ")" KOSI_USER

    echo "Please enter the password for user '$KOSI_USER'"

    if kosi login -u "$KOSI_USER"; then
        echo "Login successful"
        break
    else
        echo "Login failed. Please try again."
        sleep 1
    fi
done

# Determine Harbor values
harborns=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep -m 1 "namespace:" | awk '{print $2}')
harborep=$(kubectl get cm -n"$harborns" harbor-core -oyaml | grep "EXT_ENDPOINT" | awk '{print $2}')
harborepohnehttp=$(echo "$harborep" | awk -F"//" '{print $2}')
harborport=$(echo "$harborepohnehttp" | awk -F: '{print $NF}')
harborpw=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep "harborpass" | awk '{print $NF}' | sed 's/"//g')

echo "Harbor URL : $harborepohnehttp"
echo "Harbor PORT: $harborport"

# Podman login
if podman login "$harborepohnehttp" -u admin -p "$harborpw" --tls-verify=false; then
    echo "Podman login successful"
else
    echo "Podman login to $harborepohnehttp as admin with password $harborpw was not successful"
    exit 1
fi

# kosi pull
kosi pull --hub kubeops "$kosinginx" -o "$kosinginxfile" -r "$harborepohnehttp/kubeops" -t "localhost:$harborport/kubeops"

# Determine dname for ingress-nginx
dname=$(kosi list | grep ingress-nginx | awk '{print $2}')

# kosi update
kosi update --dname="$dname" -p "$kosinginxfile" -f $KUBEOPSROOT/kubeopsctl/tools-values.yaml

# Clean up admin Podman images
for i in $(podman images | grep "ingress-nginx\|defaultbackend" | awk '{print $3}'); do
    podman rmi -f $i
done

Complete Script for 2.0.3-2.0.6

#!/bin/bash
source ~/.bashrc

# kosi package
kosinginx="kubeops/ingress-nginx:2.0.6"

# Filename for kosi pull
kosinginxfile="ingress-nginx-$(echo $kosinginx | awk -F: '{print $NF}').tgz"

# Clean up kosi directory
sed -i "s/  housekeeping: false/  housekeeping: true/g" $KUBEOPSROOT/kosi/config.yaml
kosi version
sed -i "s/  housekeeping: true/  housekeeping: false/g" $KUBEOPSROOT/kosi/config.yaml

# kosi login

while true; do
    read -rp "$(printf "Username for kosi login: ")" KOSI_USER

    echo "Please enter the password for user '$KOSI_USER'"

    if kosi login -u "$KOSI_USER"; then
        echo "Login successful"
        break
    else
        echo "Login failed. Please try again."
        sleep 1
    fi
done

# Determine Harbor values
harborns=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep -m 1 "namespace:" | awk '{print $2}')
harborep=$(kubectl get cm -n"$harborns" harbor-core -oyaml | grep "EXT_ENDPOINT" | awk '{print $2}')
harborepohnehttp=$(echo "$harborep" | awk -F"//" '{print $2}')
harborport=$(echo "$harborepohnehttp" | awk -F: '{print $NF}')
harborpw=$(cat $KUBEOPSROOT/kubeopsctl/tools-values.yaml | grep -A50 "^- name: harbor" | grep -B50 "^- name:" | grep "harborpass" | awk '{print $NF}' | sed 's/"//g')

echo "Harbor URL : $harborepohnehttp"
echo "Harbor PORT: $harborport"

# Podman login
if podman login "$harborepohnehttp" -u admin -p "$harborpw" --tls-verify=false; then
    echo "Podman login successful"
else
    echo "Podman login to $harborepohnehttp as admin with password $harborpw was not successful"
    exit 1
fi

# kosi pull
kosi pull --hub kubeops "$kosinginx" -o "$kosinginxfile" -r "$harborepohnehttp/kubeops" -t "localhost:$harborport/kubeops"

# Determine dname for ingress-nginx
dname=$(kosi list | grep ingress-nginx | awk '{print $2}')

# kosi update
kosi update --dname="$dname" -p "$kosinginxfile" -f $KUBEOPSROOT/kubeopsctl/tools-values.yaml

# Clean up admin Podman images
for i in $(podman images | grep "ingress-nginx\|defaultbackend" | awk '{print $3}'); do
    podman rmi -f $i
done

WireGuard Installation with Local Package Repository

What is the issue?

WireGuard installation may fail when the local package repository is used on unsupported Red Hat Enterprise Linux versions.

Who is affected?

KubeOps COMPLIANCE users installing WireGuard with packagerepo: local.

What action is required?

When using packagerepo: local, ensure that all cluster nodes are running Red Hat Enterprise Linux 9.6.

Note The local package repository is built and tested for RHEL 9.6. Using other RHEL versions may result in package dependency issues during WireGuard installation.

Kubeopsctl-2.0: Fileformats in kubeopsctl

Mon, 01 Jan 0001 00:00:00 +0000

This documentation shows you all the different kind of fileformats kubeopsctl uses and how to use them.

There are currently 2 different files which can be handled by KubeOpsCtl:

cluster-values.yaml

The cluster-values.yaml defines every aspect of the cluster itself. It has no influence over which applications get installed.

apiVersion: kubeops/kubeopsctl/cluster/beta/v1 # required
imagePullRegistry: registry.kubeops.net/kubeops/kubeops # required
airgap: true # optional, default: true
clusterName: myCluster # required 
clusterUser: root # optional, default: root
kubernetesVersion: 1.32.2 # required
kubeVipEnabled: false # optional, default: true
virtualIP: 10.2.10.110 # required
firewall: nftables # optional, default: nftables
pluginNetwork: calico # optional, default: calico | possible alternative: cilium
containerRuntime: containerd # optional, default: containerd
kubeOpsRoot: /var/kubeops # optional, default: /var/kubeops
serviceSubnet: 192.168.128.0/17 # optional, default: 192.168.128.0/17
podSubnet: 192.168.0.0/17 # optional, default: 192.168.0.0/17
debug: false # optional, default: false
systemCpu: 250m # optional, default: 250m
systemMemory: 256Mi # optional, default: 256Mi
packageRepository: local # optional, default: local
changeCluster: true # optional, default: true

advanced:
  localRegistry:
    mountPath: /mnt/registry

zones: # required
- name: zone1 # required
  nodes: # required
  - name: master1 # required
    iPAddress: 10.2.10.110 # required
    type: controlplane # required
    kubeVersion: 1.32.2 # required

Detailed Parameter Information

Key	Possible Values	Additional Info
pluginNetwork	Calico, Cilium

enterprise-values.yaml

The enterprise-values.yaml defines all enterprise applications currently available for you to install in your cluster via kubeopsctl.
You can append multiple of them into a single enterprise-values.yaml as shown in the first example.

For each application you have 2 ways to change its values:

the standard values
the advanced values

While the standard values only cover predefined keys, the advanced values let you change every key available in the helm chart. Keep in mind, that the standard values overwrite the advanced values if both are set.

Each as optional marked line can be skipped unless otherwise stated. If a optional line is skipped, its default value will be used instead. If there is no default value, it can just be omitted and won’t affect the cluster and/or the application

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
  - name: opa-gatekeeper
    enabled: true
    values:
      standard:
        namespace: gatekeeper-system # optional, default is gatekeeper-system
      advanced:
  - name: filebeat-os
    enabled: false
    values:
      standard:
        namespace: logging # optional, default is logging
      advanced:

  ### Values for Rook-Ceph ###
  ### For detailed explanation for each key see: https://artifacthub.io/packages/helm/rook/rook-ceph?modal=values ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: rook-ceph
    enabled: true
    values:
      standard:
        namespace: rook-ceph # optional, default is rook-ceph
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route the ceph dashboard via Traefik
        cluster:
          spec:
            dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
          resources:
            mgr:
              requests:
                cpu: "500m" # optional, default is 500m, limit: 1000m
                memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
            mon:
              requests:
                cpu: "1" # optional, default is 1, limit: 2000m
                memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
            osd:
              requests:
                cpu: "1" # optional, default is 1, limit: 2
                memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
        operator:
          data:
            rookLogLevel: "DEBUG" # optional, default is DEBUG
      advanced: 
        cluster: # All values from https://artifacthub.io/packages/helm/rook/rook-ceph-cluster?modal=values are overwritable
        operator: # All values from https://artifacthub.io/packages/helm/rook/rook-ceph?modal=values are overwritable

  ### Values for Harbor deployment ###
  ### For detailed explanation for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: harbor
    enabled: true
    values:
      standard:
        namespace: harbor # optional, default is harbor
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
        harborpass: "password" # required: set password for harbor access
        databasePassword: "Postgres_Password" # required: set password for database access
        redisPassword: "Redis_Password" # required: set password for redis access
        externalURL: http://10.2.10.11:30002 # required, the ip address and port, from which harbor is accessable outside of the cluster
        nodePort: 30002 # required
        hostname: harbor.local # required
        harborPersistence:
          persistentVolumeClaim:
            registry:
              size: 40Gi # optional, default is 40Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            jobservice:
              jobLog:
                size: 1Gi # optional, default is 1Gi
                storageClass: "rook-cephfs" #optional, default is rook-cephfs
            database:
              size: 1Gi # optional, default is 1Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            redis:
              size: 1Gi # optional, default is 1Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            trivy: 
              size: 5Gi # optional, default is 5Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
      advanced: #  All values from https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration are overwritable

  ### Values for cert-manager deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: cert-manager
    enabled: true
    values:
      standard:
        namespace: cert-manager # optional, default is cert-manager
        replicaCount: 3
        logLevel: 2
        secretName: root-secret
      advanced: # All values from https://artifacthub.io/packages/helm/cert-manager/cert-manager?modal=values are overwritable
      ## add helm values here
      ca: # LetsEncrypt ClusterIssuer / ACME HTTP01 solver settings
        # emailLetsEncrypt: <your_email@domain.com> # default: example@example.com --> must configure
        # ingressName: <ingress_name> # default: nginx; set to "traefik" to solve ACME HTTP01 via Traefik

  ### Values for ingress-nginx deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: ingress-nginx
    enabled: true
    values:
      standard:
        namespace: ingress-nginx # optional, default is ingress
      advanced: # All values from https://artifacthub.io/packages/helm/ingress-nginx/ingress-nginx?modal=values are overwritable

### Values for Traefik deployment ###
apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false 
localRegistry: false
packages:
- name: traefik
  enabled: true
  values:
    standard:
      namespace: traefik # optional default is traefik
    advanced: # All values from https://artifacthub.io/packages/helm/traefik/traefik/39.0.0?modal=values are overwritable

### Values for keycloak deployment ###
apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
- name: keycloak
  enabled: true
  values:
    standard:
      namespace: keycloak # Optional, default is "keycloak"
      ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
      storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
      keycloak:
        auth:
          adminUser: admin # Optional, default is admin
          adminPassword: admin # Optional, default is admin, only used during initial installation
          existingSecret: "" # Optional, default is ""
      postgresql:
        auth:
          postgresPassword: "" # Optional, default is ""
          username: bn_keycloak # Optional, default is "bn_keycloak"
          password: "" # Optional, default is ""
          database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
          existingSecret: "" # Optional, default is ""
    advanced: # All values from https://artifacthub.io/packages/helm/bitnami/keycloak?modal=values are overwritable, except values that are only applied during the installation, such as the Keycloak admin password

keycloak.auth.adminPassword is only applied during the installation. After Keycloak has been installed, changing this value in enterprise-values.yaml does not change the Keycloak password. To change the Keycloak password after installation, change it directly in Keycloak.

  ### Values for filebeat deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: filebeat-os
    enabled: true
    values:
      standard:
        namespace: logging # optional, default is logging   
      advanced: # All values from https://artifacthub.io/packages/helm/elastic/filebeat?modal=values are overwritable

  ### Values for Logstash deployment ###
  ### For detailed explanation for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3 ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: logstash-os
    enabled: true
    values:
      standard:
        namespace: logging # optional, default is logging
        volumeClaimTemplate:
          accessModes: 
            - ReadWriteMany #optional, default is [ReadWriteMany]
          resources:
            requests:
              storage: 1Gi # required, depending on storage capacity
          storageClass: "rook-cephfs" #optional, default is rook-cephfs
      advanced: # All values from https://artifacthub.io/packages/helm/elastic/logstash?modal=values are overwritable

  ### Values for OpenSearch-Dashboards deployment ###
  ### For detailed explanation for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: opensearch-dashboards
    enabled: true
    values:
      standard:
        namespace: logging # optional, default is logging
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
        nodePort: 30050
        hostname: opensearch-dashboard.local # optional, default is opensearch-dashboard.local; external ingress hostname, determines the Keycloak SSO URL
        adminPassword: "Password@@123456" # optional, default is Password@@123456; must match opensearch-os adminPassword
      advanced: # All values from https://artifacthub.io/packages/helm/opensearch-project-helm-charts/opensearch-dashboards?modal=values are overwritable

  ### Values for OpenSearch deployment ###
  ### For detailed explanation for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: opensearch-os
    enabled: true
    values:
      standard:
        namespace: logging # optional, default is logging
        hostname: opensearch-dashboard.local # optional, default is opensearch-dashboard.local; Keycloak host for SSO, must match opensearch-dashboards
        adminPassword: "Password@@123456" # optional, default is Password@@123456; OpenSearch admin password, enforced after deploy
        opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
        resources:
          requests:
            cpu: "250m" # optional, default is 250m
            memory: "1024Mi" # optional, default is 1024Mi
          limits:
            cpu: "300m" # optional, default is 300m
            memory: "3072Mi" # optional, default is 3072Mi
        persistence:
          size: 4Gi # required
          enabled: "true" # optional, default is true
          enableInitChown: "false" # optional, default is false
          labels:
            enabled: "false" # optional, default is false
          storageClass: "rook-cephfs" # optional, default is rook-cephfs
          accessModes:
            - "ReadWriteMany" # optional, default is {ReadWriteMany}
        securityConfig:
          enabled: false # optional, default value: false
          ### Additional values can be set, if securityConfig is enabled:
          # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
          # actionGroupsSecret:
          # configSecret:
          # internalUsersSecret: internal-users-config-secret
          # rolesSecret:
          # rolesMappingSecret:
          # tenantsSecret:
          # config:
          #   securityConfigSecret: ""
          #   dataComplete: true
          #   data: {}
        replicas: "3" # optional, default is 3
      advanced: # All values from https://artifacthub.io/packages/helm/opensearch-project-helm-charts/opensearch?modal=values are overwritable

  ### Values for Prometheus deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: kube-prometheus-stack
    enabled: true
    values:
      standard:
        namespace: monitoring # optional, default is monitoring
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
        privateRegistry: false # optional, default is false
        grafanaUsername: "user" # optional, default is user
        grafanaPassword: "password" # optional, default is password
        grafanaResources:
          storageClass: "rook-cephfs" # optional, default is rook-cephfs
          storage: 5Gi # optional, default is 5Gi
          nodePort: 30211 # optional, default is 30211

        prometheusResources:
          storageClass: "rook-cephfs" # optional, default is rook-cephfs
          storage: 25Gi # optional, default is 25Gi
          retention: 10d # optional, default is 10d
          retentionSize: "24GB" # optional, default is 24GB
          nodePort: 32090
      advanced: # All values from https://artifacthub.io/packages/helm/prometheus-community/prometheus?modal=values-schema are overwritable

  ### Values for OPA deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: opa-gatekeeper
    enabled: true
    values:
      standard:
        namespace: gatekeeper-system # optional, default is gatekeeper-system
      advanced: # All values from https://artifacthub.io/packages/helm/gatekeeper/gatekeeper/3.1.1?modal=values are overwritable

  ### Values for KubeOps-Dashboard (Headlamp) deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: kubeops-dashboard
    enabled: true
    values:
      standard:
        namespace: monitoring # optional, default is monitoring
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
        service:
          nodePort: 30007
      advanced: # All values from https://artifacthub.io/packages/helm/headlamp/headlamp?modal=values are overwritable

  ### Values for velero deployment ###
  apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
  deleteNs: false
  localRegistry: false
  packages:
  - name: velero
    enabled: true
    values:
      standard:
        namespace: velero # Optional, default is "velero"
        accessKeyId: "your_s3_storage_username"
        secretAccessKey: "your_s3_storage_password"
        useNodeAgent: false
        defaultVolumesToFsBackup: false
        provider: "aws"
        bucket: "velero"
        useVolumeSnapshots: false
        backupLocationConfig:
          region: "minio"
          s3ForcePathStyle: true
          s3Url: "http://minio.velero.svc:9000"
      advanced: # All values from https://artifacthub.io/packages/helm/vmware-tanzu/velero?modal=values are overwritable

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
# used in deleteLogstash.sh
deleteNs: false # TODO undocumented values! dangerous if other charts are installed in the namespace :-o
# used in some packages
localRegistry: false # create pull secret for registry 
packages:
- name: kubevirt-manager
  enabled: true
  values:
    standard:
      namespace: kubevirt # optional default is kubevirt
      hostname: kubeops-vms.local
      nodeport: 30733 # optional default is 30733
      ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
    advanced: {}

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
# used in deleteLogstash.sh
deleteNs: false # TODO undocumented values! dangerous if other charts are installed in the namespace :-o
# used in some packages
localRegistry: false # create pull secret for registry 
packages:
- name: kubevirt-manager
  enabled: true
  values:
    standard:
      namespace: kubevirt # optional default is kubevirt
      hostname: kubeops-vms.local
      nodeport: 30733 # optional default is 30733
      ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
    advanced: {}
- name: kubevirt
  enabled: true
  hub: kubeops-vm
  values:
    standard:  
      namespace: kubevirt
      imagePullSecretCred:
        registry: registry.kubeops.net
        user: kubeops
        password: changeme # must be CLI Token from harbor 
      kubevirtobject:
        spec:
          configuration:
            developerConfiguration:
              logVerbosity:
                virtLauncher: 2
                virtHandler: 3
                virtController: 4
                virtAPI: 5
                virtOperator: 6
              featureGates:
              - ExpandDisks
              - CPUManager
              - ExperimentalIgnitionSupport
              - HypervStrictCheck
              - Sidecar
              - HostDevices
              - Snapshot
              - VMExport
              - HotplugVolumes
              - HostDisk
              - ExperimentalVirtiofsSupport
              - DownwardMetricsFeatureGate
              - Root
              - ClusterProfiler
              - WorkloadEncryptionSEV
              - VSOCK
              - DisableCustomSELinuxPolicy
              - KubevirtSeccompProfile
              - DisableMDEVConfiguration
              - PersistentReservation
              - VMPersistentState
              - MultiArchitecture
              - AutoResourceLimitsGate
              - AlignCPUs
              - VolumesUpdateStrategy
              - VolumeMigration
              #- NodeRestriction # Forbidden setting: Do not use!
              - InstancetypeReferencePolicy
              useEmulation: false
      cdiobject:
        spec:
          config:
            insecureRegistries:
            - registry.kubeops.net # Need to test if this is really necessary
            scratchSpaceStorageClass: rook-cephfs # Optional; Default: rook-cephfs

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
- name: rook-ceph
 enabled: true
 values:
   standard:
     namespace: rook-ceph
     cluster:
       resources:
         mgr:
           requests:
             cpu: "500m"
             memory: "512Mi"
         mon:
           requests:
             cpu: "1"
             memory: "1Gi"
         osd:
           requests:
             cpu: "1"
             memory: "1Gi"
       dashboard:
         enabled: true
     operator:
       data:
         rookLogLevel: "DEBUG"
- name: harbor
 enabled: true
 values:
   standard:
     namespace: harbor
     harborpass: "password"
     databasePassword: "password"
     redisPassword: "password"
     externalURL: http://10.2.10.11:30002
     nodePort: 30002
     hostname: harbor.local
     harborPersistence:
       persistentVolumeClaim:
         registry:
           size: 40Gi
           storageClass: "rook-cephfs"
         jobservice:
           jobLog:
             size: 1Gi
             storageClass: "rook-cephfs"
         database:
           size: 1Gi
           storageClass: "rook-cephfs"
         redis:
           size: 1Gi
           storageClass: "rook-cephfs"
         trivy: 
           size: 5Gi
           storageClass: "rook-cephfs"
   advanced:
- name: cert-manager
 enabled: true
 values:
   standard:
     namespace: cert-manager
     replicaCount: 3
     logLevel: 2
     secretName: root-secret
   advanced:
- name: ingress-nginx
 enabled: true
 values:
   standard:
     namespace: ingress-nginx
   advanced:
- name: keycloak
 enabled: true
 values:
   standard:
     namespace: keycloak
     storageClass: "rook-cephfs"
     keycloak:
       auth:
         adminUser: admin
         adminPassword: admin
         existingSecret: ""
     postgresql:
       auth:
         postgresPassword: ""
         username: bn_keycloak
         password: ""
         database: bitnami_keycloak
         existingSecret: ""
   advanced:
- name: filebeat-os
 enabled: true
 values:
   standard:
     namespace: logging
   advanced:
- name: logstash-os
 enabled: true
 values:
   standard:
     namespace: logging
     volumeClaimTemplate:
       accessModes: 
         - ReadWriteMany
       resources:
         requests:
           storage: 1Gi
       storageClass: "rook-cephfs"
   advanced:
- name: opensearch-dashboards
 enabled: true
 values:
   standard:
     namespace: logging
     nodePort: 30050
     hostname: opensearch-dashboard.local
     adminPassword: "Password@@123456"
   advanced:
- name: opensearch-os
 enabled: true
 values:
   standard:
     namespace: logging
     hostname: opensearch-dashboard.local
     adminPassword: "Password@@123456"
     opensearchJavaOpts: "-Xmx512M -Xms512M"
     resources:
       requests:
         cpu: "250m"
         memory: "1024Mi"
       limits:
         cpu: "300m"
         memory: "3072Mi"
     persistence:
       size: 4Gi
       enabled: "true"
       enableInitChown: "false"
       labels:
         enabled: "false"
       storageClass: "rook-cephfs"
       accessModes:
         - "ReadWriteMany"
     securityConfig:
       enabled: false
     replicas: "3"
   advanced:
- name: kube-prometheus-stack
 enabled: true
 values:
   standard:
     namespace: monitoring
     privateRegistry: false
     grafanaUsername: "user"
     grafanaPassword: "password"
     grafanaResources:
       storageClass: "rook-cephfs"
       storage: 5Gi
       nodePort: 30211
     prometheusResources:
       storageClass: "rook-cephfs"
       storage: 25Gi
       retention: 10d
       retentionSize: "24GB"
       nodePort: 32090
   advanced:
- name: opa-gatekeeper
 enabled: true
 values:
   standard:
     namespace: opa-gatekeeper
   advanced:
- name: kubeops-dashboard
 enabled: true
 values:
   standard:
     namespace: monitoring
     service:
       nodePort: 30007
   advanced:
- name: velero
 enabled: true
 values:
   standard:
     namespace: velero
     accessKeyId: "your_s3_storage_username"
     secretAccessKey: "your_s3_storage_password"
     useNodeAgent: false
     defaultVolumesToFsBackup: false
     provider: "aws"
     bucket: "velero"
     useVolumeSnapshots: false
     backupLocationConfig:
       region: "minio"
       s3ForcePathStyle: true
       s3Url: "http://minio.velero.svc:9000"
   advanced:
- name: kubevirt-manager
 enabled: true
 values:
   standard:
     namespace: kubevirt
     hostname: kubeops-vms.local
     nodeport: 30733
     ingressClassName: nginx
   advanced: {}
- name: kubevirt
 enabled: true
 hub: kubeops-vm
 values:
   standard:  
     namespace: kubevirt
     imagePullSecretCred:
       registry: registry.kubeops.net
       user: kubeops
       password: changeme
     kubevirtobject:
       spec:
         configuration:
           developerConfiguration:
             logVerbosity:
               virtLauncher: 2
               virtHandler: 3
               virtController: 4
               virtAPI: 5
               virtOperator: 6
             featureGates:
             - ExpandDisks
             - CPUManager
             - ExperimentalIgnitionSupport
             - HypervStrictCheck
             - Sidecar
             - HostDevices
             - Snapshot
             - VMExport
             - HotplugVolumes
             - HostDisk
             - ExperimentalVirtiofsSupport
             - DownwardMetricsFeatureGate
             - Root
             - ClusterProfiler
             - WorkloadEncryptionSEV
             - VSOCK
             - DisableCustomSELinuxPolicy
             - KubevirtSeccompProfile
             - DisableMDEVConfiguration
             - PersistentReservation
             - VMPersistentState
             - MultiArchitecture
             - AutoResourceLimitsGate
             - AlignCPUs
             - VolumesUpdateStrategy
             - VolumeMigration
             - InstancetypeReferencePolicy
             useEmulation: false
     cdiobject:
       spec:
         config:
           insecureRegistries:
           - registry.kubeops.net
           scratchSpaceStorageClass: rook-cephfs

Kubeopsctl: Fileformats in kubeopsctl

Mon, 01 Jan 0001 00:00:00 +0000

This documentation shows you all the different kind of fileformats kubeopsctl uses and how to use them.

How to configure Cluster/Nodes/Software using kubeopsctl.yaml file

You need to have a cluster definition file which describes the different aspects of your cluster. This files describes only one cluster.

Choose the appropriate imagePullRegistry based on your kubeops version.

### General values for registry access ###
imagePullRegistry: "registry.kubeops.net/kubeops/kubeops" # mandatory, the registry from which the images for the cluster are pulled
airgap: false # mandatory, set to true if you want to pull the images from a local registry, if set airgap to true then you also have to set harbor to true

### General values for registry access ###
imagePullRegistry: "registry.kubeops.net" # mandatory, the registry from which the images for the cluster are pulled
localRegistry: false # mandatory, set to true if you use a local registry, if you want to set the value to true then you also have to set harbor to true

### General values for registry access ###
imagePullRegistry: "registry.kubernative.net/lima" # mandatory, the registry from which the images for the cluster are pulled
localRegistry: false # mandatory, set to true if you use a local registry, if you want to set the value to true then you also have to set harbor to true

Full yaml syntax

apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.32.2
virtualIP: 10.2.10.11
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /var/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true
zones:
- name: zone1
  nodes:
  - name: master1
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.32.2
    user: root
    status: active
    systemCpu: 250m
    systemMemory: 256Mi
  - name: worker1
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.32.2
    user: root 
    status: active
    systemCpu: 250m
    systemMemory: 256Mi
- name: zone2
  nodes:
  - name: master2
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.32.2
    user: root
    status: active
    systemCpu: 250m
    systemMemory: 256Mi
  - name: worker2
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.32.2
    user: root
    status: active
    systemCpu: 250m
    systemMemory: 256Mi
- name: zone3
  nodes:
  - name: master3
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.32.2
    user: root
    status: active
    systemCpu: 250m
    systemMemory: 256Mi
  - name: worker3
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.32.2
    user: root
    status: active
    systemCpu: 250m
    systemMemory: 256Mi

apiVersion: kubeops/kubeopsctl/alpha/v5 # mandatory
clusterName: "example" # mandatory
clusterUser: "myuser" # mandatory
kubernetesVersion: "1.28.2" # mandatory
#masterHost: optional if you have an hostname, default value in "masterIP"
masterIP: 10.2.10.12 # mandatory
firewall: "nftables" # mandatory, default "nftables"
pluginNetwork: "calico" # mandatory, default "calico"
containerRuntime: "containerd" # mandatory
clusterOS: "Red Hat Enterprise Linux" # mandatory, must be "Red Hat Enterprise Linux"
### Additional values for cluster configuration
useInsecureRegistry: false # optional, default is false
ignoreFirewallError: false # optional, default is false
serviceSubnet: 192.168.128.0/17 # optional, default "192.168.128.0/17"
podSubnet: 192.168.0.0/17 # optional, default "192.168.0.0/17"
debug: false # optional, default is false
logLevel: vvvvv # optional, default "vvvvv"
systemCpu: "1" # optional, default "1"
systemMemory: "2G" # optional, default "2G"
sudo: true # optional, default is true
tmpCopyDir: "/tmp" # optional, default is /tmp
createCluster: true # optional, default is true
updateRegistry: true # optional, default is true

zones:
  - name: zone1
    nodes:
      master:
        - name: cluster1master1
          ipAdress: 10.2.10.11
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1master2
          ipAdress: 10.2.10.12
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.14
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1worker2
          ipAdress: 10.2.10.15
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
  - name: zone2
    nodes:
      master:
        - name: cluster1master3
          ipAdress: 10.2.10.13
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: drained
          kubeversion: 1.28.2  
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.16
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2

# set to true if you want to install it into your cluster
rook-ceph: true # mandatory
harbor: true # mandatory
opensearch: true # mandatory
opensearch-dashboards: true # mandatory
logstash: true # mandatory
filebeat: true # mandatory
prometheus: true # mandatory
opa: true # mandatory
kubeops-dashboard: true # mandatory
certman: true # mandatory
ingress: true # mandatory
keycloak: true # mandatory
velero: true # mandatory

nameSpace: "kubeops" #optional, the default value is different for each application
storageClass: "rook-cephfs" # optional, default value is "rook-cephfs"

###Values for Rook-Ceph###
rookValues:
  namespace: kubeops
  cluster:
    spec:
      dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
    storage:
      useAllNodes: true # optional, default value: true
      useAllDevices: true # optional, default value: true
      deviceFilter: "^sd[a-b]" # optional, will only be used if useAllDevices is set to false
      config:
        metadataDevice: "sda" # optional, only set this value, if there is a device available
      nodes: # optional if useAllNodes is set to true, otherwise mandatory
        - name: "<ip-adress of node_1>"
          devices:
            - name: "sdb" 
        - name: "<ip-adress of node_2>"
          deviceFilter: "^sd[a-b]"
          config:
            metadataDevice: "sda" # optional
    resources:
      mgr:
        requests:
          cpu: "500m" # optional, default is 500m, limit: 1000m
          memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
      mon:
        requests:
          cpu: "1" # optional, default is 1, limit: 2000m
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
      osd:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
  operator:
    data:
      rookLogLevel: "DEBUG" # optional, default is DEBUG
#-------------------------------------------------------------------------------------------------------------------------------
### Values for Harbor deployment ###
## For detailed explaination for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ##
harborValues: 
  namespace: kubeops # optional, default is kubeops
  harborpass: "password" # mandatory: set password for harbor access
  databasePassword: "Postgres_Password" # mandatory: set password for database access
  redisPassword: "Redis_Password" # mandatory: set password for redis access
  externalURL: http://10.2.10.11:30002 # mandatory, the ip address and port, from which harbor is accessable outside of the cluster
  nodePort: 30002 # mandatory
  hostname: harbor.local # mandatory
  harborPersistence:
    persistentVolumeClaim:
      registry:
        size: 40Gi # optional, default is 40Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      jobservice:
        jobLog:
          size: 1Gi # optional, default is 1Gi
          storageClass: "rook-cephfs" #optional, default is rook-cephfs
      database:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      redis:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      trivy: 
        size: 5Gi # optional, default is 5Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for filebeat deployment###
filebeatValues:
  namespace: kubeops # optional, default is kubeops   
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Logstash deployment###
##For detailed explaination for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3###
logstashValues:
  namespace: kubeops
  volumeClaimTemplate:
    accessModes: 
      - ReadWriteMany #optional, default is [ReadWriteMany]
    resources:
      requests:
        storage: 1Gi # mandatory, depending on storage capacity
    storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch-Dashboards deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards###
openSearchDashboardValues:
  namespace: kubeops
  nodePort: 30050
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch###
openSearchValues:
  namespace: kubeops
  opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
  resources:
    requests:
      cpu: "250m" # optional, default is 250m
      memory: "1024Mi" # optional, default is 1024Mi
    limits:
      cpu: "300m" # optional, default is 300m
      memory: "3072Mi" # optional, default is 3072Mi
  persistence:
    size: 4Gi # mandatory
    enabled: "true" # optional, default is true
    enableInitChown: "false" # optional, default is false
    labels:
      enabled: "false" # optional, default is false
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    accessModes:
      - "ReadWriteMany" # optional, default is {ReadWriteMany}
  securityConfig:
    enabled: false # optional, default value: false
    ### Additional values can be set, if securityConfig is enabled:
    # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
    # actionGroupsSecret:
    # configSecret:
    # internalUsersSecret: internal-users-config-secret
    # rolesSecret:
    # rolesMappingSecret:
    # tenantsSecret:
    # config:
    #   securityConfigSecret: ""
    #   dataComplete: true
    #   data: {}
  replicas: "3" # optional, default is 3
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Prometheus deployment###
prometheusValues:
  namespace: kubeops # optional, default is kubeops
  privateRegistry: false # optional, default is false
  grafanaUsername: "user" # optional, default is user
  grafanaPassword: "password" # optional, default is password
  grafanaResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 5Gi # optional, default is 5Gi
    nodePort: 30211 # optional, default is 30211

  prometheusResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 25Gi # optional, default is 25Gi
    retention: 10d # optional, default is 10d
    retentionSize: "24GB" # optional, default is 24GB
    nodePort: 32090
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OPA deployment###
opaValues:
  namespace: kubeops

#--------------------------------------------------------------------------------------------------------------------------------
###Values for KubeOps-Dashboard (Headlamp) deployment###
kubeOpsDashboardValues:
  service:
    nodePort: 30007
#--------------------------------------------------------------------------------------------------------------------------------
###Values for cert-manager deployment###
certmanValues:
  namespace: kubeops
  replicaCount: 3
  logLevel: 2
  secretName: root-secret 
#--------------------------------------------------------------------------------------------------------------------------------
###Values for ingress-nginx deployment###
ingressValues:
  namespace: kubeops
keycloakValues:
  namespace: "kubeops" # Optional, default is "keycloak"
  storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
  keycloak:
    auth:
      adminUser: admin # Optional, default is admin
      adminPassword: admin # Optional, default is admin
      existingSecret: "" # Optional, default is ""
  postgresql:
    auth:
      postgresPassword: "" # Optional, default is ""
      username: bn_keycloak # Optional, default is "bn_keycloak"
      password: "" # Optional, default is ""
      database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
      existingSecret: "" # Optional, default is ""
veleroValues:
  namespace: "velero"
  accessKeyId: "your_s3_storage_username"
  secretAccessKey: "your_s3_storage_password"
  useNodeAgent: false
  defaultVolumesToFsBackup: false
  provider: "aws"
  bucket: "velero"
  useVolumeSnapshots: false
  backupLocationConfig:
    region: "minio"
    s3ForcePathStyle: true
    s3Url: "http://minio.velero.svc:9000"

apiVersion: kubeops/kubeopsctl/alpha/v4 # mandatory
clusterName: "example" # mandatory
clusterUser: "myuser" # mandatory
kubernetesVersion: "1.28.2" # mandatory
#masterHost: optional if you have an hostname, default value in "masterIP"
masterIP: 10.2.10.11 # mandatory
firewall: "nftables" # mandatory, default "nftables"
pluginNetwork: "calico" # mandatory, default "calico"
containerRuntime: "containerd" # mandatory
clusterOS: "Red Hat Enterprise Linux" # mandatory, must be "Red Hat Enterprise Linux"
### Additional values for cluster configuration
useInsecureRegistry: false # optional, default is false
ignoreFirewallError: false # optional, default is false
serviceSubnet: 192.168.128.0/17 # optional, default "192.168.128.0/17"
podSubnet: 192.168.0.0/17 # optional, default "192.168.0.0/17"
debug: false # optional, default is false
logLevel: vvvvv # optional, default "vvvvv"
systemCpu: "1" # optional, default "1"
systemMemory: "2G" # optional, default "2G"
sudo: true # optional, default is true
tmpCopyDir: "/tmp" # optional, default is /tmp
createCluster: true # optional, default is true
updateRegistry: true # optional, default is true

zones:
  - name: zone1
    nodes:
      master:
        - name: cluster1master1
          ipAdress: 10.2.10.11
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1master2
          ipAdress: 10.2.10.12
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.14
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1worker2
          ipAdress: 10.2.10.15
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
  - name: zone2
    nodes:
      master:
        - name: cluster1master3
          ipAdress: 10.2.10.13
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: drained
          kubeversion: 1.28.2  
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.16
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2

# set to true if you want to install it into your cluster
rook-ceph: true # mandatory
harbor: true # mandatory
opensearch: true # mandatory
opensearch-dashboards: true # mandatory
logstash: true # mandatory
filebeat: true # mandatory
prometheus: true # mandatory
opa: true # mandatory
kubeops-dashboard: true # mandatory
certman: true # mandatory
ingress: true # mandatory
keycloak: true # mandatory
velero: true # mandatory

nameSpace: "kubeops" #optional, the default value is different for each application
storageClass: "rook-cephfs" # optional, default value is "rook-cephfs"

###Values for Rook-Ceph###
rookValues:
  namespace: kubeops
  cluster:
    spec:
      dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
    storage:
      useAllNodes: true # optional, default value: true
      useAllDevices: true # optional, default value: true
      deviceFilter: "^sd[a-b]" # optional, will only be used if useAllDevices is set to false
      config:
        metadataDevice: "sda" # optional, only set this value, if there is a device available
      nodes: # optional if useAllNodes is set to true, otherwise mandatory
        - name: "<ip-adress of node_1>"
          devices:
            - name: "sdb" 
        - name: "<ip-adress of node_2>"
          deviceFilter: "^sd[a-b]"
          config:
            metadataDevice: "sda" # optional
    resources:
      mgr:
        requests:
          cpu: "500m" # optional, default is 500m, limit: 1000m
          memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
      mon:
        requests:
          cpu: "1" # optional, default is 1, limit: 2000m
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
      osd:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
      cephFileSystems:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 1, limit: 4Gi
      cephObjectStores:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
  operator:
    data:
      rookLogLevel: "DEBUG" # optional, default is DEBUG
#-------------------------------------------------------------------------------------------------------------------------------
### Values for Harbor deployment ###
## For detailed explaination for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ##
harborValues: 
  namespace: kubeops # optional, default is kubeops
  harborpass: "password" # mandatory: set password for harbor access
  databasePassword: "Postgres_Password" # mandatory: set password for database access
  redisPassword: "Redis_Password" # mandatory: set password for redis access
  externalURL: http://10.2.10.11:30002 # mandatory, the ip address and port, from which harbor is accessable outside of the cluster
  nodePort: 30002 # mandatory
  hostname: harbor.local # mandatory
  harborPersistence:
    persistentVolumeClaim:
      registry:
        size: 40Gi # optional, default is 40Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      jobservice:
        jobLog:
          size: 1Gi # optional, default is 1Gi
          storageClass: "rook-cephfs" #optional, default is rook-cephfs
      database:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      redis:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      trivy: 
        size: 5Gi # optional, default is 5Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for filebeat deployment###
filebeatValues:
  namespace: kubeops # optional, default is kubeops   
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Logstash deployment###
##For detailed explaination for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3###
logstashValues:
  namespace: kubeops
  volumeClaimTemplate:
    accessModes: 
      - ReadWriteMany #optional, default is [ReadWriteMany]
    resources:
      requests:
        storage: 1Gi # mandatory, depending on storage capacity
    storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch-Dashboards deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards###
openSearchDashboardValues:
  namespace: kubeops
  nodePort: 30050
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch###
openSearchValues:
  namespace: kubeops
  opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
  resources:
    requests:
      cpu: "250m" # optional, default is 250m
      memory: "1024Mi" # optional, default is 1024Mi
    limits:
      cpu: "300m" # optional, default is 300m
      memory: "3072Mi" # optional, default is 3072Mi
  persistence:
    size: 4Gi # mandatory
    enabled: "true" # optional, default is true
    enableInitChown: "false" # optional, default is false
    labels:
      enabled: "false" # optional, default is false
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    accessModes:
      - "ReadWriteMany" # optional, default is {ReadWriteMany}
  securityConfig:
    enabled: false # optional, default value: false
    ### Additional values can be set, if securityConfig is enabled:
    # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
    # actionGroupsSecret:
    # configSecret:
    # internalUsersSecret: internal-users-config-secret
    # rolesSecret:
    # rolesMappingSecret:
    # tenantsSecret:
    # config:
    #   securityConfigSecret: ""
    #   dataComplete: true
    #   data: {}
  replicas: "3" # optional, default is 3
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Prometheus deployment###
prometheusValues:
  namespace: kubeops # optional, default is kubeops
  privateRegistry: false # optional, default is false
  grafanaUsername: "user" # optional, default is user
  grafanaPassword: "password" # optional, default is password
  grafanaResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 5Gi # optional, default is 5Gi
    nodePort: 30211 # optional, default is 30211

  prometheusResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 25Gi # optional, default is 25Gi
    retention: 10d # optional, default is 10d
    retentionSize: "24GB" # optional, default is 24GB
    nodePort: 32090
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OPA deployment###
opaValues:
  namespace: kubeops

#--------------------------------------------------------------------------------------------------------------------------------
###Values for KubeOps-Dashboard (Headlamp) deployment###
kubeOpsDashboardValues:
  service:
    nodePort: 30007
#--------------------------------------------------------------------------------------------------------------------------------
###Values for cert-manager deployment###
certmanValues:
  namespace: kubeops
  replicaCount: 3
  logLevel: 2
  secretName: root-secret 
#--------------------------------------------------------------------------------------------------------------------------------
###Values for ingress-nginx deployment###
ingressValues:
  namespace: kubeops
keycloakValues:
  namespace: "kubeops" # Optional, default is "keycloak"
  storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
  keycloak:
    auth:
      adminUser: admin # Optional, default is admin
      adminPassword: admin # Optional, default is admin
      existingSecret: "" # Optional, default is ""
  postgresql:
    auth:
      postgresPassword: "" # Optional, default is ""
      username: bn_keycloak # Optional, default is "bn_keycloak"
      password: "" # Optional, default is ""
      database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
      existingSecret: "" # Optional, default is ""
veleroValues:
  namespace: "velero"
  accessKeyId: "your_s3_storage_username"
  secretAccessKey: "your_s3_storage_password"
  useNodeAgent: false
  defaultVolumesToFsBackup: false
  provider: "aws"
  bucket: "velero"
  useVolumeSnapshots: false
  backupLocationConfig:
    region: "minio"
    s3ForcePathStyle: true
    s3Url: "http://minio.velero.svc:9000"

apiVersion: kubeops/kubeopsctl/alpha/v3 # mandatory
imagePullRegistry: "registry.kubeops.net"
clusterName: "example" # mandatory
clusterUser: "myuser" # mandatory
kubernetesVersion: "1.28.2" # mandatory
#masterHost: optional if you have an hostname, default value in "masterIP"
masterIP: 10.2.10.11 # mandatory
firewall: "nftables" # mandatory, default "nftables"
pluginNetwork: "calico" # mandatory, default "calico"
containerRuntime: "containerd" # mandatory
clusterOS: "Red Hat Enterprise Linux" # mandatory, must be "Red Hat Enterprise Linux"
### Additional values for cluster configuration
useInsecureRegistry: false # optional, default is false
ignoreFirewallError: false # optional, default is false
serviceSubnet: 192.168.128.0/17 # optional, default "192.168.128.0/17"
podSubnet: 192.168.0.0/17 # optional, default "192.168.0.0/17"
debug: false # optional, default is false
logLevel: vvvvv # optional, default "vvvvv"
systemCpu: "1" # optional, default "1"
systemMemory: "2G" # optional, default "2G"
sudo: true # optional, default is true
tmpCopyDir: "/tmp" # optional, default is /tmp
createCluster: true # optional, default is true
updateRegistry: true # optional, default is true

zones:
  - name: zone1
    nodes:
      master:
        - name: cluster1master1
          ipAdress: 10.2.10.11
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1master2
          ipAdress: 10.2.10.12
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.14
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1worker2
          ipAdress: 10.2.10.15
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
  - name: zone2
    nodes:
      master:
        - name: cluster1master3
          ipAdress: 10.2.10.13
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: drained
          kubeversion: 1.28.2  
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.16
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2

# set to true if you want to install it into your cluster
rook-ceph: true # mandatory
harbor: true # mandatory
opensearch: true # mandatory
opensearch-dashboards: true # mandatory
logstash: true # mandatory
filebeat: true # mandatory
prometheus: true # mandatory
opa: true # mandatory
headlamp: true # mandatory
certman: true # mandatory
ingress: true # mandatory
keycloak: true # mandatory
velero: true # mandatory

nameSpace: "kubeops" #optional, the default value is different for each application
storageClass: "rook-cephfs" # optional, default value is "rook-cephfs"

###Values for Rook-Ceph###
rookValues:
  namespace: kubeops
  cluster:
    spec:
      dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
    storage:
      useAllNodes: true # optional, default value: true
      useAllDevices: true # optional, default value: true
      deviceFilter: "^sd[a-b]" # optional, will only be used if useAllDevices is set to false
      config:
        metadataDevice: "sda" # optional, only set this value, if there is a device available
      nodes: # optional if useAllNodes is set to true, otherwise mandatory
        - name: "<ip-adress of node_1>"
          devices:
            - name: "sdb" 
        - name: "<ip-adress of node_2>"
          deviceFilter: "^sd[a-b]"
          config:
            metadataDevice: "sda" # optional
    resources:
      mgr:
        requests:
          cpu: "500m" # optional, default is 500m, limit: 1000m
          memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
      mon:
        requests:
          cpu: "1" # optional, default is 1, limit: 2000m
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
      osd:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
      cephFileSystems:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 1, limit: 4Gi
      cephObjectStores:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
  operator:
    data:
      rookLogLevel: "DEBUG" # optional, default is DEBUG
#-------------------------------------------------------------------------------------------------------------------------------
### Values for Harbor deployment ###
## For detailed explaination for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ##
harborValues: 
  namespace: kubeops # optional, default is kubeops
  harborpass: "password" # mandatory: set password for harbor access
  databasePassword: "Postgres_Password" # mandatory: set password for database access
  redisPassword: "Redis_Password" # mandatory: set password for redis access
  externalURL: http://10.2.10.11:30002 # mandatory, the ip address and port, from which harbor is accessable outside of the cluster
  nodePort: 30002 # mandatory
  hostname: harbor.local # mandatory
  harborPersistence:
    persistentVolumeClaim:
      registry:
        size: 40Gi # optional, default is 40Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      jobservice:
        jobLog:
          size: 1Gi # optional, default is 1Gi
          storageClass: "rook-cephfs" #optional, default is rook-cephfs
      database:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      redis:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
      trivy: 
        size: 5Gi # optional, default is 5Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for filebeat deployment###
filebeatValues:
  namespace: kubeops # optional, default is kubeops   
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Logstash deployment###
##For detailed explaination for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3###
logstashValues:
  namespace: kubeops
  volumeClaimTemplate:
    accessModes: 
      - ReadWriteMany #optional, default is [ReadWriteMany]
    resources:
      requests:
        storage: 1Gi # mandatory, depending on storage capacity
    storageClass: "rook-cephfs" #optional, default is rook-cephfs
#--------------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch-Dashboards deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards###
openSearchDashboardValues:
  namespace: kubeops
  nodePort: 30050
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OpenSearch deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch###
openSearchValues:
  namespace: kubeops
  opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
  resources:
    requests:
      cpu: "250m" # optional, default is 250m
      memory: "1024Mi" # optional, default is 1024Mi
    limits:
      cpu: "300m" # optional, default is 300m
      memory: "3072Mi" # optional, default is 3072Mi
  persistence:
    size: 4Gi # mandatory
    enabled: "true" # optional, default is true
    enableInitChown: "false" # optional, default is false
    labels:
      enabled: "false" # optional, default is false
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    accessModes:
      - "ReadWriteMany" # optional, default is {ReadWriteMany}
  securityConfig:
    enabled: false # optional, default value: false
    ### Additional values can be set, if securityConfig is enabled:
    # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
    # actionGroupsSecret:
    # configSecret:
    # internalUsersSecret: internal-users-config-secret
    # rolesSecret:
    # rolesMappingSecret:
    # tenantsSecret:
    # config:
    #   securityConfigSecret: ""
    #   dataComplete: true
    #   data: {}
  replicas: "3" # optional, default is 3
#--------------------------------------------------------------------------------------------------------------------------------
###Values for Prometheus deployment###
prometheusValues:
  namespace: kubeops # optional, default is kubeops
  privateRegistry: false # optional, default is false
  grafanaUsername: "user" # optional, default is user
  grafanaPassword: "password" # optional, default is password
  grafanaResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 5Gi # optional, default is 5Gi
    nodePort: 30211 # optional, default is 30211

  prometheusResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 25Gi # optional, default is 25Gi
    retention: 10d # optional, default is 10d
    retentionSize: "24GB" # optional, default is 24GB
    nodePort: 32090
#--------------------------------------------------------------------------------------------------------------------------------
###Values for OPA deployment###
opaValues:
  namespace: kubeops

#--------------------------------------------------------------------------------------------------------------------------------
###Values for Headlamp deployment###
headlampValues:
  service:
    nodePort: 30007
#--------------------------------------------------------------------------------------------------------------------------------
###Values for cert-manager deployment###
certmanValues:
  namespace: kubeops
  replicaCount: 3
  logLevel: 2
#--------------------------------------------------------------------------------------------------------------------------------
###Values for ingress-nginx deployment###
ingressValues:
  namespace: kubeops
keycloakValues:
  namespace: "kubeops" # Optional, default is "keycloak"
  storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
  keycloak:
    auth:
      adminUser: admin # Optional, default is admin
      adminPassword: admin # Optional, default is admin
      existingSecret: "" # Optional, default is ""
  postgresql:
    auth:
      postgresPassword: "" # Optional, default is ""
      username: bn_keycloak # Optional, default is "bn_keycloak"
      password: "" # Optional, default is ""
      database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
      existingSecret: "" # Optional, default is ""
veleroValues:
  namespace: "velero"
  accessKeyId: "your_s3_storage_username"
  secretAccessKey: "your_s3_storage_password"
  useNodeAgent: false
  defaultVolumesToFsBackup: false
  provider: "aws"
  bucket: "velero"
  useVolumeSnapshots: false
  backupLocationConfig:
    region: "minio"
    s3ForcePathStyle: true
    s3Url: "http://minio.velero.svc:9000"

kubeopsctl.yaml in detail

You can find a more detailed description of the individual parameters here

Cluster creation

apiVersion: kubeops/kubeopsctl/alpha/v5  # mandatory
imagePullRegistry: "registry.kubeops.net" # mandatory
localRegistry: false # mandatory
### Values for setup configuration ###
clusterName: "testkubeopsctl"  # mandatory
clusterUser: "myuser"  # mandatory
kubernetesVersion: "1.28.2" # mandatory, check lima documentation
#masterHost: optional if you have an hostname, default value in "masterIP"
masterIP: 10.2.10.11 # mandatory
firewall: "nftables" # mandatory, default "nftables"
pluginNetwork: "calico" # mandatory, default "nftables"
containerRuntime: "containerd" # mandatory, default "containerd"
clusterOS: "Red Hat Enterprise Linux" # optional, can be "Red Hat Enterprise Linux", remove this line if you want to use default installed OS on admin machine but it has to be "Red Hat Enterprise Linux"

These are parameters for the cluster creation, and software for the clustercreation, p.e. the containerruntime for running the contianers of the cluster. Also there are parameters for the lima software (see documentation of lima for futher explanation).

Network and cluster configuration

### Additional values for cluster configuration
useInsecureRegistry: false # optional, default is false
ignoreFirewallError: false # optional, default is false
serviceSubnet: 192.168.128.0/17 # optional, default "192.168.128.0/17"
podSubnet: 192.168.0.0/17 # optional, default "192.168.0.0/17"
debug: true # optional, default is true
logLevel: vvvvv # optional, default "vvvvv"
systemCpu: "1" # optional, default "1"
systemMemory: "2G" # optional, default "2G"
sudo: true # optional, default is true

Also important are parameters like for the networking like the subnets for the pods and services inside the kubernetes cluster.

Zones

# at least 3 masters and 3 workers are needed
zones:
  - name: zone1
    nodes:
      master:
        - name: cluster1master1
          ipAdress: 10.2.10.11
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1master2
          ipAdress: 10.2.10.12
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.14
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
        - name: cluster1worker1
          ipAdress: 10.2.10.15
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2
  - name: zone2
    nodes:
      master:
        - name: cluster1master3
          ipAdress: 10.2.10.13
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: active
          kubeversion: 1.28.2  
      worker:
        - name: cluster1worker1
          ipAdress: 10.2.10.16
          user: myuser
          systemCpu: 100m
          systemMemory: 100Mi 
          status: drained
          kubeversion: 1.28.2

New are the zones, which contain master and worker nodes. There are two different states: active and drained. Also there can be two different kubernetes versions. So if you want to do updates in tranches, this is possible with kubeopsctl. Also you can set system memory and system cpu of the nodes for kubernetes itself. It is not possible to delete nodes, for deleting nodes you have to use lima. Also if you want to make an update in tranches, you need at least one master with the greater version.

Tools

###Values for Rook-Ceph###
rookValues:
  namespace: kubeops
  cluster:
    spec:
      dataDirHostPath: "/var/lib/rook" # optional, default is /var/lib/rook
    storage:
      useAllNodes: true # optional, default value: true
      useAllDevices: true # optional, default value: true
      deviceFilter: "^sd[a-b]" # optional, will only be used if useAllDevices is set to false
      config:
        metadataDevice: "sda" # optional, only set this value, if there is a device available
      nodes: # optional if useAllNodes is set to true, otherwise mandatory
        - name: "<ip-adress of node_1>"
          devices:
            - name: "sdb" 
        - name: "<ip-adress of node_2>"
          deviceFilter: "^sd[a-b]"
          config:
            metadataDevice: "sda" # optional
    resources:
      mgr:
        requests:
          cpu: "500m" # optional, default is 500m, limit: 1000m
          memory: "512Mi" # optional, default is 1Gi, limit: 1Gi
      mon:
        requests:
          cpu: "1" # optional, default is 1, limit: 2000m
          memory: "1Gi" # optional, default is 1Gi, limit: 2Gi
      osd:
        requests:
          cpu: "1" # optional, default is 1, limit: 2
          memory: "1Gi" # optional, default is 4Gi, limit: 4Gi
  operator:
    data:
      rookLogLevel: "DEBUG" # optional, default is DEBUG

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Rook-Ceph	Text	kubeops	No
cluster.spec.dataDirHostPath	Data directory on the host	Text	/var/lib/rook	Yes
cluster.storage.useAllNodes	Use all nodes	Boolean	true	Yes
cluster.storage.useAllDevices	Use all devices	Boolean	true	Yes
cluster.storage.deviceFilter	Device filter	Regex	^sd[a-b]	Yes
cluster.storage.config.metadataDevice	Metadata device	Text	sda	Yes
cluster.storage.nodes.name	Node name	IP		No
cluster.storage.nodes.devices.name	Device name	Text	sdb	No
cluster.storage.nodes.deviceFilter	Device filter	Regex	^sd[a-b]	Yes
cluster.storage.nodes.config.metadataDevice	Metadata device	Text	sda	Yes
cluster.resources.mgr.requests.cpu	CPU requests for mgr	Text	500m	Yes
cluster.resources.mgr.requests.memory	Memory requests for mgr	Text	512Mi	Yes
cluster.resources.mon.requests.cpu	CPU requests for mon	Text	1	Yes
cluster.resources.mon.requests.memory	Memory requests for mon	Text	1Gi	Yes
cluster.resources.osd.requests.cpu	CPU requests for osd	Text	1	Yes
cluster.resources.osd.requests.memory	Memory requests for osd	Text	1Gi	Yes
operator.data.rookLogLevel	Log level	Text	DEBUG	Yes

### Values for Harbor deployment ###
## For detailed explaination for each key see: https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration ##
harborValues: 
  namespace: kubeops # optional, default is kubeops
  harborpass: "password" # mandatory: set password for harbor access
  databasePassword: "Postgres_Password" # mandatory: set password for database access
  redisPassword: "Redis_Password" # mandatory: set password for redis access
  externalURL: http://10.2.10.11:30002 # mandatory, the ip address and port, from which harbor is accessable outside of the cluster
  nodePort: 30002 # mandatory
  hostname: harbor.local # mandatory
harborPersistence:
  persistentVolumeClaim:
    registry:
      size: 40Gi # optional, default is 40Gi
      storageClass: "rook-cephfs" #optional, default is rook-cephfs
    jobservice:
      jobLog:
        size: 1Gi # optional, default is 1Gi
        storageClass: "rook-cephfs" #optional, default is rook-cephfs
    database:
      size: 1Gi # optional, default is 1Gi
      storageClass: "rook-cephfs" #optional, default is rook-cephfs
    redis:
      size: 1Gi # optional, default is 1Gi
      storageClass: "rook-cephfs" #optional, default is rook-cephfs
    trivy: 
      size: 5Gi # optional, default is 5Gi
      storageClass: "rook-cephfs" #optional, default is rook-cephfs

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Harbor	Text	kubeops	Yes
harborpass	Password for Harbor access	Text		No
databasePassword	Password for database access	Text		No
redisPassword	Password for Redis access	Text		No
externalURL	External URL for Harbor access	URL	http://10.2.10.11:30002	No
nodePort	NodePort for Harbor	Number	30002	No
hostname	Hostname for Harbor	Text	harbor.local	No
harborPersistence.persistentVolumeClaim.registry.size	Storage size for registry	Text	5Gi	No
harborPersistence.persistentVolumeClaim.registry.storageClass	Storage class for registry	Text	rook-cephfs	Yes
harborPersistence.persistentVolumeClaim.jobservice.jobLog.size	Storage size for job logs	Text	1Gi	No
harborPersistence.persistentVolumeClaim.jobservice.jobLog.storageClass	Storage class for job logs	Text	rook-cephfs	Yes
harborPersistence.persistentVolumeClaim.database.size	Storage size for database	Text	1Gi	No
harborPersistence.persistentVolumeClaim.database.storageClass	Storage class for database	Text	rook-cephfs	Yes
harborPersistence.persistentVolumeClaim.redis.size	Storage size for Redis	Text	1Gi	No
harborPersistence.persistentVolumeClaim.redis.storageClass	Storage class for Redis	Text	rook-cephfs	Yes
harborPersistence.persistentVolumeClaim.trivy.size	Storage size for Trivy	Text	5Gi	No
harborPersistence.persistentVolumeClaim.trivy.storageClass	Storage class for Trivy	Text	rook-cephfs	Yes

###Values for filebeat deployment###
filebeatValues:
  namespace: kubeops # optional, default is kubeops

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Filebeat	Text	kubeops	Yes

###Values for Logstash deployment###
##For detailed explaination for each key see: https://github.com/elastic/helm-charts/releases/tag/v7.16.3###
logstashValues:
  namespace: kubeops
  volumeClaimTemplate:
    accessModes: 
      - ReadWriteMany #optional, default is [ReadWriteMany]
    resources:
      requests:
        storage: 1Gi # mandatory, depending on storage capacity
    storageClass: "rook-cephfs" #optional, default is rook-cephfs

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Logstash	Text	kubeops	No
volumeClaimTemplate.accessModes	Access modes for volume claim	List of Texts	[ReadWriteMany]	Yes
volumeClaimTemplate.resources.requests.storage	Storage requests for volume claim	Text	1Gi	No
volumeClaimTemplate.storageClass	Storage class for volume claim	Text	rook-cephfs	Yes

###Values for OpenSearch-Dashboards deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch-dashboards###
openSearchDashboardValues:
  namespace: kubeops
  nodePort: 30050

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for OpenSearch-Dashboards	Text	kubeops	No
nodePort	NodePort for OpenSearch-Dashboards	Number	30050	No

###Values for OpenSearch deployment###
##For detailed explaination for each key see: https://github.com/opensearch-project/helm-charts/tree/main/charts/opensearch###
openSearchValues:
  namespace: kubeops
  opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
  resources:
    requests:
      cpu: "250m" # optional, default is 250m
      memory: "1024Mi" # optional, default is 1024Mi
    limits:
      cpu: "300m" # optional, default is 300m
      memory: "3072Mi" # optional, default is 3072Mi
  persistence:
    size: 4Gi # mandatory
    enabled: "true" # optional, default is true
    enableInitChown: "false" # optional, default is false
    labels:
      enabled: "false" # optional, default is false
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    accessModes:
      - "ReadWriteMany" # optional, default is {ReadWriteMany}
  securityConfig:
    enabled: false # optional, default value: false
    ### Additional values can be set, if securityConfig is enabled:
    # path: "/usr/share/opensearch/plugins/opensearch-security/securityconfig"
    # actionGroupsSecret:
    # configSecret:
    # internalUsersSecret: internal-users-config-secret
    # rolesSecret:
    # rolesMappingSecret:
    # tenantsSecret:
    # config:
    #   securityConfigSecret: ""
    #   dataComplete: true
    #   data: {}
  replicas: "3" # optional, default is 3

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for OpenSearch	Text	kubeops	No
opensearchJavaOpts	Java options for OpenSearch	Text	-Xmx512M -Xms512M	Yes
resources.requests.cpu	CPU requests	Text	250m	Yes
resources.requests.memory	Memory requests	Text	1024Mi	Yes
resources.limits.cpu	CPU limits	Text	300m	Yes
resources.limits.memory	Memory limits	Text	3072Mi	Yes
persistence.size	Storage size for persistent volume	Text	4Gi	No
persistence.enabled	Enable persistent volume	Boolean	true	Yes
persistence.enableInitChown	Enable initial chown for persistent volume	Boolean	false	Yes
persistence.labels.enabled	Enable labels for persistent volume	Boolean	false	Yes
persistence.storageClass	Storage class for persistent volume	Text	rook-cephfs	Yes
persistence.accessModes	Access modes for persistent volume	List of Texts	[ReadWriteMany]	Yes
securityConfig.enabled	Enable security configuration	Boolean	false	Yes
replicas	Number of replicas	Number	3	Yes

###Values for Prometheus deployment###
prometheusValues:
  namespace: kubeops # optional, default is kubeops
  privateRegistry: false # optional, default is false
  grafanaUsername: "user" # optional, default is user
  grafanaPassword: "password" # optional, default is password
  grafanaResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 5Gi # optional, default is 5Gi
    nodePort: 30211 # optional, default is 30211

  prometheusResources:
    storageClass: "rook-cephfs" # optional, default is rook-cephfs
    storage: 25Gi # optional, default is 25Gi
    retention: 10d # optional, default is 10d
    retentionSize: "24GB" # optional, default is 24GB
    nodePort: 32090

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Prometheus	Text	kubeops	Yes
privateRegistry	Use private registry	Boolean	false	Yes
grafanaUsername	Username for Grafana	Text	user	Yes
grafanaPassword	Password for Grafana	Text	password	Yes
grafanaResources.storageClass	Storage class for Grafana	Text	rook-cephfs	Yes
grafanaResources.storage	Storage size for Grafana	Text	5Gi	Yes
grafanaResources.nodePort	NodePort for Grafana	Number	30211	Yes
prometheusResources.storageClass	Storage class for Prometheus	Text	rook-cephfs	Yes
prometheusResources.storage	Storage size for Prometheus	Text	25Gi	Yes
prometheusResources.retention	Retention period for Prometheus	Text	10d	Yes
prometheusResources.retentionSize	Retention size for Prometheus	Text	24GB	Yes
prometheusResources.nodePort	NodePort for Prometheus	Number	32090	No

###Values for OPA deployment###
opaValues:
  namespace: kubeops

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for OPA	Text	kubeops	No

###Values for KubeOps-Dashboard (Headlamp) deployment###
kubeOpsDashboardValues:
  service:
    nodePort: 30007

Parameter Name	Description	Possible Values	Default Value	Optional
service.nodePort	NodePort for KubeOps-Dashboard	Number	30007	No

###Values for cert-manager deployment###
certmanValues:
  namespace: kubeops
  replicaCount: 3
  logLevel: 2
  secretName: root-secret

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Cert-Manager	Text	kubeops	No
replicaCount	Number of replicas	Number	3	No
logLevel	Log level	Number	2	No
secretName	Name of the secret	Text	root-secret	No

###Values for ingress-nginx deployment###
ingressValues:
  namespace: kubeops

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Ingress-Nginx	Text	kubeops	No

keycloakValues:
  namespace: "kubeops" # Optional, default is "keycloak"
  storageClass: "rook-cephfs" # Optional, default is "rook-cephfs"
  keycloak:
    auth:
      adminUser: admin # Optional, default is admin
      adminPassword: admin # Optional, default is admin
      existingSecret: "" # Optional, default is ""
  postgresql:
    auth:
      postgresPassword: "" # Optional, default is ""
      username: bn_keycloak # Optional, default is "bn_keycloak"
      password: "" # Optional, default is ""
      database: bitnami_keycloak # Optional, default is "bitnami_keycloak"
      existingSecret: "" # Optional, default is ""

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Keycloak	Text	kubeops	Yes
storageClass	Storage class for Keycloak	Text	rook-cephfs	Yes
keycloak.auth.adminUser	Admin username	Text	admin	Yes
keycloak.auth.adminPassword	Admin password	Text	admin	Yes
keycloak.auth.existingSecret	Existing secret	Text		Yes
postgresql.auth.postgresPassword	Password for Postgres DB	Text		Yes
postgresql.auth.username	Username for Postgres	Text	bn_keycloak	Yes
postgresql.auth.password	Password for Postgres	Text		Yes
postgresql.auth.database	Database name	Text	bitnami_keycloak	Yes
postgresql.auth.existingSecret	Existing secret for Postgres	Text		Yes

veleroValues:
  namespace: "velero"
  accessKeyId: "your_s3_storage_username"
  secretAccessKey: "your_s3_storage_password"
  useNodeAgent: false
  defaultVolumesToFsBackup: false
  provider: "aws"
  bucket: "velero"
  useVolumeSnapshots: false
  backupLocationConfig:
    region: "minio"
    s3ForcePathStyle: true
    s3Url: "http://minio.velero.svc:9000"

Parameter Name	Description	Possible Values	Default Value	Optional
namespace	Namespace for Velero	Text	velero	No
accessKeyId	Access key ID	Text		No
secretAccessKey	Secret access key	Text		No
useNodeAgent	Use node agent	Boolean	false	No
defaultVolumesToFsBackup	Use default volumes to FS backup	Boolean	false	No
provider	Provider	Text	aws	No
bucket	Bucket name	Text	velero	No
useVolumeSnapshots	Use volume snapshots	Boolean	false	No
backupLocationConfig.region	Region for backup location	Text	minio	No
backupLocationConfig.s3ForcePathStyle	Enforce S3 path style	Boolean	true	No
backupLocationConfig.s3Url	S3 URL for backup location	URL	http://minio.velero.svc:9000	No

Overwrite platform parameters

New with KubeOps 1.6.0 (apiVersion kubeops/kubeopsctl/alpha/v5)

As a user, you can overwrite platform parameters by changing the advanced parameters (e.g. keycloak, velero, harbor, prometheus, etc) to include your desired values. An exception to this is rook-ceph, which has two advancedParameters: one for the rook-operator Helm chart and another for the rook-cluster Helm chart.

...
veleroValues:
  s3ForcePathStyle: false #default would be true
  advancedParameters: 

keycloakValues:
  namespace: "kubeops" #default is "keycloak"
  advancedParameters:

harborValues: 
  namespace: harbor 
  advancedParameters:

prometheusValues: 
  privateRegistry: true
  advancedParameters:

rookValues:
  cluster:
    advancedParameters:
      dataDirHostPath: "/var/lib/rook" # Default ist "/var/lib/rook"
  operator:
    advancedParameters:
      resources:
        limits:
          cpu: "500m" # Default ist "100m"
  

...

Overwriting of list elements

If you want to overwrite paramters inside of a list, then you would need to overwrite the whole list element. If we take the rook helm package as an example:

...
cephBlockPools:
  - name: replicapool
    # see https://github.com/rook/rook/blob/master/Documentation/CRDs/Block-Storage/ceph-block-pool-crd.md#spec for available configuration
    spec:
      failureDomain: host
      replicated:
        size: 3
      # Enables collecting RBD per-image IO statistics by enabling dynamic OSD performance counters. Defaults to false.
      # For reference: https://docs.ceph.com/docs/master/mgr/prometheus/#rbd-io-statistics
      # enableRBDStats: true
    storageClass:
      enabled: true
      name: rook-ceph-block
      isDefault: true
      reclaimPolicy: Retain
      allowVolumeExpansion: true
      volumeBindingMode: "Immediate"
...

If we wanted to change the replicas of the replicapool of the ceph block pools(under spec.replicated.size), then we would also need to overwrite the storageclass and other specs of the replicapool, because list elements can only be overwritten and not merged. if we only use the replicas in the advanced parameters, then the failuredomain would be empty

Kubeopsctl-2.0: Getting-Started

Mon, 01 Jan 0001 00:00:00 +0000

Kubeopsctl: KubeOps Version

Mon, 01 Jan 0001 00:00:00 +0000

Here is the KubeOps and it’s supported tools versions list. Make sure to install or upgrade according to supported versions only.

KubeOps	Supported KOSI Version	Supported LIMA Version	Supported kubeopsctl Version	Deprecation Date
KubeOps 1.7.0	KOSI 2.12.X	LIMA 1.7.X	kubeopsctl 1.7.X	01.10.2026
KubeOps 1.6.0 (deprecated)	KOSI 2.11.X	LIMA 1.6.X	kubeopsctl 1.6.X	01.11.2025

KubeOps 1.7.9

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.18.2	V 1.18.2	kubeops/cert-manager:1.7.9	fc1b3480d344e19765226d80358439221134bef2521bf1576e9abb707c5bc965
clustercreate	V 1.7.9		kubeops/clustercreate:1.7.9	6c14a562ef41e042ed40fa1e3340cef36d6f0af5b519ed090db44b23162b63d3
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.9	6a6c3411c5e20f1ccc48f52ff78125d118a7808ace612fe23f326b9dfae18303
harbor	V 2.14.0	1.18.0	kubeops/harbor:1.7.9	12845b108ddea014ad93c2cfa175a513a96a37426feb9d019daf715ab5857825
helm	V 3.17.4		kubeops/helm:1.7.9	30977f69b9230f1b9a3a86a4ce4417d585dcdafabca3608a88d23b4fc283d38a
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.9	36ec1218235db8217a5bca9f50cf9f69c8cf84d5f6ef5c5e87c0622338b7d0c7
keycloak	V 26.1.0	1.1.0	kubeops/keycloak:1.7.9	d56a52bebc857b1bb873825cb2ef53205d8c751c80f52e3e415da2a38080e2be
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.9	09b51680db3dfe091047d437e5b65b88ef8a427e72b9a0ec15a623856230b0a3
prometheus	V 0.85.0	77.0.2	kubeops/kube-prometheus-stack:1.7.9	b9ec4d2f79f8fd450992deda47fbe53df90621afd46c7cb744d67c94153cd8b4
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.9	54e7ad26e03d8e2566ed5531ede7d066393d2a08bf96c12de368bd9b9f2775fa
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.9	3c2fbc95fee10c817ec68fe3361e92f20d35a1108414f6e8621693cb3e24831c
opensearch-dashboards	V 3.2.0	3.2.2	kubeops/opensearch-dashboards:1.7.9	764c2aa64ee98992045e3dc712205069651ad3bceda73efdef46d82edb45dc17
opensearch	V 3.2.0	3.2.1	kubeops/opensearch-os:1.7.9	a7fa1817de4b5247ad9f43eaf08804c094c1da558fa6ab7699526d7fe68aedae
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.9	42ef6f5a0f5987dc6aa64e2b5381fe9655e1c1f4e9ac2ac390615d61e520826c
setup	V 1.7.9		kubeops/setup:1.7.9	245824ab2f3af8c32acd3e0b76b5839a885cb67164789e3de66360c5f6d40cc7
velero	V 1.16.2	10.1.0	kubeops/velero:1.7.9	c52c1ea5ff53db5eddd29d2b04a21f92dced1edad179276b70a424d4ff9b50df

KubeOps 1.7.8

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.18.2	V 1.18.2	kubeops/cert-manager:1.7.8	569b4b883386083e056d724e877f9196f1cee142efd700d1f6c5a843bb786fbd
clustercreate	V 1.7.8		kubeops/clustercreate:1.7.8	bcf29f2a037f3524578f31c728b4cbf784316a94750c18f9554b8c5ef5553e6e
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.8	ef50f37b20a692ff9617ab512180b8f7e08d4306d6e4e17e85ac7f27fa20385d
harbor	V 2.14.0	1.18.0	kubeops/harbor:1.7.8	cb0954800bd389f2db48a81cb26fd04975555b9d9be904e9776ed93df4fb42b1
helm	V 3.17.4		kubeops/helm:1.7.8	85e30b111f9f283b6b6a16b59ec6ce39fcc74664b8cb288917d37ed5636729a2
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.8	67a021fa82682b0c1e8869c4a69c784c0e2c852dadee09a2e01ba8d6dad92579
keycloak	V 26.1.0	1.1.0	kubeops/keycloak:1.7.8	962b4a2c4215abbcf4a00c8ee941cfacd61c6ec4a07d54e3d840d860c3ccfc91
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.8	2d35bc2d7be166d64ab66940c145278700ec7512b9766a8184457e5e9ad9e2f8
prometheus	V 0.85.0	77.0.2	kubeops/kube-prometheus-stack:1.7.8	94e1197153d123de5a954b97b88b28299484dc2a27190bb1c9826e2e04bad8f0
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.8	6dfa35171d2a5267174ad443dd990bfaef35c07508cf24d86dc6556be1e51eb6
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.8	65d05a43a7235674aef42c4a6f727683008b96088a712239cdad6e58dcaff4cf
opensearch-dashboards	V 3.2.0	3.2.2	kubeops/opensearch-dashboards:1.7.8	99c7b48a692c7aba2d39217269a3fbf55d350a3369bdf5926db237a612b52828
opensearch	V 3.2.0	3.2.1	kubeops/opensearch-os:1.7.8	3089d61b849aaba453c98d0dca25ac87163c5c5525fde3959572c0bdda4352ce
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.8	408aa3078988abed590152f963705d6055af93b4e40ecb3f92daa31bbcea07c1
setup	V 1.7.8		kubeops/setup:1.7.8	3f5777eee05bd91a224e7addc97bd7066ecdccc09afe2cbbe509d2e20630df90
velero	V 1.16.2	10.1.0	kubeops/velero:1.7.8	d17200bf7cbd6d22146e46f6aa2b25201e36aaf9bb547c0dbc2197bf9a2d1526

KubeOps 1.7.7

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.18.2	V 1.18.2	kubeops/cert-manager:1.7.7	abdd3af8ee8d04d2a84ed2d6656d32fd82a25d2e2029a681d1d457b837e27ede
clustercreate	V 1.7.7		kubeops/clustercreate:1.7.7	d036f999501ade09028489b7ca87c7bd9e17204d3375160a2f757bd91f48c2ff
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.7	dbe1d6f745f3d9844997862c77dc73524054d76b3ae753713d0605f5f45597d6
harbor	V 2.14.0	1.18.0	kubeops/harbor:1.7.7	5939f2c6ec5d0b4ec09110ee21116dc1c6848404d7a6e7b773e7ded74096efe4
helm	V 3.17.4		kubeops/helm:1.7.7	1fa24e9f3fed60adc834f77aa80b4ea291f51041529df357d5453d23d724ccfd
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.7	2ad0e43e658ad34426513ef8d6abe7b6b4c0a425aa95aff510486bd630d0a191
keycloak	V 26.1.0	1.1.0	kubeops/keycloak:1.7.7	d57ecbb1911c58cb96525aa21d837e7fc93e4e64ea7c406659709495d4fc52a5
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.7	4fdeacb3e4940c1489eacbe517eb65510d7056a666b09c9dc790f977564773db
prometheus	V 0.85.0	77.0.2	kubeops/kube-prometheus-stack:1.7.7	391f5e5567528a4a1943901991a347ec525b3016de2bdd02b35402b315f3b4d8
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.7	67ce22b1943510205d94f47e4429cebbb1846eccf5d537ced9141c8fca7c0a2f
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.7	d51fcf23e3e47d0c63c558cd24672e0fbae6ff5c7eae2fb16f4787dda117520e
opensearch-dashboards	V 3.2.0	3.2.2	kubeops/opensearch-dashboards:1.7.7	97a815200ebb1e5c0e2b12a87556d5666fdac36dc4881428958df33707027f1f
opensearch	V 3.2.0	3.2.1	kubeops/opensearch-os:1.7.7	6a77717d10a280db3927ed89fff31998f1011e8c30cda763690d85e601bc4b02
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.7	b4a7f4970216d005d74d5ac1329d07826623670283749e0cca2623c51052be6a
setup	V 1.7.7		kubeops/setup:1.7.7	3e21930547d033181e75314cdabab85c41189f1a9366b0cf6a20967c771441e8
velero	V 1.16.2	10.1.0	kubeops/velero:1.7.7	870154d55c2b3c510c742c9e11a2d4642306623845c21b271961c49d59f3aafc

KubeOps 1.7.6

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.18.2	V 1.18.2	kubeops/cert-manager:1.7.6	5b0fbe39b664968c221bacc0415b14349c654f34e827ef4512bcd10ce9d441bf
clustercreate	V 1.7.6		kubeops/clustercreate:1.7.6	31d1bce0d2a4e6578581f47473641143b4909aa0914b023205e6dc44e909a3e6
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.6	4ae67866aa112bed7782d6af9bb95dedc713c2565df77cb664e1e2ae102af627
harbor	V 2.14.0	1.18.0	kubeops/harbor:1.7.6	1ef4de57a9f51249bbafee59d866d433d08b53193ce9dfe631440b6bbf477766
helm	V 3.17.4		kubeops/helm:1.7.6	fecb7c9201898d871fde07c2ea65fc4841a1a03feb24d4b7ec52967d8b10be2f
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.6	393d8c8b2000f06cd3fa7f8576a550038ff4a374ee06f63ee85e017c0c357ba2
keycloak	V 26.1.0	1.1.0	kubeops/keycloak:1.7.6	08d8e9dbc5990b688d1ebabd95da9accea95a1c291ecde6dcae4b76dcd6db1f4
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.6	d1517f0a822d524169c9c1797be66340e76ecb016411d80fa02c716a9b63c8f6
prometheus	V 0.85.0	77.0.2	kubeops/kube-prometheus-stack:1.7.6	55f036c0b67fa45bd81ad34bd47f45c23d847c086a0344d92a6923417f770ce5
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.6	6e870174a9ba66438201a140d2dac815fbae6e2f42edcc7f9cb050c7f8086270
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.6	6012c56e48f99f4fc0d6f5f8ddcac7f4f5d34d60331ad3c58ed80c94f26e92fa
opensearch-dashboards	V 3.2.0	3.2.2	kubeops/opensearch-dashboards:1.7.6	8a1c4f03566d6556c9675852180736184785c589b5ffb82f4167749cd9a6cebf
opensearch	V 3.2.0	3.2.1	kubeops/opensearch-os:1.7.6	1346686257a17d98c94b6390f7270a5530c05d562022a3bb1dc5c01435d19ed1
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.6	4ae7a99b928bd2c1428691c6f9d489d3550e63f7ddc6e86c8b890fd017c46508
setup	V 1.7.6		kubeops/setup:1.7.6	d63007e25e467709b6685e109add9efa7961bef5a3d50926b71095dfd64cfaff
velero	V 1.16.2	10.1.0	kubeops/velero:1.7.6	abeb215f7e367a6c2441eb37b2a80c501540cfc0fe5f1faca49a5d6dda6ef9ac

KubeOps 1.7.4

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.18.2	V 1.18.2	kubeops/cert-manager:1.7.4	03f4ad9f706dc4868effba2ffe6acf93c5b3d88633aea12b5e88138f08bbd7d6
clustercreate	V 1.7.4		kubeops/clustercreate:1.7.4	1446f10653e03552d25d89ad40bf252f06d157ce7c9c2a7d1d2e7ae3ee7dc403
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.4	330fc00b75ed4eb69042311828d35357d3c05549d839e30c93c57933804c558c
harbor	V 2.12.1	1.16.0	kubeops/harbor:1.7.4	42aba751bfe8fd219913cc26b57e38612797bf4433b8e610b26e1f741f0af85a
helm	V 3.14.4		kubeops/helm:1.7.4	1220b8bd78edb98c4a671edf81c2b21a9065f95c03c8ffe76d139a49a79d0760
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.4	c4e44d264328c900e4fb9af17634a21a51baf4a1ff2a16d628ceb91ac7f85413
keycloak	V 1.16.0	1.0.0	kubeops/keycloak:1.7.4	89ce7adde01b71b38079abe6e80d068174616bc8adaf13d994d956a076c12531
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.4	be125adf11b48a99e4f4e35e8d8b411b7dc291cd26894a17242ed11425b1ff67
prometheus	V 0.85.0	77.0.2	kubeops/kube-prometheus-stack:1.7.4	8272deec8d355b81ff14f569d708c63a0e1c259e5a1f88c60dd79db97462a26f
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.4	a71e31260914c3ca3e12e13a3b9be442b27403278c5dd7fd0fdc08e11899fbdc
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.4	d4e84d3f1f743317a6cb276c8831d5da906e3f7dd165f170ec35640b26cd4b3f
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.4	6ee3ec2e029aeb6b5fecb8768bb261b2448f0040a2dd532abc6ac71a501e6cf4
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.4	0cb197d544dd149a4c59e8fa57115c6f9e517a48297ddfb3cc3fec953448ed15
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.4	46d4f3fe3ddbe329abadf9607257bb6e609cbb93158a041690ad794f9a61e097
setup	V 1.7.4		kubeops/setup:1.7.4	bb5ce8025a193f2421704fc47d79d31721b5c57f735967c24bbf48f35a7fb1bd
velero	V 1.16.2	10.1.0	kubeops/velero:1.7.4	b698b4a95d73dda33abad4965776b04cdd0bee1c8b23e9a62854cf18b1ed5d1a

KubeOps 1.7.3

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.7.3	2331dedc11f93a0e4bc572fa9ebb55676d45294f077a9875f2c072fe6350ab89
clustercreate	V 1.7.3		kubeops/clustercreate:1.7.3	fd821f5f55c49890a33923a28c056b02813ed6a3f7c68cc5b30ea0823a9a9e58
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.3	d07f18bf27a65a73bbb43fe0b57a412d41eb19593e93727aa679e65cd8434dea
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.7.3	f5bc8df05268c76ffbc3ae91fb65b3d9a02b75b98096cf863f82777bfbbabc36
helm	V 3.14.4		kubeops/helm:1.7.3	4b372a0827f9496be18003d95af7728bfdf98970468be698f1bddc78a73a60bf
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.3	2683578ff5415669583f94796ecdef91d5629eebba29d27c2a9fe680596afb83
keycloak	V 1.16.0	1.0.0	kubeops/keycloak:1.7.3	8791ea0f8a81ca95f3b102e7462b990282c7b200f7e630b8eb52073eb9c2debc
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.3	b7434c9c88aed3d93c1633ddc1638a910cd801859b2a169203ad9b39698d51e2
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.7.3	2a7751006a4a8f07192f927ab1a5ffc4ee2daba94e25420441099d52a476f899
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.3	bfe89cdcc0f41d30d6dabb3ec5daa62422c9b990be7e353b1a36d61e51b1f40c
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.3	93f3b62c98980ae745cf13b153e47b096e8aecc55643af0477228de946d6f6da
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.3	f4b7519ae1fdc302605d3d36bfffc22f4cd7c1a47ff79f307292077c080efb9f
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.3	5afdbd475e19c355dc87c2dd3340433311f42ac42f3406763d6464cb45c8073d
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.3	a3161e54469313f95f535040430d708bbfd1894bfe53da38681ec1614ad034ac
setup	V 1.7.3		kubeops/setup:1.7.3	fbeb8ce4d68b8b3007baf51cdae2fa16d6ed8bd469cca253c96c5114c21fbd9f
velero	V 1.13.2	6.4.0	kubeops/velero:1.7.3	a0f07a1556fec1b5c54019526685e140d0e251a27f5962bd080ff32ece6a195b

KubeOps 1.7.2

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.7.2	b09f99a398f7c3f6e38aac9d7bf9dab1aecd4449fe7526dca98a6f027760d800
clustercreate	V 1.7.2		kubeops/clustercreate:1.7.2	83f048cfdb47ba014fb85667ad870bd8f2136b57a95cb51dede19732dfaec39e
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.2	ba3fa94e58757fe8ba6bebba20d187258be372dd6c2b9b41b44052cc6beeb649
harbor	V 2.12.1	1.16.0	kubeops/harbor:1.7.2	49225fe6c0dafa87656c128db1d01c132a8ea9510bc977bda47342046356adb4
helm	V 3.14.4		kubeops/helm:1.7.2	f5a712d5f506f4f8694b4c8f8b3a83ed1cf42d1472b86cc4f19b06d33f3f9f1d
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.2	75d8760f8ead58cf1bf02bde7adbdbf886cf61e5da5a6269100bdb46e8b29067
keycloak	V 1.16.0	1.0.0	kubeops/keycloak:1.7.2	e918e2070c6cfc1b5e39ff785ffbee2a4e6cba86b5d3c79ffc2ad4cd28c9a7c0
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.2	1d3dc4cd3e5fd7dc91d1f34544c7d32e79c256401d6a4bbfa5b0acae35a04401
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.7.2	9dec3864de5d8109d9eb9aa5deb081936a7fa995b8408297526bf4587716bd59
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.2	272e3381848c28ea6db54cf1b6420e7561e6168533e72eb78976614007e96eb3
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.2	991130a0fe4804664607c1c38d3ec5191af3c1a590aa87be4df3d3dd0f40a389
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.2	0437d6c5c837816aa680549a69816442a8de80e56f3cc44f57b5654eb934128a
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.2	4b07633ab55e64f6dc27a2723ae4ce30f0d013f3475ef190e8688bc761a9298b
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.2	e176bfaaf49499151ac1ff3f0bd2d65e362cfedb4f4e5c9aede146bc79b5fa1a
setup	V 1.7.2		kubeops/setup:1.7.2	8e58fa8306e845a8c40ece395773501a9babb39ae25ef7c83cc16e5b2a7ebd23
velero	V 1.13.2	6.4.0	kubeops/velero:1.7.2	be2f76d75406b618414106f04f39c5f6ee19010a955106fbe101c36c043e9df4

KubeOps 1.7.1

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.7.1	92cd83afaa5485d5cd7364747737cca9e9517ddc2af3a4935fe0e6c510c8baf2
clustercreate	V 1.7.1		kubeops/clustercreate:1.7.1	07efdacc1538d58790546309dd45a4f4fbfbe51e6d3efebd72e51418ed3b458b
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.1	32f80b86373fb975b8e2a3d82ac961a757c3de39092e144f98e04a7ee268a795
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.7.1	358bbaac307fb93cdf4966c5bd7c0ec0d26d93fcacf921f5f045f3b9f53f6eff
helm	V 3.17.4		kubeops/helm:1.7.1	2d764ce17e937ea6dea82ca91dc33b08f7f479dfa5b3ab3ea206887e9e699a5e
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.1	1ae700de128698ff4ac2b5fa3de7aff8ec6f5177215bcf5d3a974f151e778284
keycloak	V 22.0.1	1.0.0	kubeops/keycloak:1.7.1	d948ab9bb7a299bc283f49875e733f8cd2f0901e8f029c7920971c7dc320eda9
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.1	6b3e5b42ca6104825baf2530d804a06aa2f1550c933dfa7a701f6148a1cbb2fa
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.7.1	10a72a258a0adc524798ec025416ccd7a9d5c0ccad0e68d13cd24f1fdcbace9d
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.1	009966ba060492a40363b734796e3d984ad0046179b69dbffda1625c108a421c
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.1	346d5c8eb819d1ba7d1b289d84a3b0041fc9ca8f7a487813708368241aea1b23
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.1	a622817541e0bf8c506503bfab41045a7aca7dcc745641f2b7fcf888fd9e0ee3
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.1	e42a89dbf5fd71e6f1cb4ecbbf7747eaf502b4589e634ec07204da64e18700ef
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.1	f8fa712dad2cce3585f717e0ad1ce6fda081152fe46b655703a084405cf9db87
setup	V 1.7.1		kubeops/setup:1.7.1	e77fd2c0841fe3f2ad5d455053c0ece8d1287ade85d738a6fcad148a9a88ce61
velero	V 1.13.2	6.4.0	kubeops/velero:1.7.1	28022717f900ee4c2a9994d3685589ced5ee69c08faa9f54d00bc25db94de4e6

KubeOps 1.7.0

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.7.0	f4622bd0d13ec000f06ceac3bd6637dda5fb165528549aa2c392cc36ecfabd71
clustercreate	V 1.7.0		kubeops/clustercreate:1.7.0	d6e808632cfdb6d1c037ff7b9bc0c2b4fa7bdcc9a8a67ae6c5ed4fb0980308e5
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.0	18b83d6a3355e9ef27ea34f5083dc13c73167c51f9beb9d2099c8544be564f3a
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.7.0	577f8b9f68d72a6da1a22726e423cc484fa39fa674d0dfd6531ac47a588d31c0
helm	V 3.14.4		kubeops/helm:1.7.0	7cf5601288d239969e93f8ec44340505bb39474d19cfaeeb74a9c9a66d66adc5
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.0	d0e09bf32a80c4958efccd4c6fb6f69295becdfdc9d842eb81e78a9ed0cbc174
keycloak	V 26.1.0	1.0.0	kubeops/keycloak:1.7.0	3ba087ba481469afef28fea896b600ceab27b85f13235716eb82407a6b6ed057
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.0	2a0b442ef2fce5b0ef3662a6c9bc679717feacfcbb316f8f9234264e5e45e691
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.7.0	f1a8f8324216c693d143981e2171d2cf6b5978ac537e496e09c52d4773b1f91c
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.0	111f96e952f3f3010bf154d5c2e7040ee7c48ef9ff01338f9f8bf39e7276210b
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.0	751a5fa0d3382026f4c7229e3bc4535cfd6be16c4da13fc85807283eff414acc
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.0	8c9457b2c08339c1dcf253266eb07387cb0dd73e99ffc66bd0d301bfecf095a9
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.0	06327e6404ec47bc80dce1e0e2bd0511b8d4f9795d4fa70c12689769826c8425
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.0	bd7c5b920cb309d17e5fdaf50408f9f865edba2bb7566ff7e5fb732464d50b65
setup	V 1.7.0		kubeops/setup:1.7.0	580d0e079f981028a5bed0025826c6c09653a827cb7672f4eab9c98d0924d573
velero	V 1.13.2	6.4.0	kubeops/velero:1.7.0	7eaf98c79a06c6ce69e46f8e855de0c309f74460c3fe7e2ff19bea7e84334cff

KubeOps 1.6.11

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		lima/calicomultus:0.0.5	18d458d6bda62efb37f6e07378bb90a8cee824286749c42154815fae01a10b62
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.11	5e752e68416366352255c6cf53b2e90d2a8363450d2134ce16f2aa9e27207319
clustercreate	V 1.6.11		kubeops/clustercreate:1.6.11	ff225e95ff454585e612fd8555ebfb2442d3fae0d3ceff3be3f371b384404744
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.11	88c8718d427ad9ed79c1dcd921a4aad4cf9a5e2c8e0a151f67987bdd7b3a48de
harbor	V 2.12.1	1.16.0	kubeops/harbor:1.6.11	74edf7239f8b68e8560106febc784a1fff90f2a16e59b79c619215ba02635acf
helm	V 3.14.4		kubeops/helm:1.6.11	3cf43042a405828b589b50618a08f10d8bb70c9251822a629527f74abdf14d1b
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.11	50f9e81d5701a04c2b0f913c1ef8afd2026544ad160c00bc07e224e48adae0e6
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.11	78e0d20ac1ef702bf2069765c5864059db9cd8ab2ab03525611fb0c2e5f81c69
kubeops-dashboard	V 0.15.1	0.11.0	kubeops/kubeops-dashboard:1.6.11	bd8b59206243617af3aa8b098de4bcb9bfae67e1c04c86de59198365feb1c49d
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.11	15d25ae29bb860db4c91bb610315a8ab3b41a412301c9c2c19792d38f6c11af2
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.11	33d7619a6beecbc75a60c9187f3a9fc0962570ae4f5cf7d9446e2c631096bf6a
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.11	7abe283da02dae8501b469c7d3ffdcfb5e49117ce0f009a771fd1a6253dbf4d0
opensearch-dashboards	V 2.22.0	2.16.0	kubeops/opensearch-dashboards:1.6.11	042afee3696c448a2df3a00237c1745186ed5c3c4b2ee0e875d0bd59e7c47969
opensearch	V 2.23.1	2.16.0	kubeops/opensearch-os:1.6.11	1fec7b815e2d28a93d4f5610874f2efbb9e6cb7cb3d816528fb350f7802c1ee3
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.11	fef95c9ffad62e362ed02b9c1f3b8e4991866f249603c2a210e39a20c029b086
setup	V 1.6.11		kubeops/setup:1.6.11	90ec2bada0f2ec3ec11cd74a2a15d05d42170e501927697521531bc2e0c05d83
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.11	8e274df5e8c1102df057f501575cae845e70e2a2139bda12e5cdc90b1492e4dd

KubeOps 1.6.10

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		lima/calicomultus:0.0.5	18d458d6bda62efb37f6e07378bb90a8cee824286749c42154815fae01a10b62
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.10	ae98997fd4d019d8d4b42d8c1fec6278453fdd3972784af5c70b0e579fe912dd
clustercreate	V 1.6.10		kubeops/clustercreate:1.6.10	5a0771102a988b34c2425ca43810ae997199871057194d66712e99634278bebf
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.10	8b5d8b3c65b9005df0b1552664c3f0a30672653fc5ee0dea3ae79c9c7f044251
harbor	V 2.12.1	1.16.0	kubeops/harbor:1.6.10	95bb5942753dbf89b4d91a9b5aa0018687048d8dd445e7a4ef4306d4d779a382
helm	V 3.14.4		kubeops/helm:1.6.10	01b6561822ed290201fbe1d5332bac8525f608f3533ce0a81c597189a817d7e7
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.10	8d90c37a64517eae0aa33184f1b7698326cd27c68ff23be62c722cd23916992a
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.10	adea76709081f854dd3ee29361221a19cf9ebed28b9ed543174dabf730e70b0e
kubeops-dashboard	V 0.15.1	0.11.0	kubeops/kubeops-dashboard:1.6.10	d3a58b5af10be3aec5d063c18f4299ce2f2703baae0c9a67f1da57eef95f1da8
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.10	81d9769d04ae60202ecf30ddcf6f0365e1fa080f4b8776075c163cc96e798af4
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.10	1086234dce328edbef6d14dcb5d1eb0f1f5fd32657497ae3718114bb5c03f85d
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.10	38852283d48118d6090dbd2a09572bbf33c022207a409a2694e386a56f0c2fe1
opensearch-dashboards	V 2.22.0	2.16.0	kubeops/opensearch-dashboards:1.6.10	24d8a5e2778aeb163fed3f4a36190ae2ee41544cdbdb54c96e638bca605cc85b
opensearch	V 2.23.1	2.16.0	kubeops/opensearch-os:1.6.10	6adf68b069361e4d57d729571a024942a236a36b6d125efe543d806c87300d91
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.10	2fbf1e334f518b4e493dde29ddabfd95acd838a6be6270f6dcf9af68db619deb
setup	V 1.6.10		kubeops/setup:1.6.10	6af233cd2fb5b490abcf502f6909b9a9e048206df841bfb5faa63f02ade4a0b2
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.10	87a5f75ad9babf6c5958b9a737bf416bcf38ab8c578de49483d4803e17935998

KubeOps 1.6.9

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	b3d249483c10cbd226093978e4d760553f2b2bf7e4a5d8176b56af2808e70aa1
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.9	7df7fc7f008a14eb61247fc4c3de19b8e5b450071645dbb8e156accc3ae46e61
clustercreate	V 1.6.9		kubeops/clustercreate:1.6.9	9bed96f6fd7d3ad0790a9b174c6a5ff805b58c8800993c265261618ffa480d9f
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.9	60bbcdb6135f9a4d47514819a31c0afef91cb9c3bc642c949bf4d408c2ff16e9
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.6.9	b86a12034c3b50f19f99849bd34ca5c28ca4f88b7afcfcc2c56e05c0acb9b153
helm	V 3.14.4		kubeops/helm:1.6.9	8497ed8517007542c3c22fdcf071b7db3e815e3069885ed6bc2d2ce9f2669eea
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.6.9	3844b25ee7231fbbe6b1881721a196a670e761f5dedab7445280e05b1f44e1a2
keycloak	V 1.16.0	1.0.0	kubeops/keycloak:1.6.9	c4877b3f9e0d623f74106b9f4d1f9bddc7f0ffd43504f1b92898cd2030850f06
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.9	7aca956c4743f8e3607f30af2183df756952452ef6cc5aaed2944d0a71718336
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.9	f706513404a4b688ec8a8074e868a5b2e2ad384591ab413d776c9fb85f81fa2e
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.9	e961a19c19bf06845ff3816b2ff4e37f46ec7ecb4bf8face81eb45e731780b58
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.9	ab6f3f8ef380b1a8ae2f5a18194c162562e3ef9698382f128ff7741f76579dfc
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.6.9	c1f39f9c982887a7fa5d8613f360ca6681e9ac6bdd583172826a1181812886c4
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.6.9	d8c1af8e3976d85551888816781d7fb50abf3d0f1fb084b8f0115dc610d1d67e
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.6.9	c569c85de83064dc285e5b76774dc6bb5a716c44a25cfa8a4dbcdb17b75989fb
setup	V 1.6.9		kubeops/setup:1.6.9	3a4de7e654302caafaad37c03a906ae1d95cdd018f991b16ac0c0a3fa17419b5
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.9	4a6200688dc691b385ca5fa54d7b75ceb8087280bd5d45981cdd4224266e3853

KubeOps 1.7.0_Beta0 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		kubeops/calicomultus:0.0.5	6e2dfd8135160fce5643f4b2b71de6e6af47925fcbaf38340704d25a79c39c09
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.7.0_Beta0	4185b79d5d09a7cbbd6779c9e55d71ab99d6f2e46f3aed8abb2b97ba8aa586e4
clustercreate	V 1.7.0_Beta0		kubeops/clustercreate:1.7.0_Beta0	299f7ffe125e2ca8db1c8f1aacfc3b5783271906f56c78c5fc5a6e5730ca83e5
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.7.0_Beta0	d31d63fc1a5e086199c0b465dca0a8756c22c88800c4d1a82af7b8b9f108ce63
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.7.0_Beta0	38ea742f5c40bd59c217777f0707469c78353acb859296ae5c5f0fbac129fc32
helm	V 3.14.4		kubeops/helm:1.7.0_Beta0	a9e4704fdb1b60791c0ff91851a2c69ed31782c865650f66c6a3f6ab96852568
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.7.0_Beta0	0b967f3a34fea7a12b86bc226599e32adb305371f1ab5368570ebb5fbc0021c6
keycloak	V 26.1.0	1.0.0	kubeops/keycloak:1.7.0_Beta0	3aec97cbbb559954a038a2212e1a52a9720e47c4ba0d8088fe47b000f42c469a
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.7.0_Beta0	a777e3b9568cfc60d7a9adef8f81f2345b979c545b566317ed0bd8ed0cf9faf3
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.7.0_Beta0	9210e5ef28babfed186b47043e95e6014dd3eadcdb1dbd521a5903190ecd7062
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.7.0_Beta0	88d144bf4a8bdd0a78c30694aa723fd059cabfed246420251f0c35a22ba6b35f
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.7.0_Beta0	fb72ae157ece5f8b41716d6d1fe95e1a574ca7d5a9196c071f2c83bbd3faebe7
opensearch-dashboards	V 2.19.1	2.28.0	kubeops/opensearch-dashboards:1.7.0_Beta0	d4698f3252e14b12c68befb4cd3e0d6ac1f87b121f00a467722e878172effbad
opensearch	V 2.19.1	2.32.0	kubeops/opensearch-os:1.7.0_Beta0	80583692c4010b0df3ff4f02c933ce1ebd792b54e3c4e4c9d3713c2730a9e02c
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.7.0_Beta0	5720f56cde2eb37ef2b74ee0e5dc764555157503554936cc78e03514779ad2fd
setup	V 1.7.0_Beta0		kubeops/setup:1.7.0_Beta0	a6061c2795a52a772895c53ec334b595475da025b41d4cc14c6930d7d7cff018
velero	V 1.13.2	6.4.0	kubeops/velero:1.7.0_Beta0	a826c2b2189f9e0f60fcf571d7230cd079ebc2f1e7a9594a9f310ec530ea64a8

Kubeops 1.6.8 supports

Tool	App Version	Chart Version	Supported Package Version	SHA256 Checksum
harbor	v2.12.1	1.16.0	kubeops/harbor	38a1471919eb95e69196b3c34aa682cdc471177e06680fc4ccda0c6394e83561
cert-manager	v1.15.3	v1.15.3	kubeops/cert-manager	9b17986c8cb2cb18e276167dc63d3fe4a2a83d7e02c0fb7463676954d626dc88
filebeat	v8.5.1	8.5.1	kubeops/filebeat	c888885c3001d5ecac8c6fe25f2c09a3352427153dc38994d3447d4a2b7fee2b
ingress-nginx	v1.11.5	4.11.5	kubeops/ingress-nginx	664eb9b7dfba4a7516fc9fb68382f4ceaa591950fde7f9d8db6d82f2be802f3f
keycloak	v22.0.1	16.0.5	kubeops/keycloak	469edff4c01f2dcd8339fe3debc23d8425cf8f86bedb91401dc6c18d9436396c
kubeops-dashboard	v0.26.0	0.26.0	kubeops/kubeops-dashboard	0429b5dfe0dbf1242c6b6e9da08565578c7008a339cb6aec950f7519b66bcd1d
logstash	v8.4.0	8.5.1	kubeops/logstash	6586d68ed7f858722796d7c623f1701339efc11eddb71d8b02985bb643fdec2f
gatekeeper	v3.17.1	3.17.1	kubeops/gatekeeper	42bee78b7bb056e354c265384a0fdc36dc7999278ce70531219efe7b8b0759e6
prometheus	v0.76.1	62.7.0	kubeops/prometheus	987227f99dc8f57fa9ac1d5407af2d82d58ec57510ca91d540ebc0d5e0f011bc
rook-ceph	v1.15.6	v1.15.6	kubeops/rook-ceph	9dd9a5e96ccf2a7ebd1cb737ee4669fbdadee240f5071a3c2a993be1929b0905
velero	v1.13.2	6.4.0	kubeops/velero	b53948b2565c60f434dffa1dba3fc21b679c5b08308a2dde421125a4b81616cc
opensearch	v2.19.1	2.32.0	kubeops/opensearch	0e52bd9818be03c457d09132bd3c1a6790482bb7141f08987dc3bbf678d193bb
opensearch-dashboards	v2.19.1	2.28.0	kubeops/opensearch-dashboards	137dd6c80ed753a4e4637c51039788f184bdee6afb2626fddb2937aea919cbd8
clustercreate	v1.6.8		kubeops/clustercreate:1.6.8	032e67d4799ea8d56424a0173012d301118291ab3cfdd561657f2225d6446e8e
setup	v1.6.8		kubeops/setup:1.6.8	bd9c5e71dc4564bede85d27da39e5d31e286624be9efbd1e662ecc52bb8b136b
helm	v3.14.4		kubeops/helm:1.6.8	f3909a4ac8c7051cc4893a587806681bc55abdbf9a3241dc3c29c165081bc7b0
calicomultus	V 0.0.5		kubeops/calicomultus:1.6.8	b3d249483c10cbd226093978e4d760553f2b2bf7e4a5d8176b56af2808e70aa1

Kubeops 1.6.7 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.5		lima/calicomultus:0.0.5	18d458d6bda62efb37f6e07378bb90a8cee824286749c42154815fae01a10b62
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.7	39c2b7fb490dd5e3ad8b8a03ec6287a6d02dd518b86efd83b3a1e815fd641c98
clustercreate	V 1.6.7		kubeops/clustercreate:1.6.7	e83bd6e24dd5762e698d83d6e9e29480deda8bff693af7d835c83ba2e88ae3c2
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.7	64c4922429702e39fa8eed54856525161150c7c4c6b5328a2ac90ce56588fd71
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.6.7	769e914a9f02a6ca78ec03418895e67c058f851ce24560474640c60dab4c730a
helm	V 3.14.4		kubeops/helm:1.6.7	c970844547cde59195bc1c5b4f17521597b9af1012f112c24185169492d59213
ingress-nginx	V 1.11.5	4.11.5	kubeops/ingress-nginx:1.6.7	deaf25204437c2812b459c9e2b68ae83bc5343a57ac2ab87d9d8dd4b3d06039d
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.7	3829d879e3098b14f16709f97b579bb9446ff2984553b72bba39b238aaaf332a
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.7	f237297adb8b01b7ad7344321d69928273c7e1a7a342634401d71205297a90dd
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.7	832b5019fe6f8d3949d768e98b182bcb84d05019ca854c08518313875ab4eedb
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.7	955f38c63dc5d4a3a67450c85e262a2884711910cfd1ee56860279a07f5ef833
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.7	3257e829cc4829c190a069b2a6409ea32ed1a38031f45b8c880eb69b85173c64
opensearch-dashboards	V 2.16.0	2.22.0	kubeops/opensearch-dashboards:1.6.7	8c8e4dca83591ef1ff8b23d94646d0098c2c575e193f6baf746e64a03aface05
opensearch	V 2.16.0	2.23.1	kubeops/opensearch-os:1.6.7	fb80f291318a6c00696a0a8775c571dea3ed7a2bec1b8d3394c07081b2409605
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.7	7198a6b33e677399ad90a2c780a7bf8af96e00de5ed46eef8215f6626645f06f
setup	V 1.6.7		kubeops/setup:1.6.7	2de24b9e24913e5f3966069de200644ae44b7777c7f94793a6f059f112649ea5
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.7	0ab6465bd5e8e422d06ce1fc2bd1d620e36bdedbc2105bc45e9a80d9f9e71e0d

Kubeops 1.6.6 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	1.15.3	kubeops/cert-manager:1.6.6	63e2ef627351619ab9813a684106dc19b187c63d643b68096889d8e0abf0640b
clustercreate	V 1.6.6		kubeops/clustercreate:1.6.6	dc334cf0cede9352069e775c0ed4df606f468340f257c4fa5687db7a828906c9
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.6	09ad9bf914b7d694619be3195b43c51157234f8bb5b5b24adfe399298f47e495
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.6.6	4f2a1112234edb3cf69ec4447b96bbc81593f61327ac93f6576ebe0ab1ee4d9b
helm	V 3.14.4		kubeops/helm:1.6.6	9ea60096ce6faa4654b8eced71c27e733fa791bacfc40095dfc907fd9a7d5b46
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.6	f518a5d909697b0275b4515dc1bc49a411b54992db469319e070809e8bbffd9e
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.6	3d2781a454f0cbe98c611e42910fb0e199db1dec79ac970c08ed4e9735581c4c
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.6	d21106e44b52f30cb23cb01bf2217662d7b393fd11046cbcc4e9ff165a725c1b
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.6	ada9d5a69b8c277c2c9037097e6a994d6c20ff794b51a65c30bdf480cfb23e52
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.6	89b8ae46c9bbc90c9d96e45a801c580608d24e4eaef2adadadb646a3012ece72
opa-gatekeeper	V 3.17.1	3.17.1	kubeops/opa-gatekeeper:1.6.6	a9c5423fdfabf456fa18b9808b9e9c9ee9428d5f5c4035810b9dbc3bfb838e4c
opensearch-dashboards	V 2.16.0	2.22.0	kubeops/opensearch-dashboards:1.6.6	2f8f66e6e321b773fcd5fb66014600b4f9cffda4bcea9f9451802274561f3ff4
opensearch	V 2.16.0	2.23.1	kubeops/opensearch-os:1.6.6	8ab9d1d09398083679a3233aaf73f1a664bd7162e99a1aef51b716cd8daa3e55
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.6	14b8cb259d6a0bb73ac576de7a07ed76499b43551a3d8a44b76eea181013080e
setup	V 1.6.6		kubeops/setup:1.6.6	92e392f170edb2edc5c92e265e9d92a4d6db5c6226f4603b33cece7361928089
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.6	98bde279f5a8b589a5234d63fba900235b07060c6554e9f822d41b072ddbd2f9

KubeOps 1.6.5 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.5	a66cfcf7849f745033fc8d6140d7b1ebbccb013739b37e26d9eb6dd22e0fb973
clustercreate	V 1.6.5		kubeops/clustercreate:1.6.5	a577edf4ea90710d041d31f434c4114d8efb4d6d9140ce39ca3b651f637b7147
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.5	70c2fceba0bc5e3d4dc0b56ad5cae769f79dc439915b0757b9b041244582b923
harbor	V 2.12.0	1.16.0	kubeops/harbor:1.6.5	9d3283235cf41073d1ade638218d8036cb35764473edc2a6d3046ca7b5435228
helm	V 3.14.4		kubeops/helm:1.6.5	d1c67bc9084d647217ee57f2e9fd4df3cbeb50d771961423c9e8246651910daa
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.5	9453b739e927f36cebe17b4da8f08f843693a52b049a358612aab82f8d1cc659
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.5	df652caa301d5171a7a3ae1ae8191790ef9f0af6de2750edbf2629b9022ccb3b
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.5	ccbb8721aa9a5c60661726feac3b3fd63d6711875b3a8e816b1cbdc68c51f530
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.5	c3544bd6ddbac3c9ac58b3445c8a868a979ce669d1096dcdafa9842b35edd2d7
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.5	2ec4835b326afce0cdb01d15bbe84dabfe988dab295b6d12114383dd528b7807
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.5	f5e6d871c12d463430aacd5adfd9fbc728a3dbf684424c002de1ae8d0b4df014
opensearch-dashboards	V 2.22.0	2.16.0	kubeops/opensearch-dashboards:1.6.5	a28a3b2161b276385062072fa05ac9cd34447e207c701b0700c78f5e828ec133
opensearch	V 2.23.1	2.16.0	kubeops/opensearch-os:1.6.5	e8bf63cbbb452e3e5cf7e62e3c6324e7dad31d1713e306c3847770a7ef67ca3a
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.6 cluster/v1.15.6 operator	kubeops/rook-ceph:1.6.5	a8ee95eaca0705f95884d54a188fa97e5c9080601dc3722a16a80a1599783caa
setup	V 1.6.5		kubeops/setup:1.6.5	1ac0ab68976e4c6e1abd867a8d3a58391b2b0fddd24ba1aefbed6b0f5d60b9ab
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.5	7c224434008d856b9fe0275ac0c528f865fb9a549a46bbeb90a34221a4d8c187

KubeOps 1.6.4 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.4	61fea41a31cdb7fefb2f4046c9c94ef08dc57523c0b8516ebc11f278b3d79b37
clustercreate	V 1.6.4		kubeops/clustercreate:1.6.4	b9a0c9eefeebc6057abcecc7cd6e53956baf28614d48141e1530ae6a4f433f2b
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.4	23dd2674b3447b3459c8a2b65f550726b6c97ca588d5c7259eb788bec6e4d317
harbor	V 2.11.1	1.15.1	kubeops/harbor:1.6.4	b794a6504769abff5b4ebba7c6384f83409c8d7d8d7687e3e49eec8a31e1a192
helm	V 3.14.4		kubeops/helm:1.6.4	1309b1cefb132152cd6900954b6b68cce6ce3b1c9e878fc925d8ef0439eee5f1
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.4	24214c2e96cf949073ba2e132a57c03096f36f5920a6938656bd159242ce8ec2
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.4	835d63c0d905dca14ee1aa5bc830e4cb3506c948d1c076317993d2e1a8b083ba
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.4	0ebf8ef4d2bf01bc5257c0bf5048db7e785743461ce1969847de0c9605562ef4
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.4	59eb96a2f09fa8b632d4958215fd45a82df3c0f7697281ea63e54f49d4a81601
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.4	3571e2ed554c8bd57272fa8e2de85e26e67a7dbf62d8622c5d796d5e3f8b6cf5
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.4	f5ce384bd332f3b6ffccd09b5824e92976019132b324c8fecbc261d14f2df095
opensearch-dashboards	V 2.22.0	2.16.0	kubeops/opensearch-dashboards:1.6.4	6dd16d2e411bdde910fc3370c1aca73c3c934832e45174ec71887d74d70dfcec
opensearch	V 2.23.1	2.16.0	kubeops/opensearch-os:1.6.4	cab021ed5f832057f2d4a7deaaccb1e2d2ab5d29bac502fb0daeebd8692a8178
rook	v1.15.6 cluster/v1.15.6 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.4	3f7c8c22406b5dc50add81f0df45a65d6d81ec47bbf3fb9935959ff870481601
setup	V 1.6.4		kubeops/setup:1.6.4	4760479e480453029f59152839d6624f7c5a7374fbc37ec2d7d14f8253ab9204
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.4	55136b3b4ea5aa8582b1300c37f084a48870f531496ed37a0849c33a63460b15

KubeOps 1.6.3 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.3	11105f523a2d8faf3bbfdca9e4d06145b4d52bad0ee0f16586266c26b59d5fe5
clustercreate	V 1.6.3		kubeops/clustercreate:1.6.3	9bce651b5d3caa5e83bfad25ef5d2908e16b2cf854168baf59b9ff586841e856
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.3	36d0359590b0c5dd3e8f8cd4c5d769a1eea3c3009593cc465ae31f4d9fbeaa02
harbor	V 2.11.1	1.15.1	kubeops/harbor:1.6.3	9a9d46f2c81a7596c8d00e920b3a733331d2def0676cc077b00749293e24255a
helm	V 3.14.4		kubeops/helm:1.6.3	f3e90f91c99314ad8357a11129602ddb693aa7792038306f903cff3791a22a3e
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.3	97d27c7cfe437275994757e0d3395c1864fd1cd57f0441754c7ec2cf128893ab
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.3	3d300a861d8024595fbc65be6010a3738384754c574bff9aca07d3dfc988671d
kubeops-dashboard	V 0.26.0	0.26.0	kubeops/kubeops-dashboard:1.6.3	ab7a339a132138f732aa1a9b70e3308c449566920155f67e4a72a1f2591b09db
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.3	e24aa21f9bcdf900f8d15edeab380ac68921b937af2baa638971364264a9d6cd
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.3	521d5238e1e6ca5adb12f088e279e05e47432b99008f99d5aed0bee75b740582
opa-gatekeeper	V 3.17.1	v3.17.1	kubeops/opa-gatekeeper:1.6.3	73d1e72c88da83889e48a908f6bac522d416e219b4d342dbcfff7ca987f32f49
opensearch-dashboards	V 2.22.0	2.16.0	kubeops/opensearch-dashboards:1.6.3	0ef3767f2c1b134d539f5f69a5e74509c2d232ccd337f33eea1d792e0f538f43
opensearch	V 2.23.1	2.16.0	kubeops/opensearch-os:1.6.3	f9165115615e6f58ad320085bf73a37d559aa24e93225edd60cea203f8bdfe70
rook	v1.15.1 cluster/vv1.15.1 operator	v1.15.1 cluster/v1.15.1 operator	kubeops/rook-ceph:1.6.3	13b274e95da154699f72ae8442d1dca654311805d33b33f3d1eb6ea93bc8d5fe
setup	V 1.6.3		kubeops/setup:1.6.3	cbe81f4169ead9c61bf65cf7b8cc47674a61ce9a6df37e6d8f7074254ea01d7f
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.3	b27addb2fc9d7498d82a649cdda61aec32b6f257377472fed243621dbc55b68b

KubeOps 1.6.2 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.1	8ca88b91370d395ea9bcf6f1967a38a2345ea7024936a3be86c51a8079f719a7
clustercreate	V 1.6.1		kubeops/clustercreate:1.6.1	5aeec18ea4c960ee4301f9a7808f4eda7d76ec1811be15a6c8092155997a41ce
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.1	0908fa15d1d85a59a9887ac8080a5e46f3ee13167f1fcaadefbf4b6229f0cf94
harbor	V 2.11.1	1.15.1	kubeops/harbor:1.6.1	715b9ce2d0925d8207311fc078c10aa5dfe01685b47743203e17241e0c4ac3c7
helm	V 3.14.4		kubeops/helm:1.6.1	f149f8687285479e935207fc1c52e0c19e0bf21bc5b00bf11433f2fef7eb2800
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.1	6a7d6c60c26d52a6e322422655e75d8a84040e3022c74a1341b3cc7dae3f1d14
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.1	135546a99aa8f25496262ed36a910f80f35c76f0f122652bd196a68b519a41e4
kubeops-dashboard	V 1.0.0	0.11.0	kubeops/kubeops-dashboard:1.6.1	37c04d6cd7654847add82572c8b2d38520ea63aff47af3b222283b1d570f44a8
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.1	c45db783a0e5c0475d9cd8e9c1309fa9af45410a8cca055f1c4028b8488cb4c9
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.1	19171f2aab866c53733147904aed8d39981f892aac0861dfd54354cdd98d0510
opa-gatekeeper	V 3.16.0	v3.16.0	kubeops/opa-gatekeeper:1.6.1	811f14f669324a7c9bfbac04aac074945c4aecffc926fc75126b44ff0bd41eb2
opensearch-dashboards	V 2.14.0	2.18.0	kubeops/opensearch-dashboards:1.6.1	7985e684a549f2eada4f3bf9a6490dc38be9b525e8f43ad9ff0c9377bccb0b7b
opensearch	V 2.16.0	2.23.1	kubeops/opensearch-os:1.6.1	cb804a50ab971ec55c893bd949127de2011503af37e221c0eb3ad83f5c78a502
rook	v1.12.5 cluster/v1.12.5 operator	v1.12.5 cluster/v1.12.5 operator	kubeops/rook-ceph:1.6.1	25e684fdc279b4f97cf1a5039f54fffbc1cf294f45935c20167dadd81a35ad52
setup	V 1.6.1		kubeops/setup:1.6.1	5b40a96733c2e526e642f17d2941d7a9422ae0a858f14af343277051df96dc09
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.1	cb228d2c6fd69749e91444def89fd79be51bcb816cabc61c7032404f5257a767

KubeOps 1.6.1 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.15.3	V 1.15.3	kubeops/cert-manager:1.6.1	8ca88b91370d395ea9bcf6f1967a38a2345ea7024936a3be86c51a8079f719a7
clustercreate	V 1.6.1		kubeops/clustercreate:1.6.1	5aeec18ea4c960ee4301f9a7808f4eda7d76ec1811be15a6c8092155997a41ce
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.1	0908fa15d1d85a59a9887ac8080a5e46f3ee13167f1fcaadefbf4b6229f0cf94
harbor	V 2.11.1	1.15.1	kubeops/harbor:1.6.1	715b9ce2d0925d8207311fc078c10aa5dfe01685b47743203e17241e0c4ac3c7
helm	V 3.14.4		kubeops/helm:1.6.1	kubeops/helm:1.6.1.output
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.1	6a7d6c60c26d52a6e322422655e75d8a84040e3022c74a1341b3cc7dae3f1d14
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.1	135546a99aa8f25496262ed36a910f80f35c76f0f122652bd196a68b519a41e4
kubeops-dashboard	V 0.15.1	0.11.0	kubeops/kubeops-dashboard:1.6.1	37c04d6cd7654847add82572c8b2d38520ea63aff47af3b222283b1d570f44a8
prometheus	V 0.76.1	62.7.0	kubeops/kube-prometheus-stack:1.6.1	c45db783a0e5c0475d9cd8e9c1309fa9af45410a8cca055f1c4028b8488cb4c9
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.1	19171f2aab866c53733147904aed8d39981f892aac0861dfd54354cdd98d0510
opa-gatekeeper	V 3.16.0	v3.16.0	kubeops/opa-gatekeeper:1.6.1	811f14f669324a7c9bfbac04aac074945c4aecffc926fc75126b44ff0bd41eb2
opensearch-dashboards	V 2.14.0	2.18.0	kubeops/opensearch-dashboards:1.6.1	7985e684a549f2eada4f3bf9a6490dc38be9b525e8f43ad9ff0c9377bccb0b7b
opensearch	V 2.16.0	2.23.1	kubeops/opensearch-os:1.6.1	cb804a50ab971ec55c893bd949127de2011503af37e221c0eb3ad83f5c78a502
rook	v1.12.5 cluster/v1.12.5 operator	v1.12.5 cluster/v1.12.5 operator	kubeops/rook-ceph:1.6.1	25e684fdc279b4f97cf1a5039f54fffbc1cf294f45935c20167dadd81a35ad52
setup	V 1.6.1		kubeops/setup:1.6.1	5b40a96733c2e526e642f17d2941d7a9422ae0a858f14af343277051df96dc09
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.1	kubeops/velero:1.6.1.output

KubeOps 1.6.0 supports

Tools	Supported App Version	Chart Version	Supported Package Version	SHA256 Checksum
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	0e32dcea34fe707248cf2348f0ed422c278574594e440db8ceac8190e3984dac
cert-manager	V 1.14.5	v1.14.5	kubeops/cert-manager:1.6.0	1a9ed861709cbfb05158f7610026acf5199749f989e1527ad48b80a277323765
clustercreate	V 1.6.0		kubeops/clustercreate:1.6.0	730925a6231a4fc8c7abf162d5d47a0f60107cb4dfa825db6e52a15d769a812d
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.0	061dc4feed5d970db3d353d29b8ef8256a826b146e0d95befbea4d5350874b8f
harbor	V 2.9.3	1.13.3	kubeops/harbor:1.6.0	3dd7dceb969dad59140e58631fd3a0c9f60ed22e2f1c2e1d087118e9c7592f26
helm	V 3.14.4		kubeops/helm:1.6.0	cb53f7b751473dd96f435d9f614e51edeaea99f2ca57a3710b59c788540d48d5
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.0	068618eb258c2558c3097ed19344da9caad0d7b44a8252b160cd36ef4425b790
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.0	236b6955dc08707d6e645625681da0e356c865da9810695d4af7fc2509c36f25
kubeops-dashboard	0.15.1	0.11.0	kubeops/kubeops-dashboard:1.6.0	5564ec8dfa33bb332e2863b171580bffebad3dc379a1fd365bddf5fc1343caac
prometheus	V 0.73.2	58.6.0	kubeops/kube-prometheus-stack:1.6.0	0286d6a05e61081e3abe783a36512bf372a3184e6f91457819a2b2c4046ce35a
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.0	88851fe2d5ec269a21811b15bb9cd11778d6564560eefe3938e874202298f3f1
opa-gatekeeper	V 3.16.0	v3.16.0	kubeops/opa-gatekeeper:1.6.0	ad841610be5ce624abeb6e439e3e353bd2f1240ca613e24ebdc13f36e8891a1a
opensearch-dashboards	V 2.14.0	2.18.0	kubeops/opensearch-dashboards:1.6.0	6364ffb7dbe05ea16a685ddf6e3d3a2b59ef6e8b28e5a1194710a5c37ae72c40
opensearch	V 2.14.0	2.20.0	kubeops/opensearch-os:1.6.0	16e1699fe187962fc58190a79d137db4c07723f2a84a889f393830b0093dba82
rook	v1.12.5 cluster/v1.12.5 operator	v1.12.5 cluster/v1.12.5 operator	kubeops/rook-ceph:1.6.0	48f79af13a0da86ea5019c78c24aa52c719d03a6ea2ab4e332b2078df0c02a16
setup	V 1.6.0		kubeops/setup:1.6.0	e2dd0419e17bbd2deaaea1f2888d391749afc0f550145c1e6a3ef5d5fba3a6a2
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.0	2b9e27dcf3a927ebe044f332b597d399d99a1a95660f7a186cf7fb3658b3676d

KubeOps 1.6.0_Beta1 supports

Tools	supported App Version	Chart Version	supported Package Version	SHA256 Checksum
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.0_Beta1	72d21348d6153beb9b36c287900579f1100ccd7333f63ff30edc576cfcb47250
harbor	V 2.9.3	1.13.3	kubeops/harbor:1.6.0_Beta1	317c7a931bb7f1f5d1d10dd373355f048e350878c0eee086c67714b104fad7cb
helm	V 3.14.4		kubeops/helm:1.6.0_Beta1	7890eb0c45ae420b664c655175844d84194520ae20429ad3b9d894eb865a8e66
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.0_Beta1	3c90093bc613adc8919cd3cc9f7f87ecad72a89b85ea30a9c161a5c59e9e925b
opa-gatekeeper	V 3.16.0	v3.16.0	kubeops/opa-gatekeeper:1.6.0_Beta1	3f655af62141c33437fe1183700c9f5ae5bd537b84df0d649023ae2cdc83cd11
opensearch	V 2.14.0	2.20.0	kubeops/opensearch-os:1.6.0_Beta1	0953ab749ccdf8b03648f850298f259504d40338bffe03dde2d6ab27ff0cb787
opensearch-dashboards	V 2.14.0	2.18.0	kubeops/opensearch-dashboards:1.6.0_Beta1	9b15ab03a8be7c0e7515056e7b46d2ca9a425690701a7a77afb2b4455790041e
prometheus	V 0.73.2	58.6.0	kubeops/kube-prometheus-stack:1.6.0_Beta1	277131992a7b70669e8aa2a299417da15a4631c89c9cca0f89128a1f2d81e532
rook	v1.12.5 cluster /v1.12.5 operator	v1.12.5 cluster/v1.12.5 operator	kubeops/rook-ceph:1.6.0_Beta1	495a9afeb61ff50800c6bc9931b934ee75bd78c988f8fa47a8ee79299f1a3b51
cert-manager	V 1.14.5	v1.14.5	kubeops/cert-manager:1.6.0_Beta1	220c892ed25f126e63da55f942a690a4d0443f5ed27e66b6532cdf573bb597af
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.0_Beta1	c21c12901d4f0542928234a4695c579fc24588a5d83ad27a61321c6b697f5588
kubeops-dashboard	V 1.0.0	0.11.0	kubeops/kubeops-dashboard:1.6.0_Beta1	2fb230a9a9f2a3bfa5e4d588c5f63f12d1a8bc462f664ddd9b1d088f9ea141ac
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.0_Beta1	a9a3dedc583ec3d1b481e4799a50fe440af542f5b71176e0c80ba2e66d08bcdb
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.0_Beta1	9bf96e91902aa8caff4e2245ae7d08058bfa099079fbf4ba77df08b4c697d654
clustercreate	V 1.6.0 V		kubeops/clustercreate:1.6.0_Beta1	36dd0d62ff6a405d8dab1c330972cd033d73413cab578d51a9249738a0f14385
setup	V 1.6.0		kubeops/setup:1.6.0_Beta1	6c9a96e026dbe7820f56c22767765adadd12466056d02de19d9bcae2c9fbbcde
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	a58aa03128ee88f3803803186c357f7daab252d8f3ae51f4aea124e8f4939f7f

KubeOps 1.6.0_Beta0 supports

Tools	supported App Version	Chart Version	supported Package Version	SHA256 Checksum
fileBeat	V 8.5.1	8.5.1	kubeops/filebeat-os:1.6.0_Beta0	a4d399bcd9efb238b07aee4b54ad11de132599801608dffc69ca4eee04f71c07
harbor	V 2.9.3	1.13.3	kubeops/harbor:1.6.0_Beta0	c345a0bb5fd80414405588583843978f8ca7dc31e07cbbf1c0db956866bc9e4d
helm	V 3.14.4		kubeops/helm:1.6.0_Beta0	0ea84bc7b77dff23a13a1a2a9426930f68845e0b5a1481c2362f3d895215274f
logstash	V 8.4.0	8.5.1	kubeops/logstash-os:1.6.0_Beta0	75d42128535c5d30bc175287a3b9c04a193698bff64a830873c02ae697573127
opa-gatekeeper	V 3.16.0	v3.16.0	kubeops/opa-gatekeeper:1.6.0_Beta0	e19e933869c2feb73bea8838a7f4bfcf0bf19090bae97cbf84407241ea3ca973
opensearch	V 2.14.0	2.20.0	kubeops/opensearch-os:1.6.0_Beta0	d2f58718d691946ea60bebe8eec6629f78f290405fe3fa572cec41b81106526e
opensearch-dashboards	V 2.14.0	2.18.0	kubeops/opensearch-dashboards:1.6.0_Beta0	af6b12543a1e4cc863b06709ccbf67dec02db0f68d359916950a839abc581e5e
prometheus	V 0.73.2	58.6.0	kubeops/kube-prometheus-stack:1.6.0_Beta0	2d773591c3dda297c00cc37abed74d1cf1d1575feb2a69610f0bdc6fc9a69040
rook	v1.12.5 cluster /v1.12.5 operator	v1.12.5 cluster/v1.12.5 operator	kubeops/rook-ceph:1.6.0_Beta0	c3e95ec2fb9b96346cba802dd010a00fd1ddd791a2ce2cbefa464cfbb4a922cc
cert-manager	V 1.14.5	v1.14.5	kubeops/cert-manager:1.6.0_Beta0	792759e538124e8307daf9abb81aef203655a102a713a05a0b3b547b8c19dd99
ingress-nginx	V 1.10.0	4.10.0	kubeops/ingress-nginx:1.6.0_Beta0	41f64cea80d92a6356a713fb612a5bafbe6a527b2bd9e21e974347dd3f3ad0d2
kubeops-dashboard	V 1.0.0	0.11.0	kubeops/kubeops-dashboard:1.6.0_Beta0	a9522a68a6be45358b96a78527ca3653439b2c24c5ab349ac6767e003dee80a4
keycloak	V 22.0.1	16.0.5	kubeops/keycloak:1.6.0_Beta0	f3dcc5dd3b21d5da83c72f757146df3ddc32e5b793f7c6039df751ab88ccc2b4
velero	V 1.13.2	6.4.0	kubeops/velero:1.6.0_Beta0	4653976cf762030e859fe83af4ac0f0830d61dec0a9f40d33ab590743a6baebe
clustercreate	V 1.6.0 V		kubeops/clustercreate:1.6.0_Beta0	29dc5a9d903eb2d9ac836f580e1ca4ff2691f24989bdb1c31313526de29e0208
setup	V 1.6.0		kubeops/setup:1.6.0_Beta0	7c41ace358a4e62fb0c31a920456308086a1bda4294e1ff0ab26763ae41da9bd
calicomultus	V 0.0.4		lima/calicomultus:0.0.4	a58aa03128ee88f3803803186c357f7daab252d8f3ae51f4aea124e8f4939f7f

Kubeopsctl-2.0: Upgrading a Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

Upgrading a Kubernetes cluster is essential to maintain security, stability, and compatibility.Like Kubernetes itself, we adhere to the version skew policy and only allow upgrades between releases that differ by a single minor version. This ensures compatibility between components, reduces the risk of instability, and keeps the cluster in a supported and secure state.

For more information about the Version Skew Policy, see the official Kubernetes documentation:
Click here

Use the following steps to upgrade the Kubernetes version of a cluster.

Kubernetes Version Upgrade Process:

Prerequisites

Before performing any action with kubeopsctl, it is recommended to do a login with kosi. Refer to the official KOSI documentation for details here.

1. Pull required KOSI packages on your ADMIN

If you do not specify a parameter, the Kubernetes version 1.32.2 will be pulled.
With parameter --kubernetesVersion 1.34.1 you can pull an older Kubernetes version.
Available Kubernetes versions are

kubeopsctl pull --kubernetesVersion <x.xx.x>

2. Change your target version inside the cluster-values

Note The actual version is the current Kubernetes version in your cluster. The target version in the nodes, is version we want to do the upgrade (e.g., see this yaml file)

Note To upgrade your cluster, the parameter changeCluster must be set to true (e.g., see this yaml file)

3. Start the upgrade

kubeopsctl apply -f cluster-values.yaml

Important It is important to start the upgrade with the initial controlplane node. It is the first node with which the cluster was created.

Example 1 - Upgrade all nodes in the cluster to a specific version

In this example, a cluster is upgraded from Kubernetes version v1.33.5 to v1.34.1.

1. Pull required KOSI packages on your ADMIN

Pull the kubernetes v1.34.1 packages on your ADMIN machine.

kubeopsctl pull --kubernetesVersion 1.34.1

2. Update the target version in `cluster-values.yaml`

Adjust your cluster-values.yaml according to the example below. Make sure you set:

The current Kubernetes version in kubernetesVersion.
The target Kubernetes version in the kubeVersion field of each node.

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.33.5     # -> actual version
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true           # -> important! Needs to be set for an upgrade
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.34.1       # -> target version
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.34.1       # -> target version
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.34.1       # ->target version
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.34.1       # -> target version
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.34.1       # -> target version
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.34.1       # -> target version

2. Validate your values and upgrade the cluster

After cluster-values.yaml is updated, review all values carefully.
When you are ready, start the upgrade process:

kubeopsctl apply -f cluster-values.yaml

Note Tools like rook-ceph has no pdbs, so if you drain nodes for the kubernetes upgrade, rook ceph is temporarily unavailable. You should drain only one node at a time for the kubernetes upgrade.

Example 2 - Tranche upgrade of zones to a specific version

In this example, the cluster is upgraded in tranches. The order is:

zone1 (because it contains the initial control plane node),
then zone3,
and finally zone2.

Important It is important to start your tranche upgrade with zone1, because zone1 contains the initial control plane node.

1. Pull required KOSI packages on your ADMIN

Pull the kubernetes v1.33.5 packages on your ADMIN machine.

kubeopsctl pull --kubernetesVersion 1.33.5

2. Adjust your cluster-values in `zone1`

Adjust your cluster-values.yaml according to the example below. Make sure you set:

The current Kubernetes version in kubernetesVersion.
The target Kubernetes version only for the nodes in zone1. In the snippet below, only zone1 is configured with the target version.

Note Pay close attention to the format and syntax in cluster-values.yaml.

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.32.2     # -> actual version
kubeVipEnabled: false           
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.33.5       # -> target version
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.33.5       # -> target version
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.32.2       
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.32.2

3. Validate your values and upgrade the cluster

After cluster-values.yaml is updated, review all values carefully. When you are ready, start the upgrade process:

kubeopsctl apply -f cluster-values.yaml

Note Tools like rook-ceph has no pdbs, so if you drain nodes for the kubernetes upgrade, rook ceph is temporarily unavailable. You should drain only one node at a time for the kubernetes upgrade.

4. Adjust your `cluster-values.yaml` in `zone2`

Next, change the target version of zone2:

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.32.2     # -> actual version
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.33.5       # -> target version
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.33.5       # -> target version
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.33.5       # ->target version
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.33.5       # -> target version
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.32.2

5. Validate your values and upgrade the cluster

After cluster-values.yaml is updated, review all values carefully. When you are ready, start the upgrade process:

kubeopsctl apply -f cluster-values.yaml

6. Adjust your `cluster-values.yaml` `in zone3`

Finally, change the target version of zone 3.

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.32.2     # -> actual version
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: true
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.33.5       # -> target version
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.33.5       # -> target version
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.33.5       # ->target version
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.33.5       # -> target version
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.33.5       # -> target version
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.33.5       # -> target version

7. Validate your values and upgrade the cluster

After cluster-values.yaml is updated, review all values carefully. When you are ready, start the upgrade process:

kubeopsctl apply -f cluster-values.yaml

Kubeopsctl-2.0: Installing KubeOps Compliance applications

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

There is a predefined selection of applications included with KubeOps Compliance. These applications ensure a production-ready cluster deployment and can be individually configured as needed.

By separating the cluster values from the application values, the application values can be modified independently and installed at a later stage, providing greater flexibility and maintainability.

Prerequisits

Before performing any action with kubeopsctl, it is recommended to do a login with kosi. Refer to the official KOSI documentation for details here.

⚠ Warning To ensure proper DNS resolution, all components of the logging stack — Filebeat, Logstash, OpenSearch, and OpenSearch Dashboards—must be deployed within the same Kubernetes namespace (or network domain).

Example 1: Installing Applications in a non-airgap-environment

To install the KubeOps Compliance Applications in an existing cluster follow the next steps:

1. Define the Enterprise-Value-file

In the example value, the following applications are enabled:

opa-gatekeeper
rook-ceph
harbor
kubeops-dashboard

All other applications are disabled and will not be installed. For more information about available packages as well as parameters for each package check here.

The following file is only an example. Make sure to change the necessary values (ips, passwords, …) before usage

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
- name: opa-gatekeeper
  enabled: true
  values:
    standard:
      namespace: gatekeeper-system
    advanced:
- name: rook-ceph
  enabled: true
  values:
    standard:
      namespace: rook-ceph
      cluster:
        resources:
          mgr:
            requests:
              cpu: "500m"
              memory: "512Mi"
          mon:
            requests:
              cpu: "1"
              memory: "1Gi"
          osd:
            requests:
              cpu: "1"
              memory: "1Gi"
        dashboard:
          enabled: true
      operator:
        data:
          rookLogLevel: "DEBUG"
- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "password"
      redisPassword: "password"
      externalURL: http://10.2.10.110:30002
      nodePort: 30002
      hostname: harbor.local
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
- name: kubeops-dashboard
  enabled: true
  values:
    standard:
      namespace: monitoring
      hostname: kubeops-dashboard.local
      service:
        nodePort: 30007
    advanced:
- name: filebeat-os
  enabled: false
  values:
    standard:
      namespace: logging
    advanced:

2. Pull the KubeOps Compliance Applications packages
To pull the required application packages in the correct version for the release, use the following commands:

kubeopsctl pull -f enterprise-values.yaml --kubernetesVersion <x.xx.x>

kubeopsctl pull --tools enterprise-values.yaml --kubernetesVersion <x.xx.x>

3. The KubeOps Compliance Application installation process
Important for only installation of the tools is that you have set your flag changeCluster to false in your cluster-values.yaml.

The following file is only an example. Make sure to change the necessary values (ips, passwords, …) before usage

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: false                       # -> important
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.31.6         
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: https://packagerepo.kubeops.net/
changeCluster: false                # -> important
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.32.2      
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.32.2       
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.32.2       
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.32.2

⚠ Warning Both, Cluster-values.yaml and enterprise-values.yaml are required!

4. Validate your values and install the KubeOps Compliance Applications Once you finished defining your values, check them once again. If you are ready, just start the installation process with the command:

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

Example 2: Installing Applications in an airgap-environment

To install the KubeOps Compliance Applications in an existing cluster follow the next steps:

1. Define the Enterprise-Value-file

In the example value, the following applications are enabled:

opa-gatekeeper
rook-ceph
harbor
kubeops-dashboard

All other applications are disabled and will not be installed. Value-parameter will be explained in the references and can be found here.

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: true             # important for airgap, otherwise images are pulled from public registry
packages:
- name: opa-gatekeeper
  enabled: true
  values:
    standard:
      namespace: gatekeeper-system
    advanced:
- name: rook-ceph
  enabled: true
  values:
    standard:
      namespace: rook-ceph
      cluster:
        resources:
          mgr:
            requests:
              cpu: "500m"
              memory: "512Mi"
          mon:
            requests:
              cpu: "1"
              memory: "1Gi"
          osd:
            requests:
              cpu: "1"
              memory: "1Gi"
        dashboard:
          enabled: true
      operator:
        data:
          rookLogLevel: "DEBUG"
- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "password"
      redisPassword: "password"
      externalURL: http://10.2.10.110:30002
      nodePort: 30002
      hostname: harbor.local
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
- name: kubeops-dashboard
  enabled: true
  values:
    standard:
      namespace: monitoring
      hostname: kubeops-dashboard.local
      service:
        nodePort: 30007
    advanced:
- name: filebeat-os
  enabled: false
  values:
    standard:
      namespace: logging
    advanced:

2. Pull the KubeOps Compliance Applications packages
To pull the required application packages in the correct version for the release, use the following commands:

kubeopsctl pull -f enterprise-values.yaml --kubernetesVersion <x.xx.x>

kubeopsctl pull --tools enterprise-values.yaml --kubernetesVersion <x.xx.x>

Note The airgap-packages will be pulled automatically

3. The KubeOps Compliance Application installation process
Important for only installation of the tools is that you have set your flag changeCluster to false in your cluster-values.yaml.

The following file is only an example. Make sure to change the necessary values (ips, passwords, …) before usage

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true                        # -> important
clusterName: myCluster
clusterUser: root
kubernetesVersion: 1.32.2         
kubeVipEnabled: false
virtualIP: 10.2.10.110
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: /home/myuser/kubeops
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
systemCpu: 250m
systemMemory: 256Mi
packageRepository: local
changeCluster: false                # -> important
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.32.2      
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.32.2       
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.32.2      
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.32.2      
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.32.2      
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.32.2

⚠ Warning Both, Cluster-values.yaml and enterprise-values.yaml are required!

Note There is the possibility to do an Upgrade of the kubernetes version and do an installation of the KubeOps Compliance Tools

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

Kubeopsctl-2.0: Updating KubeOps Compliance applications

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

There is a predefined selection of applications included with KubeOps Compliance. These applications ensure a production-ready cluster deployment and can be configured individually as needed.

By separating cluster values from application values, application values can be modified independently and installed later, providing greater flexibility and maintainability.

kubeopsctl automatically detects whether an application is already deployed and updates it accordingly. de]

Important If you have made changes to your deployments, reapply them afterward or secure them using the advancedValues configuration.

Prerequisites

Before performing any action with kubeopsctl, it is recommended to do a login with kosi. Refer to the official KOSI documentation for details here.

Updated KubeOpsctl

If you have an older kubeopsctl version installed, update it before starting with updating Compliance appliactions.

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/deb/pool/main/
sudo apt update
sudo apt install -y kubeopsctl=<kubeopsctl-version>

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/rpm/
sudo dnf install -y --disableexcludes=kubeops-repo <kubeopsctl-version>

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/deb/pool/main/
wget https://packagerepo.kubeops.net/deb/pool/main/<kubeopsctl-version>.deb
sudo dpkg --install <kubeopsctl-version>.deb

# kubeopsctl-versions can be found under: https://packagerepo.kubeops.net/rpm
sudo rpm kubeopsctl
wget https://packagerepo.kubeops.net/rpm/<kubeopsctl-version>.rpm
sudo rpm --install <kubeopsctl-version>.rpm

Example 1: Updating Applications in a non-airgap-environment

To update the KubeOps Compliance Applications in an existing cluster follow the next steps:

Note All KubeOps Compliance applications versions are linked to the specific KubeOps Compliance version. A detailed version reference can be found here.

1. Define the Enterprise-Value-file

In the example value, the following applications are enabled:

opa-gatekeeper
rook-ceph
harbor
kubeops-dashboard

All other applications are disabled and will not be updated. Value-parameter will be explained in the references and can be found here.

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
- name: opa-gatekeeper
  enabled: true
  values:
    standard:
      namespace: gatekeeper-system
    advanced:
- name: rook-ceph
  enabled: true
  values:
    standard:
      namespace: rook-ceph
      cluster:
        resources:
          mgr:
            requests:
              cpu: "500m"
              memory: "512Mi"
          mon:
            requests:
              cpu: "1"
              memory: "1Gi"
          osd:
            requests:
              cpu: "1"
              memory: "1Gi"
        dashboard:
          enabled: true
      operator:
        data:
          rookLogLevel: "DEBUG"
- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "password"
      redisPassword: "password"
      externalURL: http://10.2.10.110:30002
      nodePort: 30002
      hostname: harbor.local
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
- name: kubeops-dashboard
  enabled: true
  values:
    standard:
      namespace: monitoring
      hostname: kubeops-dashboard.local
      service:
        nodePort: 30007
    advanced:
- name: filebeat-os
  enabled: false
  values:
    standard:
      namespace: logging
    advanced:

2. Pull the KubeOps Compliance Applications packages
To pull the required application packages in the correct version for the release, use the following commands:

kubeopsctl pull -f enterprise-values.yaml --kubernetesVersion <x.xx.x>

kubeopsctl pull --tools enterprise-values.yaml --kubernetesVersion <x.xx.x>

3. The KubeOps Compliance Application update process
Important for only update of the tools is that you have set your flag changeCluster to false in your cluster-values.yaml.

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: false                       # -> important
clusterName: <your cluster name>
clusterUser: <your user name>
kubernetesVersion: <your kubernetesversion>
kubeVipEnabled: false
virtualIP: <your master1 ip>
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: <your kubeopsroot path>
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
packageRepository: local            # If you have incorporated package repository, replace “local” with the URL https://packagerepo.kubeops.net/
changeCluster: false                # -> important
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.31.6       
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.30.8       
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.31.6

⚠ Warning Both, Cluster-values.yaml and enterprise-values.yaml are required!

4. Validate your values and update the KubeOps Compliance Applications Once you finished defining your values, check them once again. If you are ready, just start the update process with the command:

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

Example 2: Updating Applications in an airgap-environment

To update the KubeOps Compliance Applications in an existing cluster follow the next steps:

Note All KubeOps Compliance applications versions are linked to the specific KubeOps Compliance version. A detailed version reference can be found here.

1. Define the Enterprise-Value-file

In the example value, the following applications are enabled:

opa-gatekeeper
rook-ceph
harbor
kubeops-dashboard

All other applications are disabled and will not be updated. Value-parameter will be explained in the references and can be found here.

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: true             # important for airgap, otherwise images are pulled from public registry
packages:
- name: opa-gatekeeper
  enabled: true
  values:
    standard:
      namespace: gatekeeper-system
    advanced:
- name: rook-ceph
  enabled: true
  values:
    standard:
      namespace: rook-ceph
      cluster:
        resources:
          mgr:
            requests:
              cpu: "500m"
              memory: "512Mi"
          mon:
            requests:
              cpu: "1"
              memory: "1Gi"
          osd:
            requests:
              cpu: "1"
              memory: "1Gi"
        dashboard:
          enabled: true
      operator:
        data:
          rookLogLevel: "DEBUG"
- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "password"
      redisPassword: "password"
      externalURL: http://10.2.10.110:30002
      nodePort: 30002
      hostname: harbor.local
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
- name: kubeops-dashboard
  enabled: true
  values:
    standard:
      namespace: monitoring
      hostname: kubeops-dashboard.local
      service:
        nodePort: 30007
    advanced:
- name: filebeat-os
  enabled: false
  values:
    standard:
      namespace: logging
    advanced:

2. Update kubeopsctl

If you have an older kubeopsctl version installed, update it using the following commands.

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/deb/pool/main/
sudo apt update
sudo apt install -y kubeopsctl=<kubeopsctl-version>

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/rpm/
sudo dnf install -y --disableexcludes=kubeops-repo <kubeopsctl-version>

# kubeopsctl-version can be found under : https://packagerepo.kubeops.net/deb/pool/main/
wget https://packagerepo.kubeops.net/deb/pool/main/<kubeopsctl-version>.deb
sudo dpkg --install <kubeopsctl-version>.deb

# kubeopsctl-versions can be found under: https://packagerepo.kubeops.net/rpm
sudo rpm kubeopsctl
wget https://packagerepo.kubeops.net/rpm/<kubeopsctl-version>.rpm
sudo rpm --install <kubeopsctl-version>.rpm

2. Pull the KubeOps Compliance Applications packages
To pull the required application packages in the correct version for the release, use the following commands:

kubeopsctl pull -f enterprise-values.yaml --kubernetesVersion <x.xx.x>

kubeopsctl pull --tools enterprise-values.yaml --kubernetesVersion <x.xx.x>

Note The airgap-packages will be pulled automatically

3. The KubeOps Compliance Application update process
Important for only the update of the tools is that you have set your flag changeCluster to false in your cluster-values.yaml.

# file cluster-values.yaml
apiVersion: kubeops/kubeopsctl/cluster/beta/v1
imagePullRegistry: registry.kubeops.net/kubeops/kubeops
airgap: true                          # -> important
clusterName: <your cluster name>
clusterUser: <your user name>
kubernetesVersion: <your kubernetesversion>
kubeVipEnabled: false
virtualIP: <your master1 ip>
firewall: nftables
pluginNetwork: calico
containerRuntime: containerd
kubeOpsRoot: <your kubeopsroot path>
serviceSubnet: 192.168.128.0/17
podSubnet: 192.168.0.0/17
debug: true
packageRepository: local            # If you have incorporated package repository, replace “local” with the URL https://packagerepo.kubeops.net/
changeCluster: false                # -> important
zones:
- name: zone1
  nodes:
  - name: demo-controlplane01
    iPAddress: 10.2.10.110
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker01
    iPAddress: 10.2.10.210
    type: worker
    kubeVersion: 1.31.6       
- name: zone2
  nodes:
  - name: demo-controlplane02
    iPAddress: 10.2.10.120
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker02
    iPAddress: 10.2.10.220
    type: worker
    kubeVersion: 1.30.8       
- name: zone3
  nodes:
  - name: demo-controlplane03
    iPAddress: 10.2.10.130
    type: controlplane
    kubeVersion: 1.31.6       
  - name: demo-worker03
    iPAddress: 10.2.10.230
    type: worker
    kubeVersion: 1.31.6

⚠ Warning Both, Cluster-values.yaml and enterprise-values.yaml are required!

Note There is the possibility to do an Upgrade of the kubernetes version and do an installation of the KubeOps Compliance Tools

kubeopsctl apply -f cluster-values.yaml -f enterprise-values.yaml

Kubeopsctl-2.0: Harbor Deployment with CloudNativePG

Mon, 01 Jan 0001 00:00:00 +0000

This guide describes how to deploy Harbor on Kubernetes using a CloudNativePG (CNPG) PostgreSQL cluster managed by the CloudNativePG operator in an air‑gap and non-airgap environments.

Harbor Deployment with CloudNativePG in Non-airgap enviroment

Prerequisites

A running Kubernetes cluster.
kubectl, kosi, and kubeopsctl installed and configured.
You need to login with kosi . Refer to the official KOSI documentation for details here.

Step 1 — Install CloudNativePG operator

Deploy the operator with Kosi:

kosi install --hub kubeops kubeops/cloudnative-pg-operator:<kubeopsctl_version> --dname cnpg-operator

With this step, it Installs the CloudNativePG operator into the cluster and the operator manages PostgreSQL clusters and their lifecycle.

Step 2 — Create PostgreSQL cluster for Harbor

1. Apply the following Cluster manifest to create a Postgres cluster with 2 instances and 1Gi storage:

cat <<EOF | kubectl apply -f -
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cloudnative-pg
  namespace: harbor
spec:
  instances: 2
  imagePullSecrets:
  - name: registry-pullsecret
  storage:
    size: 1Gi
EOF

2. Services and pods created for the cluster `cloudnative-pg`:

cloudnative-pg-rw → primary (read/write) cloudnative-pg-ro → replicas (read-only) cloudnative-pg-r → all pods

3. Verify pods are Running:

kubectl get pods -n harbor

Step 3 — Retrieve application user credentials

CNPG automatically creates a Secret named cloudnative-pg-app in the harbor namespace.

1. Verify the Secret exists:

kubectl get secret cloudnative-pg-app -n harbor

2. Decode the base64-encoded fields:

kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.username}" | base64 -d 
kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.password}" | base64 -d 
kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.dbname}" | base64 -d

Example values (for illustration only):

username: app
password: Hw2t7hXuKPfZrVjVDwCc4PeKTevlB7ORmzQeW50JtEqiwHl40xkxuhVHeRIU3fX2
database: app

Important:
Use the non-superuser application credentials from this Secret in Harbor’s configuration.

Step 4 — Update Harbor tools.yaml for an external database

Edit your tools.yaml and set Harbor values under the helm chart configuration.
Example snippet:

- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "<DB_PASSWORD>"
      redisPassword: "Redis_Password"
      externalURL: <your_domain_name>
      nodePort: 30002
      hostname: <your_domain_name>
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
      database:
        type: external
        external:
          host: "cloudnative-pg-rw.harbor.svc.cluster.local"
          port: "5432"
          username: "app"
          password: "Hw2t7hXuKPfZrVjVDwCc4PeKTevlB7ORmzQeW50JtEqiwHl40xkxuhVHeRIU3fX2"
          coreDatabase: "app"

Important: Use the -rw service host (cloudnative-pg-rw…) for write operations.
Do not use a superuser account.
Ensure the password matches the CNPG Secret.

Step 5 — Install Harbor with Kosi

1. Deploy Harbor using the updated tools.yaml:

kosi install --hub kubeops kubeops/harbor:2.0.3 -f tools.yaml --dname harbor

2. Verify Harbor pods:

kubectl get pods -n harbor

3. Access Harbor at: <your_domain_name>:30002 (or as configured)

Harbor Deployment with CloudNativePG in Airgap envoirnment

Prerequisites

A running Kubernetes cluster.
podman, kubectl, kosi, and kubeopsctl installed and configured.
You need to login with kosi . Refer to the official KOSI documentation for details here.

First, log in to your Harbor registry and pull the CloudNativePG operator package using kosi.

podman login <ip_address>:<nodePort> -u <user_name> -p <password> --tls-verify=false   
kosi pull --hub kubeops kubeops/cloudnative-pg-operator:<kubeopsctl_version> -o cloudnative-pg.tgz -r <ip_address>:<NodePort>/kubeops -t localhost:<NodePort>/kubeops

kosi install -p cloudnative-pg.tgz --dname cld-pg

Step 2: Manually Mirror the PostgreSQL Image to the KubeOps Project

In an air‑gapped environment, you must manually pull the PostgreSQL image from the external registry and push it to the KubeOps project in Harbor.

podman pull ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie

podman tag ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie <ip_address>:<NodePort>/kubeops/cloudnative-pg/postgresql:18.1-system-trixie

podman push <ip_address>:<NodePort>/kubeops/cloudnative-pg/postgresql:18.1-system-trixie --tls-verify=false

Step 3: Create the PostgreSQL Cluster for Harbor

Create a CNPG PostgreSQL cluster with two instances and 1Gi of storage in the harbor namespace:

cat <<EOF | kubectl apply -f -
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cloudnative-pg
  namespace: harbor
spec:
  instances: 2
  storage:
    size: 1Gi
EOF

Step 4: Configure the Cluster to Use the Mirrored Image

Update the CNPG cluster to use the image you pushed to Harbor in Step 2.

1.List the cluster:

kubectl get cluster -n harbor

2.Edit the cluster:

kubectl edit cluster -n harbor cloudnative-pg

3.Replace the image name in the editor:

From:

ghcr.io/cloudnative-pg/postgresql:18.1-system-trixie

To:

localhost:<NodePort>/kubeops/cloudnative-pg/postgresql:18.1-system-trixie

Step 5: Retrieve CNPG Application User Credentials

CNPG automatically creates a Secret named cloudnative-pg-app in the harbor namespace.

1. Verify the Secret exists:

kubectl get secret cloudnative-pg-app -n harbor

2. Decode the base64-encoded fields:

kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.username}" | base64 -d 
kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.password}" | base64 -d 
kubectl get secret cloudnative-pg-app -n harbor -o jsonpath="{.data.dbname}" | base64 -d

Example values (for illustration only):

username: app
password: Hw2t7hXuKPfZrVjVDwCc4PeKTevlB7ORmzQeW50JtEqiwHl40xkxuhVHeRIU3fX2
database: app

Important:
Use the non-superuser application credentials from this Secret in Harbor’s configuration.

Step 6: Configure Harbor in enterprise.yaml

Update your enterprise-values.yaml to configure Harbor with the external CNPG database.
Example snippet:

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
  - name: harbor
    enabled: true
    values:
      standard:
        namespace: harbor
        harborpass: "password"
        databasePassword: "<DB_PASSWORD>"
        redisPassword: "Redis_Password"
        externalURL: <your_domain_name>
        nodePort: 30002
        hostname: <your_domain_name>
        harborPersistence:
          persistentVolumeClaim:
            registry:
              size: 40Gi
              storageClass: "rook-cephfs"
            jobservice:
              jobLog:
                size: 1Gi
                storageClass: "rook-cephfs"
            database:
              size: 1Gi
              storageClass: "rook-cephfs"
            redis:
              size: 1Gi
              storageClass: "rook-cephfs"
            trivy: 
              size: 5Gi
              storageClass: "rook-cephfs"
      advanced:
        database:
          type: external
          external:
            host: "cloudnative-pg-rw.harbor.svc.cluster.local"
            port: "5432"
            username: "app"
            password: "Hw2t7hXuKPfZrVjVDwCc4PeKTevlB7ORmzQeW50JtEqiwHl40xkxuhVHeRIU3fX2"
            coreDatabase: "app"

Important: Use the -rw service host (cloudnative-pg-rw…) for write operations.
Do not use a superuser account.
Ensure the password matches the CNPG Secret.

Apply the updated configuration:

kubeopsctl apply -f enterprise-values.yaml

Step 6 — Verify the Harbor Deployment

Check that Harbor pods are running in the harbor namespace:

kubectl get pods -n harbor

All Harbor pods should eventually reach a Running or Ready state

General Notes and best practices

Always decode the CNPG app Secret to obtain the correct username, password, and database name used by Harbor.
Use the cloudnative-pg-rw service for write traffic; use -ro services for reads if required.
Production recommendations (follow your organizational security policies):
Create a dedicated DB user (for example, harbor) rather than reusing the default app user.
Enable TLS for Postgres connections.
Implement backups (for example, CNPG Barman or object storage like S3).
To scale the CNPG cluster, update the instances field in the Cluster spec and reapply.

Kubeopsctl-2.0: Ingress Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Manual configuration of the Nginx-Ingress-Controller

Right now the Ingress Controller Package is not fully configured. To make complete use of the Ingress capabilities of the cluster, the user needs to manually update some of the settings of the corresponding service.

Choosing an ingress controller. KubeOps ships two ingress controllers: ingress-nginx and traefik. NGINX has reached end of support, so new clusters should prefer Traefik and existing clusters should plan a migration. To replace nginx-ingress with Traefik (including how SSO, certificates and dashboards keep working), follow How to migrate from nginx to traefik ingress. The rest of this page covers the manual nginx service configuration.

Locating the service

The service in question is called “ingress-nginx-controller” and can be found in the same namespace as the ingress package itself. To locate the service across all namespaces, you could use the following command.

kubectl get service -A | grep ingress-nginx-controller

This command should return two entries of services, “ingress-nginx-controller” and “ingress-nginx-controller-admission”, though only the first one needs to be further adjusted.

Setting the Ingress-Controller service to type NodePort

To edit the service, you can use the following command, although the actual namespace may be different. This will change the service type to NodePort.

kubectl patch service ingress-nginx-controller -n ingress -p '{"spec":{"type":"NodePort"}}'

Kubernetes will now automatically assign unused portnumbers for the nodePort to allow http and https connections to the service. These can be retrieved by running the same command, used to locate the service. Alternatively, you can use the following command, which adds the portnumbers 30080 and 30443 for the respective protocols. By doing so, you have to make sure, that these portnumbers are not being used by any other NodePort service.

kubectl patch service ingress-nginx-controller -n ingress --type=json -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}, {"op":"add","path":"/spec/ports/0/nodePort","value":30080}, {"op":"add","path":"/spec/ports/1/nodePort","value":30443}]'

Configuring external IPs

If you have access to external IPs that route to one or more cluster nodes, you can expose your Kubernetes-Services of any type through these addresses. The command below shows how to add an external IP-Adress to the service with the example value of “192.168.0.1”. Keep in mind that this value has to be changed in order to fit your networking settings.

kubectl patch service ingress-nginx-controller -n ingress -p '{"spec":{"externalIPs":["192.168.0.1"]}}'

Kubeopsctl-2.0: Accessing Dashboards installed with KubeOps

Mon, 01 Jan 0001 00:00:00 +0000

To access an application dashboard, an SSH tunnel to one of the control planes is needed. The following dashboards are available and configured with the following NodePorts by default:

⚠ Warning Change default credentials before going to production.

Note The NodePorts and hostnames shown on this page reflect the default values. Both can be configured in the enterprise-values.yaml. See Fileformats for details.

NodePort

32090 (if not set otherwise in the enterprise-values.yaml)

Connecting via SSH Tunnel

To establish a tunnel, forward the NodePort from one of the control planes to your local machine. You can use the command line, PuTTY, or MobaXterm for this.
After that, the dashboard can be accessed at localhost:32090/prometheus/.

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/prometheus/.

No credentials are necessary for login.

NodePort

30211 (if not set otherwise in the enterprise-values.yaml)

Connecting via SSH Tunnel

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/grafana/.

username: the username set in the enterprise-values.yaml of Prometheus (default: user)
password: the password set in the enterprise-values.yaml of Prometheus (default: password)

NodePort

30050 (if not set otherwise in the enterprise-values.yaml)

Connecting via SSH Tunnel

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/opensearch/.

OpenSearch Dashboards supports Single Sign-On (SSO) via Keycloak as the primary login, with username/password as a fallback.

SSO (recommended): open https://<your-domain>/opensearch/ and choose Log in with single sign-on. You are redirected to Keycloak. See the SSO for OpenSearch guide for setup and how to grant users access.
Username / password (fallback):
- username: admin
- password: the value of adminPassword in the enterprise-values.yaml (default: Password@@123456)

Note

SSO is only wired up when the OpenSearch hostname points to a reachable Keycloak (non-.local). For *.local hostnames only the username/password login is available. See Changing the OpenSearch password to change admin.

NodePort

HTTP: 30002
HTTPS: 30003

Connecting via SSH Tunnel

To establish a tunnel, forward port 30002 from one of the control planes to your local machine. You can use the command line, PuTTY, or MobaXterm for this.
After that, the dashboard can be accessed at localhost:30002/.

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/.

username: admin
password: the password set in the enterprise-values.yaml (default: password)

NodePort

The Rook/Ceph dashboard has no fixed NodePort. To find the NodePort in use, run one of the following:

kubectl get svc -n rook-ceph

Look for the rook-ceph-mgr-dashboard-external-http service:

NAME                                      TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)                                     AGE
rook-ceph-mgr-dashboard-external-http     NodePort    192.168.197.13    <none>        7000:31268/TCP                              21h

Or retrieve the NodePort directly:

echo $(kubectl get -n rook-ceph -o jsonpath="{.spec.ports[0].nodePort}" services rook-ceph-mgr-dashboard-external-http)

In the example above, the NodePort is 31268.

Connecting via SSH Tunnel

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/ceph-dashboard/.

echo Username: admin
echo Password: $(kubectl get secret rook-ceph-dashboard-password -n rook-ceph --template={{.data.password}} | base64 -d)

NodePort

30007 (if not set otherwise in the enterprise-values.yaml)

Connecting via SSH Tunnel

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/kubeops-dashboard/.

kubectl -n monitoring create token headlamp-admin

NodePort

30180

Connecting via SSH Tunnel

Connecting via Ingress

The dashboard can also be accessed via https://<your-domain>/keycloak/.

echo Username: $(kubectl get secret --namespace keycloak keycloak-kubeops -o jsonpath="{.data.ADMIN_USER}" | base64 -d)
echo Password: $(kubectl get secret --namespace keycloak keycloak-kubeops -o jsonpath="{.data.ADMIN_PASSWORD}" | base64 -d)

Kubeopsctl-2.0: Accessing Dashboards with OIDC

Mon, 01 Jan 0001 00:00:00 +0000

This how-to guide explains how to enable OIDC-based access to your Kubernetes application dashboards using kubeopsctl. You will configure a single enterprise-values.yaml file that defines the required components. Once the configuration is applied, the dashboards for Harbor, Grafana, Rook-Ceph, OpenSearch, Keycloak, the KubeOps dashboard, and Prometheus become accessible via their configured hostnames and paths. Following the steps in this guide, you will set up a consistent, centralized OIDC integration and make your dashboards securely available through a browser.

Prerequisites

yq should be installed on all master nodes. For more infomration refer yq installation guide.
Keycloak should be installed before other packages.
Pull all the required packages with ‘kubectl pull’ command. For more information refer Pull Packages with Kubeopsctl

Step 1: Configure enterprise-values.yaml

In order to connect to dashboad with OIDC, you need to configure your enterprise-values.yaml file as below

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1
deleteNs: false
localRegistry: false
packages:
- name: rook-ceph
  enabled: true
  values:
    standard:
      hostname: <domain_name>          # required for OIDC dashboard access
      namespace: rook-ceph
      cluster:
        resources:
          mgr:
            requests:
              cpu: "500m"
              memory: "512Mi"
            mon:
              requests:
                cpu: "1"
                memory: "1Gi"
            osd:
              requests:
                cpu: "1"
                memory: "1Gi"
          dashboard:
            enabled: "true"
        operator:
          data:
            rookLogLevel: "DEBUG"
- name: harbor
  enabled: true
  values:
    standard:
      namespace: harbor
      harborpass: "password"
      databasePassword: "password"
      redisPassword: "password"
      externalURL: http://10.2.10.110:30002
      nodePort: 30002
      hostname: <domain_name>          # required for OIDC dashboard access
      harborPersistence:
        persistentVolumeClaim:
          registry:
            size: 40Gi
            storageClass: "rook-cephfs"
          jobservice:
            jobLog:
              size: 1Gi
              storageClass: "rook-cephfs"
          database:
            size: 1Gi
            storageClass: "rook-cephfs"
          redis:
            size: 1Gi
            storageClass: "rook-cephfs"
          trivy: 
            size: 5Gi
            storageClass: "rook-cephfs"
    advanced:
- name: cert-manager 
  enabled: true
  values:
    standard:
      namespace: cert-manager
      replicaCount: 3
      logLevel: 2
    advanced:
    ca:      
      emailLetsEncrypt: <email>
      ingressName: nginx
- name: ingress-nginx
  enabled: true
  values:
    standard:
      namespace: ingress-nginx
      externalIPs: 
        - <>                 
      advanced:
- name: keycloak
  enabled: true
  values:
    standard:
      namespace: "keycloak"
      storageClass: "rook-cephfs"
      nodePort: "30180"
      hostname: <domain_name>          # required for OIDC dashboard access
      keycloak:
        auth:
          adminUser: admin
          adminPassword: topsecret
      postgresql:
        auth:
          postgresUserPassword: "changeme"
          username: bn_keycloak
          password: "changeme"
          database: bitnami_keycloak
        volumeSize: "8Gi"
    advanced:
- name: filebeat-os
  enabled: true
  values:
    standard:
      namespace: logging
    advanced:
- name: logstash-os
  enabled: true
  values:
    standard:
      namespace: logging
      volumeClaimTemplate:
        accessModes: 
          - ReadWriteMany
        resources:
          requests:
            storage: 1Gi
        storageClass: "rook-cephfs"
    advanced:
- name: opensearch-dashboards
  enabled: true
  values:
    standard:
      hostname: <domain_name>          # required for OIDC dashboard access 
      namespace: logging
      nodePort: 30050
    advanced:
- name: opensearch-os
  enabled: true
  values:
    standard:
      hostname: <domain_name>          # required for OIDC dashboard access
      namespace: logging
      opensearchJavaOpts: "-Xmx512M -Xms512M" # optional, default is -Xmx512M -Xms512M
      resources:
        requests:
          cpu: "250m" # optional, default is 250m
          memory: "1024Mi" # optional, default is 1024Mi
        limits:
           cpu: "300m" # optional, default is 300m
           memory: "3072Mi" # optional, default is 3072Mi
      persistence:
        size: 4Gi # mandatory
        enabled: "true" # optional, default is true
        enableInitChown: "false" # optional, default is false
        labels:
          enabled: "false" # optional, default is false
        storageClass: "rook-cephfs" # optional, default is rook-cephfs
        accessModes:
          - "ReadWriteMany" # optional, default is {ReadWriteMany}
        securityConfig:
          enabled: "false" # optional, default value: false
        replicas: "3" # optional, default is 3
    advanced:
- name: kube-prometheus-stack
  enabled: true
  values:
    standard:
      namespace: monitoring
      grafanaUsername: admin
      grafanaPassword: topsecret
      retentionSize: "24GB"
      grafanaResources:
        hostname: <domain_name>          # required for OIDC dashboard access
        nodePort: 30211
        retention: 10d
        retentionSize: "24GB"
        storageClass: "rook-cephfs"
        storage: 25Gi
      prometheusResources:
        hostname: <domain_name>          # required for OIDC dashboard access
        retentionSize: "24GB"
        storageClass: "rook-cephfs"
        storage: 25Gi
    advanced:
- name: opa-gatekeeper
  enabled: true
  values:
    standard:
      namespace: opa-gatekeeper
    advanced:
- name: kubeops-dashboard
  enabled: true
  values:
    standard:
      namespace: monitoring
      hostname: <domain_name>          # required for OIDC dashboard access
      service:
        nodePort: 30007
    advanced:
- name: velero
  enabled: true
  values:
    standard:
      namespace: "velero"
      accessKeyId: "your_s3_storage_username"
      secretAccessKey: "your_s3_storage_password"
      useNodeAgent: false
      defaultVolumesToFsBackup: false
      provider: "aws"
      bucket: "velero"
      useVolumeSnapshots: false
      backupLocationConfig:
        region: "minio"
        s3ForcePathStyle: true
        s3Url: "http://minio.velero.svc:9000"
    advanced:

step 2: Apply the Configuration with ‘KubeOpctl’

Apply the enterprise-values.yaml configuration using the following command:

kubeopsctl apply -f enterprise-values.yaml

Wait until all components are deployed and running before proceeding.

Step 3: Connect to Dashboards via Hostname

To access the dashboards via OIDC, use the hostnames configured in your enterprise-values.yaml (uservalues). Harbor is an exception: you can access it directly using the base domain name.

Use the following paths, replacing <domain_name> with your configured domain:

<domain_name>/ for Harbor
<domain_name>/grafana for grafana
<domain_name>/ceph-dashboard for rook-ceph
<domain_name>/opensearch for opensearch
<domain_name>/keycloak for keycloak
<domain_name>/kubeops-dashboard for Kubeops Dashboard
<domain_name>/prometheus for Prometheus

Kubeopsctl-2.0: Changing a User Password in OpenSearch

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to change passwords in OpenSearch.

Recommended: Change the `admin` password via the package value

The OpenSearch admin password is managed through the adminPassword package value. Set it for the opensearch-os package (and the matching opensearch-dashboards package) in your enterprise-values.yaml and run an update:

packages:
  - name: opensearch-os
    enabled: true
    values:
      standard:
        adminPassword: "<your-new-password>" # must match opensearch-dashboards
  - name: opensearch-dashboards
    enabled: true
    values:
      standard:
        adminPassword: "<your-new-password>" # must match opensearch-os

On the next kubeopsctl apply/update the new password is enforced automatically after the deployment. This also works on existing clusters and does not remove any other internal users.

Note Internal users created in the OpenSearch Dashboards Security UI persist across package updates. The update only enforces the admin password and reapplies the authentication/authorization configuration; it never overwrites your internal users.

Advanced: Change any user’s password manually via `securityadmin`

Use this method to change the password of any internal user (for example testuser or kibanaro) without a package update.

This process reads the live internal users from the security index (via securityadmin -backup), so every other user — including users you created in the OpenSearch Dashboards Security UI — is preserved. Do not build the file from the internal-users-config-secret: that secret only holds the packaged baseline users and would remove UI-created users when applied.

Prerequisites

Access to the Kubernetes cluster and kubectl.
The admin certificate is available in the pod at /usr/share/opensearch/config/certs/ (root-ca.pem, admin.pem, admin-key.pem) in a KubeOps deployment.

Set a shell variable for convenience (adjust the namespace and pod name as needed):

NS=logging; POD=opensearch-cluster-master-0

Step 1: Back up the current internal users (live state)

kubectl exec -it $POD -n $NS -- bash -c "\
    /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh -backup /tmp/os-sec \
    -icl -nhnv \
    -cacert /usr/share/opensearch/config/certs/root-ca.pem \
    -cert /usr/share/opensearch/config/certs/admin.pem \
    -key /usr/share/opensearch/config/certs/admin-key.pem"

This writes the current configuration — including all internal users — to /tmp/os-sec/ in the pod.

Note securityadmin -backup may end with Configuration for 'audit' failed because of empty source and a non-zero exit code. This is harmless: internal_users.yml is written before that step, so the file you need is already present.

Step 2: Generate a new password hash for each user

Run this once per user and note the resulting hash:

kubectl exec -it $POD -n $NS -- bash -c "sh /usr/share/opensearch/plugins/opensearch-security/tools/hash.sh -p <new_password>"

Step 3: Update the hashes in the backup file

Copy the file out, edit it locally, and copy it back:

kubectl cp $NS/$POD:/tmp/os-sec/internal_users.yml ./internal_users.yml
# edit ./internal_users.yml: replace the hash: value under each user you want to change
kubectl cp ./internal_users.yml $NS/$POD:/tmp/os-sec/internal_users.yml

Only change the hash: line of the target users; leave everyone else untouched:

testuser:
  hash: "<new-hash-for-testuser>"
  reserved: false
  backend_roles:
  - "testrole"
kibanaro:
  hash: "<new-hash-for-kibanaro>"
  reserved: false
  backend_roles:
  - "kibanauser"

Step 4: Apply only the internal users configuration

Note

The DN of the certificate specified with -cert must be listed under plugins.security.authcz.admin_dn in opensearch.yml. In a KubeOps deployment the admin certificate subject is C=DE,L=test,O=client,OU=client,CN=admin.

More information: Opensearch Documentation.

Apply only the internalusers config type. Do not use -cd (the whole directory) here: -cd requires the complete security config set (including nodes_dn.yml, allowlist.yml, audit.yml), which a KubeOps deployment does not ship, so it would fail with nodes_dn.yml ... No such file or directory.

kubectl exec -it $POD -n $NS -- bash -c "\
    /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh \
    -f /tmp/os-sec/internal_users.yml -t internalusers \
    -icl -nhnv \
    -cacert /usr/share/opensearch/config/certs/root-ca.pem \
    -cert /usr/share/opensearch/config/certs/admin.pem \
    -key /usr/share/opensearch/config/certs/admin-key.pem"

The change is written to the security index and takes effect immediately — no pod restart required.

Important

Once applied with securityadmin.sh, credentials are stored in the OpenSearch security index, which lives on the persistent volume and therefore survives pod and container restarts as well as package updates. Editing files inside the container without running securityadmin.sh only changes the local file and is not applied to the cluster. For the admin user, prefer the adminPassword package value described above — it is enforced again on every update and keeps all other internal users intact.

Kubeopsctl-2.0: Backup and restoring artifacts

Mon, 01 Jan 0001 00:00:00 +0000

What is Velero?

Velero uses object storage to store backups and associated artifacts. It also optionally integrates supported block storage systems to snapshot your persistent volumes. Before beginning the installation process, you should identify the object storage provider and optional block storage provider(s) you’ll be using from the list of compatible providers.

Velero supports storage providers for both cloud-provider environments and on-premises environments.

Velero prerequisites:

Access to a Kubernetes cluster, v1.16 or later, with DNS and container networking enabled.
kubectl installed locally
Object Storage (S3, Cloud Provider Environment, On-Premises Environment)

Note : Compatible providers and on-premises documentation can be read on https://velero.io/docs

Install Velero

This command is an example on how you can install velero into your cluster:

velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.2.1 --bucket velero --secret-file ./credentials-velero --use-volume-snapshots=false --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000

NOTE:

s3Url has to be the url of your s3 storage login.

example for credentials-velero file:

[default]
aws_access_key_id = your_s3_storage_username
aws_secret_access_key = your_s3_storage_password

Backup the cluster

Scheduled Backups

This command creates a backup for the cluster every 6 hours:

velero schedule create cluster --schedule "0 */6 * * *"

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete cluster

Restore Scheduled Backup

This command restores the backup according to a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the cluster

velero backup create cluster

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Backup a specific deployment

Scheduled Backups

This command creates a backup for the namespace “logging” every 6 hours:

velero schedule create filebeat --schedule "0 */6 * * *" --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “filebeat” every 6 hours:

velero schedule create filebeat --schedule "0 */6 * * *" --include-namespaces logging --selector app=filebeat-filebeat,release=filebeat --include-resources serviceaccount,deployment,daemonset,configmap,clusterrolebinding,clusterrole --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete filebeat

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “logging”:

velero backup create filebeat --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “filebeat”:

velero backup create filebeat --include-namespaces logging --selector app=filebeat-filebeat,release=filebeat --include-resources serviceaccount,deployment,daemonset,configmap,clusterrolebinding,clusterrole --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “logging” every 6 hours:

velero schedule create logstash --schedule "0 */6 * * *" --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “logstash” every 6 hours:

velero schedule create logstash --schedule "0 */6 * * *" --include-namespaces logging --selector app=logstash-logstash,chart=logstash,release=logstash --include-resources StatefulSet,ServiceAccount,Service,Secret,RoleBinding,Role,PodSecurityPolicy,PodDisruptionBudget,Ingress,ConfigMap --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete logstash

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “logging”:

velero backup create logstash --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “logstash”:

velero backup create logstash --include-namespaces logging --selector app=logstash-logstash,chart=logstash,release=logstash --include-resources StatefulSet,ServiceAccount,Service,Secret,RoleBinding,Role,PodSecurityPolicy,PodDisruptionBudget,Ingress,ConfigMap --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “logging” every 6 hours:

velero schedule create opensearch --schedule "0 */6 * * *" --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “opensearch” every 6 hours:

velero schedule create opensearch --schedule "0 */6 * * *" --include-namespaces logging --selector app.kubernetes.io/instance=opensearch,app.kubernetes.io/name=opensearch --include-resources ConfigMap,Ingress,NetworkPolicy,PodDisruptionBudget,PodSecurityPolicy,Role,RoleBinding,Secret,Service,ServiceAccount,StatefulSet --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete opensearch

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “logging”:

velero backup create opensearch --include-namespaces logging --include-cluster-resources=true

This command creates a backup for the deployment “opensearch”:

velero backup create opensearch --include-namespaces logging --selector app.kubernetes.io/instance=opensearch,app.kubernetes.io/name=opensearch --include-resources ConfigMap,Ingress,NetworkPolicy,PodDisruptionBudget,PodSecurityPolicy,Role,RoleBinding,Secret,Service,ServiceAccount,StatefulSet --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “monitoring” every 6 hours:

velero schedule create prometheus --schedule "0 */6 * * *" --include-namespaces monitoring --include-cluster-resources=true

This command creates a backup for the deployment “prometheus” every 6 hours:

velero schedule create prometheus --schedule "0 */6 * * *" --include-namespaces monitoring --include-resources Alertmanager,Secret,Ingress,List,PodDisruptionBudget,Role,RoleBinding,PodSecurityPolicy,Service,ServiceAccount,ServiceMonitor,Endpoints,ConfigMap,ConfigMapList,ClusterRole,ClusterRoleBinding,SecretProviderClass,PodMonitor,Prometheus,Job,NetworkPolicy,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,Issuer,Deployment,VerticalPodAutoscaler,ThanosRuler --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete prometheus

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “monitoring”:

velero backup create prometheus --include-namespaces monitoring --include-cluster-resources=true

This command creates a backup for the deployment “prometheus”:

velero backup create prometheus --include-namespaces monitoring --include-resources Alertmanager,Secret,Ingress,List,PodDisruptionBudget,Role,RoleBinding,PodSecurityPolicy,Service,ServiceAccount,ServiceMonitor,Endpoints,ConfigMap,ConfigMapList,ClusterRole,ClusterRoleBinding,SecretProviderClass,PodMonitor,Prometheus,Job,NetworkPolicy,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,Issuer,Deployment,VerticalPodAutoscaler,ThanosRuler --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “harbor” every 6 hours:

velero schedule create harbor --schedule "0 */6 * * *" --include-namespaces harbor --include-cluster-resources=true

This command creates a backup for the deployment “harbor” every 6 hours:

velero schedule create harbor --schedule "0 */6 * * *" --include-namespaces harbor --include-resources ConfigMap,Deployment,PersistentVolumeClaim,Secret,Service,StatefulSet,Ingress,ServiceMonitor --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete harbor

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “harbor”:

velero backup create harbor --include-namespaces harbor --include-cluster-resources=true

This command creates a backup for the deployment “harbor”:

velero backup create harbor --include-namespaces harbor --include-resources ConfigMap,Deployment,PersistentVolumeClaim,Secret,Service,StatefulSet,Ingress,ServiceMonitor --include-cluster-resources=true --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “gatekeeper-system” every 6 hours:

velero schedule create gatekeeper --schedule "0 */6 * * *" --include-namespaces gatekeeper-system --include-cluster-resources=true

This command creates a backup for the deployment “gatekeeper” every 6 hours:

velero schedule create gatekeeper --schedule "0 */6 * * *" --include-namespaces gatekeeper-system --include-resources PodSecurityPolicy,ServiceAccount,Deployment,PodDisruptionBudget,ResourceQuota,ClusterRole,Role,ClusterRoleBinding,RoleBinding,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,Secret,Service,Job --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete gatekeeper

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “gatekeeper-system”:

velero backup create gatekeeper --include-namespaces gatekeeper-system --include-cluster-resources=true

This command creates a backup for the deployment “gatekeeper-system”:

velero backup create gatekeeper --include-namespaces gatekeeper-system --include-resources PodSecurityPolicy,ServiceAccount,Deployment,PodDisruptionBudget,ResourceQuota,ClusterRole,Role,ClusterRoleBinding,RoleBinding,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,Secret,Service,Job --include-cluster-resources=true --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

Scheduled Backups

This command creates a backup for the namespace “rook-ceph” every 6 hours:

velero schedule create rook-ceph --schedule "0 */6 * * *" --include-namespaces rook-ceph --include-cluster-resources=true

Get Schedules

This command lists all schedules for backups:

velero schedule get

Delete Schedules

This command deletes the specified schedule:

velero schedule delete rook-ceph

Restore Scheduled Backup

This command restores the backup from a schedule:

velero restore create --from-backup <SCHEDULE NAME>-<TIMESTAMP>

Backup

This command creates a backup for the namespace “rook-ceph”:

velero backup create rook-ceph --include-namespaces rook-ceph --include-cluster-resources=true

Get Backups

This command lists all created backups:

velero backup get

Delete Backups

This commands deletes the specified backup:

velero backup delete <BACKUP NAME>

Restore Backup

This commands restores the specified backup:

velero restore create <RESOURCE NAME> --from-backup <BACKUP NAME>

restore databases

keycloak

create backup of keycloak namespace, in this example the backup is called keycloak1.

This command creates a backup for the namespace “rook-ceph”:

velero backup create keycloak1 --include-namespaces keycloak --include-cluster-resources=true

restore backup, in this example keycloak1

velero restore create keycloak1 --from-backup keycloak1

restore the database dump:

kubectl -n <keycloak-namespace> exec keycloak-postgres-0 -- pg_restore -v --jobs=4 --clean --if-exists -d bitnami_keycloak /backup/keycloak-db.dump

Kubeopsctl-2.0: About KubeOps Compliance

Mon, 01 Jan 0001 00:00:00 +0000

What is KubeOps Compliance?

kubeopsctl serves as a versatile utility designed specifically to efficiently manage both the configuration and status of a cluster.

With its capabilities, users can articulate their desired cluster state in detail, outlining configurations and specifications.

Subsequently, kubeopsctl orchestrates the creation of a cluster that precisely matches the specified state, ensuring alignment between intentions and operational reality.

Why use KubeOps Compliance?

In kubeopsctl, configuration management involves defining, maintaining, and updating the desired state of a cluster, including configurations for nodes, pods, services, and other resources in the application environment.

The main goal of kubeopsctl is to match the clusters actual state with the desired state specified in the configuration files. With a declarative model, kubeopsctl enables administrators to express their desired system setup, focusing on „what“ they want rather than „how“ to achieve it.

This approach improves flexibility and automation in managing complex systems, making the management process smoother and allowing easy adjustment to changing needs.

kubeopsctl uses YAML files to store configuration data in a human-readable format. These files document important metadata about the objects managed by kubeopsctl, such as pods, services, and deployments.

Highlights

creating a cluster
adding nodes to your cluster
drain nodes
updating single nodes
label nodes for zones
adding platform software into your cluster

Kubeopsctl: About KubeOps Compliance

Mon, 01 Jan 0001 00:00:00 +0000

What is KubeOps Compliance?

kubeopsctl serves as a versatile utility designed specifically to efficiently manage both the configuration and status of a cluster.

With its capabilities, users can articulate their desired cluster state in detail, outlining configurations and specifications.

Subsequently, kubeopsctl orchestrates the creation of a cluster that precisely matches the specified state, ensuring alignment between intentions and operational reality.

Why use KubeOps Compliance?

This approach improves flexibility and automation in managing complex systems, making the management process smoother and allowing easy adjustment to changing needs.

Highlights

creating a cluster
adding nodes to your cluster
drain nodes
updating single nodes
label nodes for zones
adding platform software into your cluster

Kubeopsctl-2.0: Add certificate as trusted

Mon, 01 Jan 0001 00:00:00 +0000

1. Download the certificate

As soon as Chrome issues a certificate warning, click on Not secure to the left of the address bar.
Show the certificate (Click on Certificate is not valid).
Go to Details tab.
Click Export... at the bottom and save the certificate.

As soon as Firefox issues a certificate warning, click on Advanced....
View the certificate (Click on View Certificate).
Scroll down to Miscellaneous and save the certificate.

2. Install the certificate

Press Windows + R.
Enter mmc and click OK.
Click on File > Add/Remove snap-in....
Select Certificates in the Available snap-ins list and click on Add >, then on OK. Add the snap-in.
In the tree pane, open Certificates - Current user > Trusted Root Certification Authorities, then right-click Certificates and select All tasks > Import....
The Certificate Import Wizard opens here. Click on Next.
Select the previously saved certificate and click Next.
Click Next again in the next window.
Click on Finish. If a warning pops up, click on Yes.
The program can now be closed. Console settings do not need to be saved.
Clear browser cache and restart browser.

The procedures for using a browser to import a certificate as trusted (on Linux systems) vary depending on the browser and Linux distribution used. To manually cause a self-signed certificate to be trusted by a browser on a Linux system:

Distribution	Copy certificate here	Run following command to trust certificate
RedHat	`/etc/pki/ca-trust/source/anchors/`	`update-ca-trust extract`

Note

If the directory does not exist, create it.

Note

If you do not have the ca-certificates package, install it with your package manager.

Kubeopsctl-2.0: Deploy Package on a cluster

Mon, 01 Jan 0001 00:00:00 +0000

You can install artifacts in your cluster in several ways. When creating a package, you can use one of the following plugins:

helm
kubectl
cmd
kosi

This guide demonstrates a complete, reproducible deployment of nginx-ingress using Helm and kosi.

Using the Helm Plugin

Pre-requisites

Before you begin, make sure the following tools are available in your environment:

helm
kosi

This example uses the official NGINX Helm repository and deploys nginx-ingress with Helm and KOSI.

Prepare the Files

In order to install an artifact with the Helm plugin, the Helm chart must first be downloaded.

To keep the environment clean and reproducible, first create a dedicated working directory and change into it:

mkdir nginx-ingress-kosi && cd nginx-ingress-kosi

The example uses the official NGINX Helm repository. Add the repository before performing any Helm operations:

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update

To verify which chart versions are available, run:

helm search repo nginx-stable/nginx-ingress --versions

This guide uses chart version 0.16.1. Download the chart as a packaged Helm artifact:

helm pull nginx-stable/nginx-ingress --version 0.16.1

This command creates the following file in your working directory:

nginx-ingress-0.16.1.tgz

Create KOSI package

Once the chart is available locally, initialize the kosi package structure. The following command creates the necessary files in the current directory:

kosi create

The Helm chart remains the central input artifact for deployment, while configuration is handled via values.yaml. Helm already includes default values, which can be inspected or exported if needed:

helm show values nginx-stable/nginx-ingress --version 0.16.1 > values.yaml

Review and update values.yaml before packaging and deploying the chart. For example, in environments without a cloud-based LoadBalancer, you may need to change the service type in values.yaml.

controller:
  service:
    type: LoadBalancer

Only required overrides should be defined in the customized values.yaml file. To customize the deployment of the Helm chart, a new values.yaml file in the current directory must be created and edited with this content:

controller:
  service:
    type: NodePort

This override is merged with the chart defaults at runtime.

Helm charts do not directly contain the full list of container images required for deployment. To identify them, render the chart into Kubernetes manifests:

helm template nginx-ingress nginx-stable/nginx-ingress --version 0.16.1 > manifests.yaml

Then extract all referenced images from the rendered manifest:

grep image: manifests.yaml

For this example, the required image is:

nginx/nginx-ingress:3.0.1

With all artifacts identified, the package.kosi file defines what is required for deployment. It explicitly references:

- the Helm chart archive
- the custom values.yaml
- all required container images

nginx-ingress-0.16.1.tgz
values.yaml
package.kosi

Note

The downloaded Helm chart must also be located in the current directory. To customize the deployment of the Helm chart, the values.yaml file must be edited.

All files required by a task in the package must be named in the package.kosi file under files. The container images required by the Helm chart must also be listed in the package.kosi under containers.

In the example below, only two files are required for the installation: the Helm Chart for the nginx-ingress and the values.yaml to configure the deployment. To install nginx-ingress you will also need the nginx/nginx-ingress image with the tag 3.0.1.

To install nginx-ingress with the Helm plugin, call the plugin as shown in the example under install. The deployment configuration file is listed under values and the packed Helm chart is specified with the key tgz. Furthermore, it is also possible to specify the namespace in which the artifact should be deployed and the name of the deployment. The full documentation for the Helm plugin can be found here.

Use the following configuration:

languageversion = "1.0.0";
apiversion = "kubernative/kubeops/sina/user/v4";
name = "deployexample1";
description = "It shows how to deploy an artifact to your cluster using the helm plugin.";
version = "0.1.0";
docs = "docs.tgz";
logo = "logo.png";

files =
{
        valuesFile = "values.yaml";
        nginxHelmChart="nginx-ingress-0.16.1.tgz";
}

containers =
{
        nginx = ["docker.io", "nginx/nginx-ingress", "3.0.1"];
}

install
{
        helm
        (
            command = "install";
            tgz = "nginx-ingress-0.16.1.tgz";
            values = "['values.yaml']";
            namespace = "dev";
            deploymentName = "nginx-ingress"
        );
}

Build the Package

Once the package.kosi file has been fully configured, all files must be combined into a kosi package. To do this, execute the following command in the directory where the package.kosi file is located.

kosi build

Push the Package

To make the generated kosi package available on other machines, it is pushed to the user’s private KubeOps Hub. To do this, the user must first log in to the hub and then push it to hub.

$ kosi login -u <username>
2023-02-04 11:19:43 Info:      KOSI version: 2.13.0
2023-02-04 11:19:43 Info:      Please enter password
****************
2023-02-04 11:19:26 Info:      Login Succeeded to Hub.
$ kosi push --hub kosi
2023-02-04 11:23:18 Info:      KOSI version: 2.13.0
2023-02-04 11:23:19 Info:      Push to Private Registry registry.kubeops.net/<username>/

Deploy the Package

Once the KOSI package has been created and published, it needs to be installed on the admin node. The following command will download and execute the package. The package name and version refer to the values defined in package.kosi with the keys name and version.

kosi install --hub <username> <username>/<packagename>:<version>

For the example package, the command would be: kosi install --hub <username> <username>/deployExample:0.1.0.

Using the Kubectl Plugin

Prerequisites

Before you begin, make sure the following tools and components are available:

kosi
the kubeops-kubernetes-plugins package installed on the admin node

This example deploys nginx-ingress from a Kubernetes manifest using the Kubectl plugin.

Create KOSI package

First you need to create a KOSI package. The following command creates the necessary files in the current directory:

kosi create

The NGINX ingress controller YAML manifest can either be automatically downloaded and applied directly with kubectl apply or it can be downloaded manually if you want to customize the deployment. The YAML manifest can be downloaded from the NGINX GitHub Repo and must be placed in the same directory as the files for the KOSI package.

All files required by a task in the package must be named in the package.kosi file under files. The container images required by the YAML manifest must also be listed in the package.kosi under containers. In the example below, only one file is required for the installation: the YAML manifest for the nginx-ingress controller. To install nginx-ingress you will also need the registry.k8s.io/ingress-nginx/controller image with the tag v1.5.1 and the image registry.k8s.io/ingress-nginx/kube-webhook-certgen with tag v20220916-gd32f8c343.

To install nginx-ingress with the Kubectl plugin, call the plugin as shown in the example under installs. The full documentation for the Kubectl plugin can be found here.

languageversion = "1.0.0";
apiversion = "kubernative/kubeops/sina/user/v4";
name = "deployexample2";
description = "It shows how to deploy an artifact to your cluster using the helm plugin.";
version = "0.1.0";
docs = "docs.tgz";
logo = "logo.png";

files =
{
     manifest: "deploy.yaml"
}

containers =
{
    nginx = ["registry.k8s.io", "ingress-nginx/controller", "v1.5.1"];
    certgen= ["registry.k8s.io","ingress-nginx/kube-webhook-certgen","v20220916-gd32f8c343"];
}

install
{
    kubectl
    (
      operation="apply",
      flags="-f deploy.yaml";
      sudo = true;
      sudoPassword="toor"
    );
}

Build the Package

Once the package.kosi file has been fully configured, all files must be combined into a KOSI package. To do this, execute the following command in the directory where the package.kosi file is located.

kosi build

Publish the Package

To make the generated KOSI package available on other machines, it is pushed to the user’s private KubeOps Hub. To do this, the user must first log in to the hub.

$ kosi login -u <username>
2023-02-04 11:19:43 Info:      kosi version: 2.13.0
2023-02-04 11:19:43 Info:      Please enter password
****************
2023-02-04 11:19:26 Info:      Login Succeeded to Hub.
$ kosi push --hub kosi
2023-02-04 11:23:18 Info:      kosi version: 2.13.0
2023-02-04 11:23:19 Info:      Push to Private Registry registry.kubeops.net/<username>/

Deploy the Package

kosi install --hub <username> <username>/<packagename>:<version>

For the example package, the command would be: kosi install --hub <username> <username>/deployExample:0.1.0.

Kubeopsctl: Getting-Started

Mon, 01 Jan 0001 00:00:00 +0000

Kubeopsctl: KubeOps Compliance

Mon, 01 Jan 0001 00:00:00 +0000

Try kubeopsctl

Follow quickstart guides to learn how to setup secure clusters.

Kubeopsctl-2.0: KubeOps Compliance 2.0

Mon, 01 Jan 0001 00:00:00 +0000

Start with Kubeops Compliance 2.0

Follow the quickstart guides to learn how to create cluster in a secure, fast way with kubeopsctl.

Kubeopsctl-2.0: Software Bill of Materials (SBOM)

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps publishes a Software Software Bill of Materials (SBOM) for the kubeopsctl command-line tooling that ships in the package repositories. It lists every bundled component with its exact version and license, in CycloneDX format. The published document is the authoritative source — link to it rather than copying its contents, so the list never drifts out of date.

Download

The SBOM is served at a fixed path in each package repository. Both repositories ship the same SBOM document, so the only difference is the retrieval URL for your platform.

curl -fsSLO https://packagerepo.kubeops.net/rpm/sbom/kubeopsctl-2.2.0.0_Beta1-0.x86_64.xml

You can checkout all SBOMs here sbom

curl -fsSLO https://packagerepo.kubeops.net/deb/pool/main/sbom/kubeopsctl-2.2.0.0_Beta1-0.x86_64.xml

You can checkout all SBOMs here sbom

Using the SBOM

The CycloneDX document can be fed directly into SBOM-aware vulnerability scanners and license-compliance tooling without re-resolving dependencies, for example:

grype sbom:./kubeopsctl-bom.xml   # vulnerability scan
trivy sbom ./kubeopsctl-bom.xml   # vulnerability scan

Scanner output reflects advisories at the time you run the scan, not at the time the SBOM was generated. Re-run scans regularly rather than relying on a single point-in-time result.

Kubeopsctl-2.0: How to migrate from nginx to traefik ingress

Mon, 01 Jan 0001 00:00:00 +0000

Installation

Kubeops supports deploying Traefik as a dynamic ingress controller and reverse proxy. This guide describes a concise, safe migration from an existing nginx-ingress controller to Traefik and explains how to install Traefik and replace a deprecated nginx-ingress deployment. The migration from nginx to Traefik is straightforward; the steps below show the process in order.

Prerequisites

A running Kubernetes cluster with an existing nginx-ingress controller.
It is recommended to do a login with kosi. Refer to the official KOSI documentation for details here.

1.Create Values file

Create a values.yaml file for the Traefik installation:

# values.yaml
packages:
- name: traefik
  enabled: true
  values:
    standard:
      namespace: traefik
      externalIPs: []
    advanced: {}

Note: Update externalIPs and any other values as required for your environment.

2.Install Traefik

non-airgap-environment

After creating values.yaml, install Traefik:

# get your desired version/s
kosi search --hub kubeops --ps traefik
# install traefik
kosi install --hub kubeops kubeops/traefik:<desired_version> -f values.yaml --dname traefik

Example Non-airgap

kosi install --hub kubeops kubeops/traefik:2.1.0_Beta0 -f values.yaml --dname traefik

Airgap-environment

# log in to Harbor
podman login <ip_address>:<NodePort> -u <username> -p <password> --tls-verify=false
# pull your desired version
kosi pull --hub kubeops kubeops/<desired_version> -o traefik-<desired_version>.tgz -r <ip_address>:<NodePort>/kubeops -t 127.0.0.1:<NodePort>/kubeops
 # install Traefik
kosi install --dname traefik -p traefik-:<desired_version>.tgz -f values.yaml

Example Airgap

kosi pull --hub kubeops kubeops/traefik:2.1.0_Beta0 -o traefik-2.1.0-Beta0.tgz -r 10.2.10.11:30002/kubeops -t 127.0.0.1:30002/kubeops
 
kosi install --dname traefik -p traefik-2.1.0-Beta0.tgz -f values.yaml

3.Verify Deployment

Verify that the Traefik pods and services are running in the traefik namespace:

kubectl get pods -n traefik
kubectl get svc -n traefik

3b. Switch applications to the Traefik ingress class

This is the load-bearing step of the migration. Traefik ships a backwards-compatibility shim (an IngressClass named nginx plus the experimental providers.kubernetesIngressNginx provider) so that existing ingressClassName: nginx ingresses should keep working unchanged. This shim is experimental and does not work reliably for all workloads — Keycloak in particular has been observed to be unreachable through the nginx class while Traefik is the controller. Do not rely on the shim as the primary mechanism; switch each application’s ingress to ingressClassName: traefik.

Every ingress package exposes a uniform standard.ingressClassName key (defaulting to nginx). Set it to traefik on each package, and set the cert-manager solver class:

- name: keycloak
  values:
    standard:
      ingressClassName: traefik
- name: kubeOpsDashboard
  values:
    standard:
      ingressClassName: traefik
- name: kube-prometheus-stack
  values:
    standard:
      ingressClassName: traefik
- name: harbor
  values:
    standard:
      ingressClassName: traefik
- name: rook-ceph
  values:
    standard:
      ingressClassName: traefik
- name: opensearch-dashboards
  values:
    standard:
      ingressClassName: traefik
- name: kubevirt-manager
  values:
    standard:
      ingressClassName: traefik
- name: cert-manager
  values:
    ca:
      ingressName: traefik         # ACME HTTP01 solver ingress class

cert-manager is the only exception to the standard.ingressClassName key: its solver class lives under values.ca.ingressName. Certificates and the cert-manager.io/cluster-issuer annotations are otherwise controller-agnostic.

Note: the plain OpenSearch data package (opensearch-os) and logstash ship their ingress disabled by default, so they need no change. If you enable them, set the class via that package’s advanced block (ingress.ingressClassName / ingress.className).

Apply the updated values and confirm every ingress now uses the Traefik class:

kubectl get ingress -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.ingressClassName

SSO / certificates / dashboards: SSO is app-level OIDC (Keycloak reachable at https://<host>/keycloak), so it is not tied to the ingress controller and keeps working once the class is switched. The same is true for cert-manager and dashboard access.

nginx-only annotations do not migrate. Annotations of the form nginx.ingress.kubernetes.io/* are ignored by Traefik. OpenSearch Dashboards ships buffer/ header-size annotations (proxy-buffer-size, large-client-header-buffers, …) that matter for large OIDC headers; if login fails with 4xx/431/502, add an equivalent Traefik buffering middleware / entrypoint transport setting via the package’s advanced block. Harbor auto-injects nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" when internalTLS.enabled — verify Harbor over TLS after the switch.

4.Remove old nginx-ingress deployment and service

# get version of installed nginx-ingress and its deployment name (--dname)
kosi list
# delete old nginx-ingress
kosi delete --hub kubeops kubeops/ingress-nginx:<installed_version> -f enterprise-values.yaml --dname <kosi_deployment_name>

Example


kosi delete --hub kubeops ingress-nginx:2.1.0_Beta0 -f values.yaml --dname ingress-nginx

Clean up the leftover nginx IngressClass. The ingress-nginx chart annotates its IngressClass with helm.sh/resource-policy: keep, so it survives kosi delete. Both ingress-nginx and the Traefik package define an IngressClass named nginx for controller k8s.io/ingress-nginx, so they must not run at the same time. After removing ingress-nginx, delete any leftover class if it is no longer wanted:
kubectl get ingressclass
kubectl delete ingressclass nginx   # only if you no longer need the nginx compat class

Edit Traefik service

If nginx-ingress used specific NodePorts that you need to reuse, update the Traefik Service:

kubectl edit svc traefik -n traefik

Note: Default NodePorts (for example, 31080 / 31443) might not be reachable in your environment. If these ports are not accessible, determine the NodePorts previously used by nginx-ingress (for example, 30080 / 30443) and configure Traefik to use the same ports.

Update the ports:

Adjust the ports section to match the previous nginx NodePorts if required:

ports:
- name: web
  nodePort: 30080
  port: 80
  targetPort: web
- name: websecure
  nodePort: 30443
  port: 443
  targetPort: websecure

Verify Port Change

kubectl get svc -n traefik

Note: Ensure that the nginx Service is removed or that its NodePorts are freed before reusing those NodePorts on the Traefik Service.

KubeOpsDoku – kubeopsctl

Kubeopsctl-2.0: KubeOpsCtl CLI Commands

General commands

Overview of all KUBEOPSCTL commands

Command ‘kubeopsctl –version’

Command ‘kubeopsctl –help’

Command ‘kubeopsctl login’

Command ‘kubeopsctl logout’

Command ‘kubeopsctl pull’

Command ‘kubeopsctl apply’

-f flag

–delete flag

Kubeopsctl-2.0: Joining a Node in a Kubernetes cluster

Join Node Process:

Prerequisits

KOSI Login Recommendation

ETCD Backup Recommendation

Example 1: Joining a Control-Plane Node to a Kubernetes Cluster

3. Validate your values and join the node to the cluster

Example 1: Joining a Worker Node to a Kubernetes Cluster

3. Validate your values and join the node to the cluster

Kubeopsctl-2.0: KubeOps Versions

Kubeopsctl: Commands

KubeOps kubeopsctl

General commands

Overview of all KUBEOPSCTL commands

Command ‘kubeopsctl –version’

Command ‘kubeopsctl –help’

Command ‘kubeopsctl pull’

Command ‘kubeopsctl apply’

-f flag

-l flag

Command ‘kubeopsctl change registry’

-f flag

-r flag

-t flag

Command ‘kubeopsctl drain’

Command ‘kubeopsctl uncordon’

Command ‘kubeopsctl upgrade’

-v flag

Command ‘kubeopsctl status’

Kubeopsctl-2.0: Deleting a Node from a Kubernetes cluster

Deleting a Node from a Kubernetes cluster

Prerequisites

Worker

Scale down the rook-ceph-operator deployment to 0

Check which hosts and OSDs belong to each zone

Scale down the OSD deployment

Remove the OSD via ceph-tools

Check the MON running on the node

Delete OSD and MON deployments

Remove the deleted mon from the ceph tools

Delete the node from the kubernetes cluster

Scale the rook-ceph-operator deployment back to 1

Timing and health checks

Kubeopsctl: Documentation-kubeopsctl

KubeOps kubeopsctl

General commands

Overview of all KUBEOPSCTL commands

Command ‘kubeopsctl –version’

Command ‘kubeopsctl –help’

Command ‘kubeopsctl apply’

-f flag

-l flag

Command ‘kubeopsctl change registry’

-f flag

-r flag

-t flag

Command ‘kubeopsctl drain’

Command ‘kubeopsctl uncordon’

Command ‘kubeopsctl upgrade’

-v flag

Command ‘kubeopsctl status’

Prerequisites

Requirements on admin

Warning

Requirements for each node

How to Configure Cluster/Nodes/Software using yaml file

Full yaml syntax

how to use kubeopsctl

2. Update the target version in `cluster-values.yaml`

2. Adjust your cluster-values in `zone1`

4. Adjust your `cluster-values.yaml` in `zone2`

6. Adjust your `cluster-values.yaml` `in zone3`