KubeOpsDoku – documentation

Kubeopsvm: About KubeOpsVM

Mon, 01 Jan 0001 00:00:00 +0000

This article provides an overview of KubeOpsVM, a virtualization solution designed for Kubernetes-based environments. It explains how KubeOpsVM enables the operation and management of virtual machines alongside containerized workloads within a shared orchestration platform.

What Is KubeOpsVM?

KubeOpsVM is a virtualization platform that extends Kubernetes-based infrastructure to support virtual machines. It allows organizations to manage VM-based and container-based workloads within the same environment, using Kubernetes as the orchestration layer.

By consolidating these workload types on a single platform, KubeOpsVM reduces the need for separate virtualization and container management systems. This approach simplifies infrastructure operations, improves consistency across environments, and supports more efficient resource utilization.

Why KubeOpsVM Matters

Many organizations adopting cloud-native technologies must continue to operate existing applications that depend on traditional virtual machines. Managing these workloads separately from containerized services often increases operational complexity and creates fragmentation across tools, processes, and infrastructure.

KubeOpsVM addresses this challenge by enabling virtual machines to run within Kubernetes environments. This makes it possible to manage legacy and modern workloads through a common control plane, helping organizations streamline administration and support gradual modernization strategies.

Key Features Driving Innovation

KubeOpsVM distinguishes itself by delivering a robust virtualization experience through Kubernetes integration. Its primary features include:

Unified Workload Management - Manage virtual machines and containerized workloads through a shared Kubernetes-based platform and operational model.
Scalability - Scale virtual machines in response to changing demand by using Kubernetes scheduling and resource management capabilities.
High Availability - Support highly available virtualized workloads in multi-node environments through Kubernetes self-healing, scheduling, and failover mechanisms.
Resource Efficiency - Run virtualized and containerized workloads on the same infrastructure to improve resource utilization and operational efficiency.

A Vision for the Future

KubeOpsVM is designed for virtualization in cloud-native environments. It addresses the growing need to operate virtual machines alongside container-based workloads in Kubernetes, helping organizations connect existing systems with modern platform architectures. This supports a more practical and gradual path toward modernization.

Whether the goal is to maintain legacy applications, improve deployment consistency, or increase workload portability, KubeOpsVM offers a unified approach to running VMs within Kubernetes-based platforms. As cloud-native adoption continues to grow, KubeOpsVM is positioned to support future infrastructure models where virtualized and containerized workloads operate together more seamlessly.

Kubeopsvm: Detailed Explanation for Configuration File for Custom VMs

Mon, 01 Jan 0001 00:00:00 +0000

To set up a virtual machine you need to create a configuration file. The configuration file contains information, such as name of the VM, used resources (e.g. RAM) and states (e.g. running, halted).

Heed the comments (#) in the file below.

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/vm: example-vm-cirros
  name: example-vm
  namespace: {{ values.vmValues.namespace | object.default "kubevirt" }}
spec:
  runStrategy: Always
  template:
    metadata:
      labels:
        kubevirt.io/vm: example-vm-cirros
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: virtio
            name: containerdisk
          - disk:
              bus: virtio
            name: cloudinitdisk
        machine:
          type: ""
        resources:
          requests:
            memory: 64M
      terminationGracePeriodSeconds: 0
      volumes:
      - name: containerdisk
        containerDisk:
          image: {{ package.includes.containers.example.registry }}/{{ package.includes.containers.example.image }}:{{ package.includes.containers.example.tag }}
          imagePullSecret: kubeops-vm
          imagePullPolicy: Always
      - cloudInitNoCloud:
          userDataBase64: IyEvYmluL3NoCgplY2hvICdwcmludGVkIGZyb20gY2xvdWQtaW5pdCB1c2VyZGF0YScK
        name: cloudinitdisk

YAML	Value	Description
metadata.labels	kubevirt.io/vm: example-vm-cirros	Labels of the virtual machine. Used to identifiy a specific VM.
metadata.name	example-vm	The display name of the VM.

Kubeopsvm: How to install KubeOps Virtualization (kubevirt)

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance Enterprise license.

KubeVirt provides a powerful way to run virtual machines on top of Kubernetes, creating a unified platform for both containerized and traditional workloads. This guide walks through the essential steps for getting KubeVirt up and running using the KOSI package manager - from pulling and preparing the KubeVirt package, to creating a custom configuration file with your credentials, and finally verifying a successful installation in the kubevirt-namespace. Once these steps are complete, you’ll be ready to deploy and manage virtual machines alongside your existing container infrastructure - all within a single Kubernetes cluster.

KubeVirt is a specialized package that extends the capabilities of the KubeOps platform to support machine virtualization.

Requirements

To follow the next steps, make sure you have your credentials ready from your KubeOps.net account.

Make sure you have prepared the following points:

A KubeOps cluster with one master node.
Access to the master node with root privileges.

Step 1 - Download KubeOpsVM

Log in to the master node remotely. For example, using ssh.
If necessary, switch to your cluster user. If you forgot your cluster user, check your cluster-values.yaml.

su <your linux username>

kosi login -u [USERNAME]

Search for available software packages by using the KOSI package manager. In this case, we restrict the search to KubeOps virtualization using kubeops-vm.

kosi search --hub kubeops-vm

Log in to the local container registry on the master node using podman. Replace <master-ip> with the IP address of your master node.

podman login <master-ip>:30002 -u admin -p <harbor admin password> --tls-verify=false

The package is named kubeops/kubevirt:2.2.0_Beta0 and lies on the hub kubeops-vm. Download the package using the kosi pull command and save it with the desired file name. In this example, the file is saved as kubevirt-2.2.0_Beta0.tgz. The -r flag sets the source registry the container images are pulled from, and -t sets the local target registry the images are pushed to.

kosi pull --hub kubeops-vm kubeops/kubevirt:2.2.0_Beta0 -o kubevirt-2.2.0_Beta0.tgz -r <master-ip>:30002/kubeops -t 127.0.0.1:30002/kubeops

Step 2 - Remove the Taint of the Master Node (only necessary on single node cluster)

This step is only required for a single-node cluster.

If your cluster consists of only one node, you must remove the taint from the master node so that workloads can be scheduled on it.

Replace [NODENAME] with the name of you master node.

kubectl taint nodes [NODENAME] node-role.kubernetes.io/control-plane:NoSchedule-

Step 3 - Create Kubevirt Configuration File

Before installing the package, create a YAML configuration file. Save the following content in the same directory as the package file and name the file kubevirt-values.yaml.

Replace [your username] and [your harbor cli secret] with your KOSI username and your Harbor CLI secret.

The harbor-values must be adapted to your harbor instance.

apiVersion: kubeops/kubeopsctl/enterprise/beta/v1

deleteNs: false
localRegistry: true

packages:
  - name: kubevirt-manager
    enabled: true
    values:
      standard:
        namespace: kubevirt
        hostname: kubeops-vms.local
        nodeport: 30733
        ingressName: nginx
      advanced: {}

  - name: kubevirt
    enabled: true
    hub: kubeops-vm
    values:
      standard:
        namespace: kubevirt

        imagePullSecretCred:
          registry: 127.0.0.1:30002/kubeops
          user: [your username]
          password: [your harbor cli secret] # must be CLI Token from harbor

        kubevirtobject:
          spec:
            configuration:
              developerConfiguration:
                logVerbosity:
                  virtLauncher: 2
                  virtHandler: 3
                  virtController: 4
                  virtAPI: 5
                  virtOperator: 6

                featureGates:
                  - ExpandDisks
                  - CPUManager
                  - ExperimentalIgnitionSupport
                  - HypervStrictCheck
                  - Sidecar
                  - HostDevices
                  - Snapshot
                  - VMExport
                  - HotplugVolumes
                  - HostDisk
                  - ExperimentalVirtiofsSupport
                  - DownwardMetricsFeatureGate
                  - Root
                  - ClusterProfiler
                  - WorkloadEncryptionSEV
                  - VSOCK
                  - DisableCustomSELinuxPolicy
                  - KubevirtSeccompProfile
                  - DisableMDEVConfiguration
                  - PersistentReservation
                  - VMPersistentState
                  - MultiArchitecture
                  - AutoResourceLimitsGate
                  - AlignCPUs
                  - VolumesUpdateStrategy
                  - VolumeMigration
                  - InstancetypeReferencePolicy

                useEmulation: false

        cdiobject:
          spec:
            config:
              insecureRegistries:
                - registry.kubeops.net
              scratchSpaceStorageClass: rook-cephfs # Optional; Default: rook-cephfs
  - name: harbor
    enabled: false
    values:
      standard:
        namespace: harbor # optional, default is harbor
        ingressClassName: nginx # optional, default is nginx; set to "traefik" to route via Traefik
        harborpass: "password" # required: set password for harbor access
        databasePassword: "Postgres_Password" # required: set password for database access
        redisPassword: "Redis_Password" # required: set password for redis access
        externalURL: http://10.2.10.11:30002 # required, the ip address and port, from which harbor is accessable outside of the cluster
        nodePort: 30002 # required
        hostname: harbor.local # required
        harborPersistence:
          persistentVolumeClaim:
            registry:
              size: 40Gi # optional, default is 40Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            jobservice:
              jobLog:
                size: 1Gi # optional, default is 1Gi
                storageClass: "rook-cephfs" #optional, default is rook-cephfs
            database:
              size: 1Gi # optional, default is 1Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            redis:
              size: 1Gi # optional, default is 1Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
            trivy: 
              size: 5Gi # optional, default is 5Gi
              storageClass: "rook-cephfs" #optional, default is rook-cephfs
      advanced: #  All values from https://artifacthub.io/packages/helm/harbor/harbor/1.8.1#configuration are overwritable

Step 4 - Install KubeOps-VM

Install the KubeVirt package using the prepared configuration file you created.

kosi install -p kubevirt-2.2.0_Beta0.tgz -f kubevirt-values.yaml

Verify the installation by checking the pods in the kubevirt namespace.

kubectl get pods -n kubevirt

The installation is complete when the status of all pods show the status “Running”.

For example:

Troubleshoot

If you receive an ImagePullBackOff error, recreate the image pull secret

If the kubevirt-manager pod is in ImagePullBackOff and reports an error similar to the following: Unable to retrieve some image pull secrets (kubeops-harbor-secret)

create the required registry secret in the kubevirt namespace:

 kubectl -n kubevirt create secret docker-registry kubeops-harbor-secret \
  --docker-server=127.0.0.1:30002 \
  --docker-username=<_user_name_> \
  --docker-password=<_password_>

After creating the secret, restart the affected kubevirt-manager pod.

the following example, the kubevirt namespace is used. If KubeVirt is deployed in a different namespace, replace the value passed to the -n parameter.

kubectl -n kubevirt create secret docker-registry kubeops-vm --docker-server=127.0.0.1:30002 --docker-username=admin --docker-password=<admin password - see harborpass value in kubeopsctl.yaml>

Step 5 - Pull and Install KubeVirt Manager Package

Pull the KubeVirt Manager package.

kosi pull --hub kubeops kubeops/kubevirt-manager:2.2.0_Beta0 -o kubevirt-manager-2.2.0_Beta0.tgz -r master-ip:30002/kubeops -t 127.0.0.1:30002/kubeops

Install the KubeVirt Manager package.

kosi install -p kubevirt-manager-2.2.0_Beta0.tgz -f kubevirt-values.yaml

Next Steps

After the installation is complete, you can start creating and deploying a custom virtual machine. For more information, see How to create and deploy a Custom Virtual Machine using KubeOps.

Kubeopsvm: Release Notes

Mon, 01 Jan 0001 00:00:00 +0000

Kubeopsvm: Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

Troubleshooting

virt-handler pods only 2/3 ready and status is CrashLoopBackOff

show details

Container ‘virt-handler’ has the following error log:

{"component":"virt-handler","level":"fatal","msg":"Failed to create an inotify watcher","pos":"cert-manager.go:105","reason":"too many open files","timestamp":"2026-07-14T07:55:17.060525Z"}              │
│ panic: Failed to create an inotify watcher

Solution

The fs.inotify.user_watches and fs.inotify.user_instances must be increased. To do so, ssh to the corresponding node and execute the following commands

sudo sysctl -w fs.inotify.max_user_watches=1048576

sudo sysctl -w fs.inotify.max_user_instances=512

Kubeopsvm: Commandline Operations

Mon, 01 Jan 0001 00:00:00 +0000

Learn the most common command-line operations for managing KubeOps virtual machines - including creating a VM with a YAML configuration file, starting or stopping the VM by patching its runStrategy, viewing the VM or instance status with kubectl describe, restarting the VM (which deletes the instance and causes data loss), and deleting the VM either in a cascading manner (removing the VM and its instance) or by leaving orphaned instances.

Create a new virtual machine

The command kubectl with the paramater create uses the VM configuration file vm-config.yaml. For more detailed explaination on VM configuration files, see [Detailed Explanation for Configuration File for Custom VMs](Detailed Explanation for Configuration File for Custom VMs).

kubectl create -f vm-config.yaml

Start the virtual machine

Start the virtual machine VMNAME. Replace VMNAME with the name of your virtual machine,

kubectl patch virtualmachine VMNAME --type merge -p \
'{"spec":{"runStrategy":"Always"}}'

Status of a virtual machine

Print the status of the virtual machine VMNAME. Replace VMNAME with the name of your virtual machine,

kubectl describe virtualmachine VMNAME

Status of a virtual machine instance

kubectl describe virtualmachineinstance VMNAME

Stop the virtual machine instance

kubectl patch virtualmachine VMNAME --type merge -p \
'{"spec":{"runStrategy":"Halted"}}'

Restart the virtual machine

{{ hazard_notice }} Restarting the virtual machine will delete the virtual machine instance. This action cannot be undone. {{ /hazard_notice }}

kubectl delete virtualmachineinstance vm

Cascade delete (implicit)

Kubectl first deletes the virtual machine, followed by deleting the corresponding virtual machine instance.

Delete the virtual machine VMNAME. Replace VMNAME with the name of your virtual machine,

{{ hazard_notice }} This action cannot be undone. {{ /hazard_notice }}

kubectl delete virtualmachine VMNAME

Cascade delete (explicit)

Kubectl first deletes the virtual machine, followed by deleting the corresponding virtual machine instance.

Delete the virtual machine VMNAME. Replace VMNAME with the name of your virtual machine,

{{ hazard_notice }} This action cannot be undone. {{ /hazard_notice }}

kubectl delete virtualmachine VMNAME --cascade=true

Delete orphans

The running virtual machine is only detached, but not deleted.

Delete orphans of the virtual machine VMNAME. Replace VMNAME with the name of your virtual machine,

kubectl delete virtualmachine VMNAME --cascade=false

Kubeopsvm: General Concepts of KubeOpsVM

Mon, 01 Jan 0001 00:00:00 +0000

This article explores the core concepts of KubeOpsVM, a virtualization solution that integrates with Kubernetes to manage virtual machines alongside containerized workloads. It highlights how KubeOpsVM bridges traditional VMs and cloud-native systems, offering scalability and streamlined orchestration for modern infrastructure.

What is the purpose of a Virtual Machine?

A virtual machine (VM) is a software-based environment that simulates a physical computer, enabling it to run an operating system and applications independently from the host machine. Its primary purpose is to optimize hardware usage by allowing multiple operating systems to run on a single device. This setup is crucial for tasks like software development, where developers can test applications in various environments without needing separate hardware.

Security and isolation are other key advantages, as VMs keep applications separate from the host system, reducing the risk of system-wide damage from potential threats. They also simplify disaster recovery, since entire VM states can be backed up, cloned, or transferred easily. Additionally, VMs provide flexibility by enabling cross-platform compatibility, allowing users to run different operating systems simultaneously. This feature supports legacy software that might not be compatible with modern systems, ensuring continued access to older applications.

Virtual machines are integral to cloud computing, IT infrastructure, and personal computing due to their scalability, flexibility, and efficiency.

How does KubeOpsVM work?

KubeOpsVM is like a “computer inside a computer.” It allows you to run an operating system (OS) and applications in a sandboxed environment, separate from your actual physical machine.

In the example shown, any VM can be hosted on any agent (worker or master) within the cluster.

A virtual machine does not necessarily have to run on a specific machine. Depending on the current requirements, the VM is deployed on the appropriate machine in the cluster. kubevirt takes over the complete administration - no further intervention is necessary.

One of the main advantages of KubeOps virtualization is the asynchronous use and access of a VM and the deployment and execution of the VM depending on the available resources or machines. This ensures that each VM can always work independently of workloads.

Kubeopsvm: Getting Started

Mon, 01 Jan 0001 00:00:00 +0000

Kubeopsvm: How to create and deploy a Custom Virtual Machine using KubeOps

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

Creating custom virtual machines on top of Kubernetes is a powerful way to unify containerized and traditional workloads. In this introductory guide, you’ll learn how to set up your KubeOps cluster to deploy a Cirros-based VM using KubeVirt, from checking off a few key prerequisites to preparing and applying a concise YAML configuration file.

Whether you’re a seasoned Kubernetes user looking to expand your virtualization capabilities or just getting started with hybrid infrastructure, these steps will walk you through everything from generating the custom VM file to verifying its running status and accessing it over SSH. By following this process, you’ll have a seamlessly integrated virtual machine environment in no time - ready to support your application needs.

Requirements

Make sure you have prepared the following points:

A KubeOps cluster with at least one master node and root privileges.
A running kubeVirt instance. (see How to install KubeOps Virtualisation (KubeVirt)).
Enough resources to support a virtual machine

For example:

To follow this guide, use the following values to configure the custom virtual machine. If you wish, you can adjust the values yourself. Make sure that the adjusted values are applied accordingly everywhere.


Filename (VM configuration file)	`custom-vm.yaml`
Virtual Machine Label	`kubevirt.io/vm: customvm`
Virtual Machine Name	`customvm`
Virtual Machine Namespace	`kubevirt`
Virtual Machine Memory	`64M`

Additional Information

For users to understand the most basic functionality of KubeVirt, one must know about the following fundemental concepts:

KubeVirt offers different methods to engage with the virtual machines through specialized custom resources (CR). On the lowest level is the VirtualMachineInstance (VMI) which represents the stateless instance of a virtual machine. These VMIs can be managed by higher level resources, such as VirtualMachine (VM), which realizes a stateful VMI that can be stopped and started while maintaining its data and state.

The manifests for these resources contain a multitude of parameters that surpass the needs of most users. Consequently this guide will not explain all parameters.

How to create and deploy a Custom Virtual Machine

The following steps will show you, how to manually create and access a virtual machine, using Kubernetes with a running KubeVirt instance.

Step 1 - Create a Configuration File for the Virtual Machine

To set up a virtual machine you need to create a configuration file. The configuration file contains information, such as name of the vm as well as its namespace, used resources (e.g. RAM) and states (e.g. running, halted). Use the file contents below and make desired changes.

Use the following content to create a basic vm configuration file custom-vm.yaml.

This file needs to be adapted at spec.spec.volumes[0].containerDisk.image. You need to insert your local harbor deployments’ port.

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/vm: example-vm-cirros
  name: example-vm
  namespace: kubevirt
spec:
  runStrategy: Always
  template:
    metadata:
      labels:
        kubevirt.io/vm: example-vm-cirros
    spec:
      domain:
        devices:
          disks:
          - disk:
              bus: virtio
            name: containerdisk
          - disk:
              bus: virtio
            name: cloudinitdisk
        machine:
          type: ""
        resources:
          requests:
            memory: 64M
      terminationGracePeriodSeconds: 0
      volumes:
      - name: containerdisk
        containerDisk:
          image: "127.0.0.1:5001/kubeops/kubevirt/cirros-container-disk-demo:v1.4.0"
          imagePullSecret: kubeops-vm
          imagePullPolicy: Always
      - cloudInitNoCloud:
          userDataBase64: IyEvYmluL3NoCgplY2hvICdwcmludGVkIGZyb20gY2xvdWQtaW5pdCB1c2VyZGF0YScK
        name: cloudinitdisk

The image, from which the virtual machine is created, can be found in the first section of spec.templates.spec.volumes of the manifest. It is used in the form of a ContainerDisk, meaning the image is part of a container image, within a remote repository.

This manifest will use a preconfigured image from the repository kubevirt/cirros-container-disk-demo:latest with the following credentials:


User (cirros image)	`cirros`
Password (cirros image)	`gocubsgo`

KubeVirt supports the ability to assign a startup script to a virtual machine. The script is executed automatically when the VM initializes. There are multiple datasources supported, which will inject those scripts through the use of ephemeral disks.

One of those datasources is called cloudInitNoCloud, which is mentioned here as the second volume in spec.templates.spec.volumes. VMs with the cloud-init package installed will detect the ephemeral disk and execute custom userdata scripts at boot.

Step 2 - Transfer the VM Image to your Local Registry

Pull the image from the kubeops registry

 podman pull registry.kubeops.net/kubeops/kubeops/kubevirt/cirros-container-disk-demo:v1.4.0-patch0

Tag the image for the local registry

In the following command you need to insert your master1 ip address into the placeholder

podman tag registry.kubeops.net/kubeops/kubeops/kubevirt/cirros-container-disk-demo:v1.4.0-patch0 [your master1 ip]:5001/kubeops/kubevirt/cirros-container-disk-demo:v1.4.0-patch0

Push the image to the local registry

In the following command you need to insert your master1 ip address into the placeholder

podman push [your master1 ip]:5001/kubeops/kubevirt/cirros-container-disk-demo:v1.4.0-patch0 --tls-verify=false

Step 3 - Deploy the Virtual Machine

Deploy the vm using the kubectl command and the custom-vm.yaml file.

kubectl apply -f custom-vm.yaml

Check the state of the vm.

kubectl get vmi -A

This command lists all virtual machine instances. Since we only have one VM installed, the output looks something like this.

NAMESPACE   NAME         AGE   PHASE     IP              NODENAME       READY
kubevirt    customvm   22h   Running   192.168.16.48   stackedadmin   True

Notice We highly recommend documenting the basic settings of your virtual machines (especially the configurations). This includes in particular: the used configuration file (yaml), the label of the vm, the **name of the vm **and the namespace of the vm. You can also document the image used. Documentation makes it easier to maintain and expand the virtual machines.

Step 4 - Access the Custom VM

To connect to your newly created VM follow the documentation on how to access a Virtual Machine in a KubeOps Cluster

References

To know how to create a ContainerDisk from a local image, see Create a ContainerDisk from a local image guide
For more detailed explanation for virtual machine configuration files , see Detailed Explanation for Configuration File for Custom VMs.

Kubeopsvm: How to access a virtual machine in a KubeOps Cluster

Mon, 01 Jan 0001 00:00:00 +0000

Requirements

Note This guide requires a Compliance enterprise license.

Before proceeding, ensure that the following prerequisites are met:

A KubeOps cluster with at least one master node.
A running virtual machine in said cluster.
The credentials of the running image within the virtual machine (username and password).

For example:

In the example shown, the custom VM is accessed via the master node**, Worker Node 1**, or Worker Node 2.

A KubeOps virtual machine is not exposed by default and therefore cannot be accessed remotely by the admin node.

Step 1 - Identify your Virtual Machine

Log in to the master or worker machine.
Most kubeops software requires root privileges. Switch to the root user.

sudo -i

Identify your virtual machine by running the kubectl get vmi command.

kubectl get vmi -A

This command lists all virtual machine instances. Choose the IP of the desired virtual machine.

NAMESPACE   NAME         AGE   PHASE     IP              NODENAME       READY

kubevirt    [CUSTOMVM-1]   22h   Running   [192.168.16.XXX]   stackedadmin   True
kubevirt    [CUSTOMVM-2]   22h   Running   [192.168.16.YYY]   stackedadmin   True

Step 2 - Access the Virtual Machine

Log in to the example VM remotely (e.g. using ssh). Use the IP address from the previous step. Use the credentials of the image. Replace [USERNAME] and [IP-ADDRESS] accordingly.

ssh [USERNAME]@[IP-ADDRESS]

Step 1 - Create a Service forwarding the SSH port of the VM

The following YAML defines a Kubernetes Service that forwards the SSH port of your VM to the spec.ports.nodePort value. The Service connects to your VM by using the selector, so make sure that the given label matches the one of your VM. Store it in a local YAML file serviceforwarding.yaml

apiVersion: v1
kind: Service
metadata:
  name: vm-ssh-forwarding
  namespace: kubevirt
spec:
  type: NodePort
  ports:
    - port: 22
      nodePort: 30226
      targetPort: 22
      protocol: TCP
  selector:
    kubevirt.io/vm: customvm # Binds directly to the underlying VM pod

Apply the Service with the following command:

 kubectl apply -f serviceforwarding.yaml

To verify that the Service is connected to your VM, check the Endpoints resource in your Kubernetes cluster. The output should look similar to this:

NAME                ENDPOINTS          AGE
vm-ssh-forwarding   10.244.138.61:22   5d23h

Step 2 - Access the Custom VM

You can access the custom VM remotely from any host that has network access to the cluster machines, for example by using ssh.

Log in to the machine.
Identify the IP address of your custom VM by using the kubectl get vmi command.

kubectl get vmi -A

This command lists all virtual machine instances. Since we only have one VM installed, the output looks similar to this:

NAMESPACE   NAME         AGE   PHASE     IP              NODENAME       READY

kubevirt    example-vm   22h   Running   192.168.16.48   stackedadmin   True

To connect to the example VM remotely using ssh, use the IP address of a cluster node because the Service type is NodePort. In this example, the master node IP address is used. Also use the credentials of the running cirros image.

ssh [vm user]@[your master1 ip] -p 30226 # this port must match the one defined in your service

Kubeopsvm: How To Guides

Mon, 01 Jan 0001 00:00:00 +0000

In the following sections, you will find everything from initial setup and configuration, to advanced tips and tricks that will help you get the most out of the software. Our aim is to assist you in becoming proficient with KubeOpsVM, enhancing both your productivity and your user experience.

Lets get started on your journey to mastering KubeOpsVM!

Kubeopsvm: Install KubeOps virtualization on a Dedicated Machine

Mon, 01 Jan 0001 00:00:00 +0000

This guide describes how to set up a Kubernetes-based virtualization environment by using kubevirt automated installation routines. The setup is performed on a physical or virtual machine and runs largely automatically, requiring only minimal user input.

As part of the installation, KubeVirt and additional KubeOps components, such as the KOSI package manager, are installed automatically on the target node.

KubeVirt provides the virtualization layer within Kubernetes and enables virtual machines to run as Kubernetes-managed resources.

System architecture after installation

After the installation is complete, the environment consists of:

-a single-node Kubernetes cluster

one running node functioning as both the control-plane and worker node
one virtual machine ubuntu vm running in the namespace kubevirt
one preconfigured ubuntu image running on that virtual machine

The ubuntu image includes predefined login credentials.

Notice

A virtual machine is not permanently bound to a specific machine. In Kubernetes-based virtualization environments, VM placement depends on resource availability and scheduling decisions within the cluster. KubeVirt manages this placement and execution automatically, so no manual intervention is required during normal operation.

This behavior supports flexible resource usage and allows workloads to operate independently of a fixed host system.

One of the main advantages of KubeOps virtualization is the asynchronous use and access of a VM and the deployment and execution of the VM depending on the available resources or machines. This allows virtual machines to be scheduled independently based on available cluster resources.

Requirements

Before you begin the installation, ensure that you have the credentials for the following accounts available:

your KubeOps.net account
your Red Hat Subscription Manager account

For more information about Red Hat Subscription Manager, see the Red Hat online documentation.

System Requirements

The ISO can be installed on either a physical machine or a virtual machine. Ensure that the target system meets the following minimum requirements.

The KubeOps-VM ISO is based on Red Hat Enterprise Linux (RHEL). Only RHEL 9.6 and 9.7 are supported.


CPU	4 cores
Memory	8 GB
Disk space	50 GB of unformatted, non-partitioned disk space

Notice

Actual system requirements depend on the number and type of virtual machines that will run on the system.

In general, the more virtual machines you plan to run, the more disk space is required. Depending on the expected workload, increasing CPU and memory resources may also be necessary.

Step 1 - Download the KubeOps-VM ISO File

Download the ISO file from the following location:
https://kubeops.net/users-only/welcome.

Step 2 - Install the KubeOps-VM ISO

During installation, you will be prompted to provide configuration values. Follow the on-screen instructions and enter the required information.

Boot your machine from the ISO.
Select Install Red Hat Enterprise to start the installation.

Select the hard disk on which the system will be installed.

Configure the network settings:

Select your preferred networking device.
Select whether the IP configuration is dynamic or static.
Enter the IP address.
Enter the subnet mask address.
Enter the gateway address.
Enter the DNS server address.
Enter your preferred hostname. This will be the name of the machine. ( Do not use special characters in the hostname.)

Select your preferred activation method for the Red Hat Subscription Manager.
Select your preferred security method for the GRUB bootloader.
We recommend the option “random password” - this will set up the security automatically. For more information about the GRUB bootloader and its security, refer to the respective documentation.
Enter your KubeOps.net account credentials.
Notice The setup process may take several minutes. Do not shut down the machine while the installation is in progress.

Follow the remaining on-screen instructions to complete the installation.

During setup using an ISO file, the control-plane node was automatically preconfigured with a user (root) and its password.


User	`ko-admin`
Password	`KubeOps2024`

Log in to the control-plane node remotely.
For example, using ssh. The first time you log in, you must use the preconfigured password (see above). Use the hostname or the IP address that you assigned in a previous step during the installation.

ssh ko-admin@[HOSTNAME]

ssh ko-admin@[IP ADDRESS]

At first login, you will be prompted to change the default password. ollow the on-screen instructions. After the password change is complete, you will be automatically logged out.
Notice During the first boot of the machine, additional software and parameters are configured in the background, such as the cluster setup and configuration.
Login to the machine again using your new credentials.
Most kubeops software require root privileges. Switch to root user.

sudo -i

To check the state of background configuration tasks, run the journalctl command.

journalctl -u clustersetup.service

Keep this view open while the installation finishes.

Check your kubeops virtualization by using the kubectl get pods command.

kubectl get pods -n kubevirt

The installation is complete when the status of all pods is set to “Running”. For example:

Note: Pods in the kubevirt namespace may initially remain in the Pending state. If this happens, you can either add worker nodes as described in How to join Nodes to a KubeOps Cluster or or remove the control-plane taint from the single node with kubectl taint nodes <control-plane-node-name> node-role.kubernetes.io/control-plane:NoSchedule-

To change the password of the root user, run command:

passwd

Step 4 - Install the example VM

After the platform installation is complete, you can deploy the example virtual machine.

Login to the machine using your updated credentials.
Switch to root user as most kubeops software needs root privileges.

sudo -i

Create a values file for the virtual machines:

Example:

ubuntuvmValues:
  name: ubuntu-vm
  namespace: kubevirt-vm 
  sshkeys:
    - key: <public ssh key, p.e. id_rsa.pub file>
  dataVolume:
    storageClassName: rook-cephfs
    source:
      secret: kubeops-auth

In this example,these values are saved in the vmvalues.yaml file.

Ensure that you are still logged in to KOSI using your kosi username. KOSI sessions remain valid for 24 hours, after that time you need to login again.

kosi login -u [USERNAME]

Use the KOSI package manager to search for available software packages. In this case, we restrict the search to KubeOps virtualization using the hub kubeops-vm and filter for ubuntu.

kosi search --hub kubeops-vm --ps ubuntu

Install the package using the kosi install command. The example VM configuration is named kubevirt-ubuntu-vm

kosi install --hub kubeops-vm kubeops/kubevirt-ubuntu-vm:1.0.0 -f vmvalues.yaml --dname ubuntu

Create the required secret

Create a custom secret for image access:

kubectl create secret generic kubeops-auth --from-literal=accessKeyId=<kubeopsaccount> --from-literal=secretKey="<password-token for imagepullregistry>" --namespace=<namespace of vm>

Note: The image pull token is not the same as the account password. If the registry uses OIDC integration, for example with Keycloak, use the appropriate token instead of the user password.

Additional requirements for air-gapped environments

For an airgap installation,

Use a domain name for Harbor.
Do not use an IP address with a port
Do not manually copy images to the nodes or load them with containerd
Ensure that all required images are available through the configured registry

You may also need to create an image pull secret named kubeops-vm. This secret uses the same registry credentials as the kubeops-auth secret created by the KubeOps platform tool, like rook-ceph.

You can create it with the following command:

kubectl -n kubevirt create secret docker-registry kubeops-vm --docker-server=<registry for image pulling> --docker-username=<username of registry> --docker-password=<password-token for imagepullregistry>

Step 5 - Access the example virtual machine

After deployment, you can access the example virtual machine remotely from the control-plane node. For example, using ssh.

Login to the machine using your updated credentials.
Retrieve the IP address of your example VM with following command:

kubectl get vmi -A

This command lists all virtual machine instances. Since we only have one VM installed, the output looks something like this.

NAMESPACE   NAME         AGE   PHASE     IP              NODENAME       READY

kubevirt    ubuntu-vm   22h   Running   192.168.16.48   stackedadmin   True

Log in to the example VM remotely (e.g. using ssh). Use the IP address from the previous step. Use the credentials of the running ubuntu image from the table below.

ssh ubuntu@192.168.16.48


User (ubuntu image)	`ubuntu`
Password (ubuntu image)	`ubuntu`

Next Steps

For additional commands and operational tasks for KubeOps virtual machines, see Commandline Operations.

Kubeopsctl-2.0: KubeOps Compliance 2.0

Mon, 01 Jan 0001 00:00:00 +0000

Start with Kubeops Compliance 2.0

Follow the quickstart guides to learn how to create cluster in a secure, fast way with kubeopsctl.

Kubeopsvm: KubeOpsVM

Mon, 01 Jan 0001 00:00:00 +0000

Start with KubeopsVM

Follow the getting started guides to learn what KubeOpsVM is, and how to install the required components.

Kubeopsvm: Reference

Mon, 01 Jan 0001 00:00:00 +0000

Kubeopsvm: How to create a ContainerDisk from a local image

Mon, 01 Jan 0001 00:00:00 +0000

Note This feature requires a Compliance enterprise license.

KubeVirt supports multiple ways of using images to create virtual machines. One of the most convenient methods is using the ContainerDisk feature, which allows VM disks to be stored and distributed as container images through a registry.

To create a ContainerDisk, users can simply inject the image into a container in qcow2 or raw format. The container should be based on scratch and the images must be placed into the /disk directory. No other content is required.

The following Dockerfile builds a ContainerDisk, which you can store in a OCI registry such as harbor or the Docker Hub:

FROM kubevirt/container-disk-v1alpha
ADD <your Image>.qcow2 /disk

You can the build the OCI arteiact with the command:

podman build . -t

Note: The ContainerDisk must be readable by the user with the UID 107 (qemu).

If you want to access the virtual machine, it is recommended to use a SSH key in the userdata startup section of your YAML file:

      - name: cloudinitdisk
        cloudInitNoCloud:
          userData: |-
            #cloud-config
            ssh_authorized_keys:
              - ssh-rsa AAAAB3NzaK8L93bWxnyp test@test.com

Kubeopsvm: How to import a VM from vSphere and deploy it in a Cluster using KubeVirt

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

How to Import a VM from vSphere and Deploy it in a Cluster using KubeVirt

This guide outlines the steps to export a virtual machine from vSphere, convert it to the qcow2 format, and deploy it inside a Kubernetes cluster using KubeVirt.

Requirements

To follow these steps, ensure you have:

A VM in the vSphere environment.

A Kubernetes cluster with KubeVirt installed.

Access to a container registry to store the VM image.

Step 1 - Export OVF Template from vSphere

In vSphere, locate the VM you want to export.

Ensure the VM is powered off before exporting.

Right-click the VM, navigate to Template, and select Export OVF Template.

Provide a name for the exported template and click OK.

The export process will generate three files: *.vmdk, *.nvram, and *.ovf.

Only the *.vmdk file is required for further processing.

Step 2 - Convert the VMDK File to QCOW2 Format

KubeVirt requires disk images in qcow2 format. Use the qemu-img tool to convert the .vmdk file:

qemu-img convert -f vmdk -O qcow2 [input-file].vmdk [output-file].qcow2

Replace [input-file].vmdk with the name of your downloaded VMDK file and [output-file].qcow2 with your desired output name.

Step 3 - Create a Container Image from the QCOW2 File

To use the converted disk inside KubeVirt, create a ContainerDisk by building a container image with the qcow2 file.

Create a Dockerfile

Create a Dockerfile with the following content:

FROM scratch 

ADD --chown=107:107 [filename].qcow2 /disk/

Replace [filename].qcow2 with the actual filename of your converted disk image.

Build and Push the Image

Execute the following commands to build the image and push it to a container registry:

podman build -t [registry]/[namespace]/[vm-image-name]:[tag] -t localhost/[namespace]/[vm-image-name]:[tag] .

podman push [registry]/[namespace]/[vm-image-name]:[tag]

Replace the placeholders as follows:

[registry]: The URL of your container registry

[namespace]: Your project name

[vm-image-name]: A name for the VM image

[tag]: The version tag (e.g., 1.0.0)

Once the image is pushed, it can be used as a ContainerDisk in KubeVirt to deploy a virtual machine.

Step 4 - Deploy the VM using KubeVirt

Create a secret for authentication:


kubectl create secret generic kubeops-vm-sec --from-literal=accessKeyId=kubeops --from-literal=secretKey=registry-token --namespace kubevirt-vms-import

kubectl -n kubevirt-vms-import create secret docker-registry kubeops-vm --docker-server=registry.kubeops.net --docker-username="<username>" --docker-password="<registry-token>"

 kubectl patch serviceaccount default -n kubevirt-vms-import -p '{"imagePullSecrets": [{"name": "kubeops-vm"}]}'

Create a manifest file for the virtual machine:


apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels:
    kubevirt.io/vm: sk2-master01
  name: sk2-master01-vm
  namespace: kubevirt-vms-import
spec:
  runStrategy: Always
  template:
    metadata:
      labels:
        kubevirt.io/vm: sk2-master01
    spec:
      domain:
        devices:
          disks:
          - name: datavolumedisk
            disk:
              bus: virtio
          - name: cloudinitvolume
            disk:
              bus: virtio
        resources:
          requests:
            memory: 4092M
      volumes:
      - name: datavolumedisk
        dataVolume:
          name: sk2-master01-dv
      - name: cloudinitvolume
        cloudInitNoCloud:
          userData: |-
            #cloud-config
            chpasswd:
              list: |
              expire: False
            ssh_pwauth: True
            disable_root: false            
  dataVolumeTemplates:
  - metadata:
      name: sk2-master01-dv
    spec:
      storage:
        resources:
          requests:
            storage: 68Gi
        storageClassName: rook-cephfs
        accessModes:
          - ReadWriteMany
      source:
        registry:
          url: "docker://registry.kubeops.net/kubeops-vm/kubeops/kubevirt/sk2_master01_export:1.0.0" # (change the url to the image you want to deploy)
          secretRef: kubeops-vm-sec

kubectl apply -f filename.yaml

Step 5 - Verify Deployment

Run the following commands to check if the VM is deployed successfully:

kubectl get dv -n kubevirt-vms-import

kubectl get vmi -n kubevirt-vms-import

Ensure that the DataVolume is successfully downloaded and the VirtualMachineInstance is running.

Kubeopsvm: How to Monitor KubeVirt in Grafana

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

Monitor KubeVirt VMIs with Grafana and Prometheus

This guide explains how to monitor KubeVirt Virtual Machine Instances (VMIs) using Grafana and Prometheus. By the end of this guide, you will have a Grafana dashboard that displays metrics related to KubeVirt VMIs.

Prerequisites

A running multi-node Kubernetes cluster with KubeVirt installed

⚠️ Note: Single-node clusters are not supported due to component scheduling and storage limitations.
A functional storage provider (e.g., Rook-Ceph or any other CSI-compatible solution) must be installed and available

⚠️ Required for deploying both Prometheus and Grafana, as they rely on PersistentVolumeClaims (PVCs)
Prometheus deployed in the cluster
Grafana installed and configured to connect to Prometheus

Step 1: Ensure Prometheus is Collecting KubeVirt Metrics

KubeVirt exposes various metrics that Prometheus can scrape. These metrics are documented here: KubeVirt Metrics.

To check whether Prometheus is collecting KubeVirt metrics:

Open the Prometheus Dashboard
Search for KubeVirt-related metrics, for example kubevirt_vmi_memory_available_bytes.
If no metrics appear, ensure the KubeVirt servicemonitor has been added to Prometheus:
```
 kubectl get servicemonitor -A | grep kubevirt
```
Verify that KubeVirt was installed correctly and that monitorNamespace and monitorAccount are configured correctly. How to install KubeOps Virtualization (kubevirt)
Reload Prometheus operator.
Reload KubeVirt operator.

Step 2: Import the KubeVirt Grafana Dashboard

KubeVirt provides a ready-to-use Grafana dashboard JSON file: KubeVirt Grafana Dashboard.

To import it into Grafana:

Open the Grafana Dashboard.
Navigate to Dashboards > New > Import.
Paste the JSON content from the KubeVirt dashboard file.
Select Prometheus as the data source.
Click Load.

Step 3: Customize the Dashboard (Optional)

To add specific VMI-related metrics to the dashboard:

Click Edit on a panel.
Use PromQL queries to fetch desired VMI metrics, e.g.,
- CPU Usage: rate(kubevirt_vmi_cpu_usage_seconds_total[5m])
- Memory Usage: kubevirt_vmi_memory_available_bytes
- Network Traffic: rate(kubevirt_vmi_network_receive_bytes_total[5m])
Save the dashboard.

Note: You can view query output in the Prometheus dashboard.

Conclusion

You have successfully set up Grafana to monitor KubeVirt VMIs using Prometheus. You can now track VMI performance and resource usage effectively.

Kubeopsvm: How to create L3 networking for VMs

Mon, 01 Jan 0001 00:00:00 +0000

Note This guide requires a Compliance enterprise license.

This guide explains how to configure L3 networking for KubeVirt virtual machines (VMs) using Multus. With this setup, VMs can communicate across nodes through dedicated secondary networks, which improves network segmentation and deployment flexibility in Kubernetes environments.

Prerequisites

Before you begin, ensure that the following requirements are met:

A running Kubernetes cluster
kubectl command-line tool configured to connect to your cluster
Calico and Multus installed and configured
KOSI is installed and authenticated successfully

Configuring Node Networking using Network Manager State (NMState)

Network Manager State (nmstate) is a declarative network configuration tool for Linux. It provides an API and CLI to manage network settings in a structured, state-driven manner.

In Kubernetes, nmstate is often used with NodeNetworkConfigurationPolicy (NNCP) to configure node networking dynamically, allowing for changes such as setting up VLANs, bridges, bonds, and other networking features.

1. Deploy the nmstate Operator

First, create the nmstate-values.yaml:

# nmstate-values.yaml
nmstateValues:
  namespace: nmstate

Deploy the nmstate operator and its CRDs using this file:

kosi install --hub kubeops kubeops/nmstate:2.0.1 -f nmstate-values.yaml

2. Configure Node Networking with NNCP

Create an nncp-values.yaml file to define network configurations dynamically, including bridges and routes:

# nncp-values.yaml
namespace: "nmstate"
nodes:
  - nodeName: cluster1master1
    nodeIP: 10.2.10.11
    bridgeIP: 10.10.1.1 # bridge IP for cluster1master1.
    bridgeSubnet: 10.10.1.0 # `/24` subnet for bridge
    nodeInterface: ens192 # use `ip a` or `ifconfig` to identify interface

  - nodeName: cluster1master2
    nodeIP: 10.2.10.12
    bridgeIP: 10.10.2.1
    bridgeSubnet: 10.10.2.0
    nodeInterface: ens192

  - nodeName: cluster1worker1
    nodeIP: 10.2.10.14
    bridgeIP: 10.10.3.1
    bridgeSubnet: 10.10.3.0
    nodeInterface: ens192

  - nodeName: cluster1worker2
    nodeIP: 10.2.10.15
    bridgeIP: 10.10.4.1
    bridgeSubnet: 10.10.4.0
    nodeInterface: ens192

Note:

If you’re adding a new node, assign it a unique bridgeIP and bridgeSubnet, that follow the existing addressing pattern.

For example, if your last node uses:

bridgeIP: 10.10.4.1

bridgeSubnet: 10.10.4.0/24

add the next node like this:
  - nodeName: <new_node>
    nodeIP: <node_ip>
    bridgeIP: 10.10.5.1
    bridgeSubnet: 10.10.5.0
    nodeInterface: ens192

Use this values file to deploy the node network configuration policies on all nodes:

kosi install --hub kubeops kubeops/nncp:2.0.1 -f nncp-values.yaml

3. Allow Forwarded Traffic on the Bridge

Run the following commands on the relevant nodes to allow traffic through the secondary bridge:

iptables -A FORWARD -i br-secondary -j ACCEPT
iptables -A FORWARD -o br-secondary -j ACCEPT

4. Verify Bridge Configuration and Node Connectivity

Verify the Bridge on Each Node

To confirm that the bridge exists and has the expected IP address:

ip a show br-secondary

Verify Attached Interfaces

List the bridge interfaces and verify that the expected interfaces are attached:

ip link show type bridge

Verify Routes and Connectivity

Check if the expected routes are in place:

ip route show

Test Connectivity Between Nodes

Test connectivity to another node’s bridge IP:

ping -c 4 <another_node_bridgeIP>

Install KubeVirt

For installation instructions, please refer to the KubeVirt Installation Guide

Networks and Interfaces

L3 Networking with Multus

To set up L3 networking with Multus, each worker node requires a separate NetworkAttachmentDefinition (NAD) with a unique IP address and subnet matching the node’s network configuration. This allows isolated network environments for each pod, enabling multi-interface support on pods and communication between nodes in the cluster.

Plan the Network

Before creating the NADs, define the IP ranges and subnets for each worker node to avoid conflicts and routing conflicts.

Example IP range mapping for worker nodes:

Node	Subnet	Example IP for Pod
cluster1worker1	10.10.3.0/24	10.10.3.10
cluster1worker2	10.10.4.0/24	10.10.4.10

Example NAD Configuration for Worker Node 1

This example shows how to configure a NetworkAttachmentDefinition (NAD) for worker node 1 (cluster1worker1), with a static IP range assigned to its pods:

# nad_cluster1worker1.yaml
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: secondary-network-node1 # Assign unique names to each node's NAD
  annotations:
    k8s.v1.cni.cncf.io/nodeSelector: '{"kubernetes.io/hostname": "cluster1worker1"}'
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "secondary-network",
      "type": "bridge",
      "bridge": "br-secondary",
      "ipam": {
        "type": "static",
        "addresses": [
          {
            "address": "10.10.3.10/24", # use correct worker node's subnet and mention the appropriate IP for pod's network
            "gateway": "10.10.3.1"
          }
        ],
        "routes": [
          {
            "dst": "10.10.4.0/24",
            "gw": "10.10.3.1"
          }
        ]
      }
    }

Apply the K8s Manifest Definition files

To deploy the NetworkAttachmentDefinition (NAD) objects for your nodes, apply the following Kubernetes manifest files:

kubectl apply -f nad_cluster1worker1.yaml
kubectl apply -f nad_cluster1worker2.yaml

Verify the created Network Attachment Definition Objects

To ensure that the NetworkAttachmentDefinition (NAD) objects have been successfully created, run the following command:

kubectl get net-attach-def -A

Deploy Virtual Machines (VMs) and Attach the Secondary Networks

Once the NADs are created, you can deploy Virtual Machines (VMs) and attach them to the secondary networks. The following is an example Kubernetes manifest for deploying a VM with a secondary network interface attached.

Network Selection from NADs

Each VM must be connected to the correct network defined by the NetworkAttachmentDefinition (NAD) for its corresponding node. For example, if you have created a NAD for cluster1worker1 named secondary-network-node1, ensure that the VM manifest refers to this network in the annotations section and correctly links to the networkName under the multus configuration.

Ensure that the network name in the manifest matches the NAD definition.

Create Image Pull Secret

Please create a secret for pulling image from registry.kubeops.net.

kubectl -n "l3-networking" create secret docker-registry kubeops-vm \
  --docker-server=registry.kubeops.net \
  --docker-username=<kubeops-user> \
  --docker-password=<harbor_token>

Example VM Manifest for Secondary Network

The following is an example Kubernetes manifest for deploying a VM with a secondary network interface attached to a specific NAD:

# vm1.yaml
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: "ubuntu-vm"
  namespace: "l3-networking"
  annotations:
    k8s.v1.cni.cncf.io/networks: secondary-network-node1 # Ensure this matches the NAD network name
spec:
  nodeSelector:
    kubernetes.io/hostname: cluster1worker1 # edit for scheduling node on other node
  runStrategy: Always
  template:
    metadata:
      labels:
        kubevirt.io/size: small
    spec:
      domain:
        cpu:
          cores: 1
        devices:
          disks:
            - name: containervolume
              disk:
                bus: virtio
            - name: cloudinitvolume
              disk:
                bus: virtio
          interfaces:
          - name: default
            masquerade: {}
          - name: secondary-network
            bridge: {}
        resources:
          requests:
            memory: 2048M
      networks:
      - name: default
        pod: {}
      - name: secondary-network
        multus:
          networkName: secondary-network-node1 # Ensure this matches the NAD network name
      volumes:
        - name: containervolume
          containerDisk:
            image: registry.kubeops.net/kubeops-vm/kubeops/ubuntu-cloudimg-focal:2.0.1
            imagePullSecret: kubeops-vm
            imagePullPolicy: Always
        - name: cloudinitvolume
          cloudInitNoCloud:
            userData: |-
              #cloud-config
              chpasswd:
                list: |
                  ubuntu:ubuntu
                  root:toor
                expire: False
              ssh_pwauth: True
              disable_root: false

Apply the VM Manifest

Once the NetworkAttachmentDefinition (NAD) objects have been created, you can deploy the Virtual Machines (VMs) and apply the manifests. The following commands apply the VM manifests to your Kubernetes cluster.

kubectl apply -f vm1.yaml
kubectl apply -f vm2.yaml

Verify the Networking

After the VMs start successfully, verify that the secondary network interfaces and routes are configured correctly.

On each VM

1. Check the IP Address

First, check the IP address assigned to the pod (VM). This is crucial to ensure that the secondary network interface has been correctly assigned an IP.

kubectl exec -n <namespace> <pod-1> -- ip a

2. Assign IP Address if Not Automatically Assigned (Optional)

kubectl exec -n <namespace> <pod-1> -- sudo dhclient enp2s0

3. Check the Routes

Verify the routing configuration on the VM to ensure that traffic can flow through the secondary network. Use the following command:

kubectl exec -n <namespace> <pod-1> -- ip r

This will display the routing table for the VM and confirm if the secondary network is correctly set up.

Verify VM Communication

After ensuring the network settings are correct, test the communication between VMs by pinging the secondary network IP of another pod.

Test Connectivity Between VMs

Use the ping command to check if the VMs can communicate over the secondary network. Replace <pod2_secondary_network_IP> with the actual IP address of the secondary network interface on the second VM.

kubectl exec -n <namespace> <pod-1> -- ping -c 2 <pod2_secondary_network_IP>

If the ping is successful, it confirms that the VMs are able to communicate over the secondary network.

Additional Notes:

Ensure that you use the correct namespace and pod name while executing the commands.

Make sure the secondary network interface is correctly configured on each VM, and the IP address and routes are properly assigned.

If communication fails, check for issues in the NetworkAttachmentDefinition (NAD) configuration or verify that the correct CNI plugins are running.

Kubeopsvm: How to back up VMs and vm snapshots with Velero

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This guide describes how to configure Velero to back up and restore KubeVirt virtual machines in a Kubernetes cluster.

Velero stores backup data in an S3-compatible object storage backend. In this example, MinIO is used as the backup storage target.

Prerequisites

Note This feature requires a Compliance enterprise license.

Before you begin, ensure that the following prerequisites are met:

A running Kubernetes cluster is available.
Rook-Ceph is configured as the storage provider.
An S3-compatible object storage service is available.( in this case MinIO)
A KubeVirt virtual machine is running in the cluster

In following example MinIO is used.

Step 1 - Configure Velero

Define the Velero configuration in your values file:

veleroValues:
 namespace: "velero"
 accessKeyId: "minio"
 secretAccessKey: "minio123"
 backupLocationConfig:
   s3Url: "http://10.2.10.110:30090" # nodeport of minio

Enable Velero in the kubeopsctl.yaml file :

velero: true

Apply the configuration:

kubeopsctl apply -f kubeopsctl.yaml

Step 2 - Verify S3 Bucket Access

The bucket for Velero ( default Velero ) in the s3 storage should be accessible for Velero, so ensure that the bucket exists and that Velero has permission to write to it.

In this example, the bucket is configured so that Velero has write access:

Step 3 - Create a Backup

To create a backup of the namespace containing the KubeVirt virtual machine, run:

velero backup get
velero backup create kubeops-vm-backup --include-namespaces <Namespace of kubevirt vm>

The name of the backup must be unique.

Step 4 - Verify the Backup

To review the backup status and details, run:

velero backup get
velero backup describe kubeops-vm-backup
velero backup logs kubeops-vm-backup

Confirm that the backup completed successfully before proceeding.

Step 5 - Restore the Backup

To restore the backup, run:

velero restore get
velero restore create kubeops-vm-restore --from-backup kubeops-vm-backup

The restore name must be unique.
After the restore is complete, verify that the virtual machine resources have been recreated successfully in the target namespace.

velero restore get
velero restore describe kubeops-vm-restore
velero restore logs kubeops-vm-restore

Kubeopsctl: KubeOps 1.7.9

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.9 - Release Date 02.03.2026

changelog harbor 1.7.9

Bugfix

Implemented fixes for the following CVEs:
- CVE-2025-27151
- CVE-2025-49844

Kubeopsctl: KubeOps 1.7.8

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.8 - Release Date 04.02.2026

changelog ingress-nginx 1.7.8

Bugfix

Implemented fixes for the following CVEs:
- CVE-2025-15467
- CVE-2025-49794
- CVE-2025-49796

Kubeopsctl: KubeOps 1.7.7

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.7 - Release Date 28.01.2026

changelog Lima 1.7.7

New

Added support for Kubernetes 1.34.1

Note: Kubernetes 1.34.1 can be used with earlier versions of Kubeopsctl 1.7.x.

changelog rook-ceph 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog harbor 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog kube-prometheus-stack 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog kubeops-dashboard 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog logstash 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog filebeat 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog opensearch 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog opensearch-dashboards 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog keycloak 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog opa-gatekeeper 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog velero 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

changelog cert-manager 1.7.7

New

Updated Image Pull Policy to ensure that it will stay on Always

Kubeopsctl: KubeOps 1.7.6

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.6 - Release Date 30.12.2025

changelog Lima 1.7.6

changed

increased Port range of hosts for rook-ceph to 6800-7568.

changelog rook-ceph 1.7.6

New

Included Network Policies

changelog harbor 1.7.6

New

Included Network Policies

Fixed

fixed issue where harbor upgrade was not possible because of postgres version mismatch
fixed cve CVE-2024-56171, CVE-2025-49794, CVE-2025-49796, CVE-2024-3596, CVE-2025-5914, CVE-2024-46981 and CVE-2025-6965 - updated to appversion 2.14.0 and chartversion 1.18.0

changelog kube-prometheus-stack 1.7.6

New

Included Network Policies

changelog kubeops-dashboard 1.7.6

New

Included Network Policies

changelog logstash 1.7.6

New

Included Network Policies

Fixed

changed default loglevel to “INFO”

changelog filebeat 1.7.6

New

Included Network Policies

Fixed

changed default loglevel to “INFO”

changelog opensearch 1.7.6

New

Included Network Policies

Fixed

changed default loglevel to “INFO”
updated package to version 3.2.1 and appversion 3.2.0 for compatibility

changelog opensearch-dashboards 1.7.6

New

Included Network Policies

fixed

fixed CVE CVE-2025-7783, and updated package to version 3.2.2 with appVersion 3.2.0

changelog keycloak 1.7.6

New

Included Network Policies

changelog opa-gatekeeper 1.7.6

New

Included Network Policies

changelog velero 1.7.6

New

Included Network Policies

changelog cert-manager 1.7.6

New

Included Network Policies

Kubeopsctl: KubeOps 1.7.4

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.4 - Release Date 30.09.2025

changelog kube-prometheus-stack 1.7.4

fixed

fixed cve CVE-2024-45337, updated helm chart to version 77.0.2 ( appVersion v0.85.0)

changelog kubeops-dashboard 1.7.4

added

Resolved issue with service account

changelog cert-manager 1.7.4

fixed

fixed cve CVE-2024-45337, updated helm chart to version v1.18.2 ( appVersion v1.18.2)

changelog velero 1.7.4

fixed

fixed cve CVE-2024-24790, updated helm chart to version 10.1.0 ( appVersion 1.16.2)

Kubeopsctl: KubeOps 1.7.3

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.3 - Release Date 23.09.2025

changelog kubeopsctl 1.7.3

fixed

fixed issue where plattform upgrade was not possible

Kubeopsctl: KubeOps 1.7.2

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.2 - Release Date 05.09.2025

changelog kubeopsctl 1.7.2

fixed

Prevented failure by adding fallback creation logic for missing namespace and secret
fixed issue where auditlog path was not templatable

changelog lima 1.7.2

new

- added logic for updating from RHEL 8 to RHEL 9

fixed

- fixed issue where auditlog path was not templatable

changelog rook-ceph 1.7.2

fixed

- fixed issue where the lableing was only after installing platform packages
- added logic for cerating crushmap for rook-ceph 
- added topologySpreadConstraints for spreading across the kubeops zones
- Removed external ingress and deployed using helm chart
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog harbor 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Removed external ingress and deployed using helm chart
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog opensearch-os 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opensearch-dashboards 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Removed external ingress and deployed using helm chart

changelog opa-gatekeeper 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog cert-manager 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog ingress-nginx 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog keycloak 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Removed external ingress and deployed using helm chart
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.

changelog velero 1.7.2

fixed

- added topologySpreadConstraints for spreading across the kubeops zones
- Updated `imagePullPolicy` to `Always` to comply with BSI security requirements.
- Addressed the following CVEs:
  - CVE-2024-24790
  - CVE-2024-32002
  - CVE-2024-45491
  - CVE-2024-45492
  - CVE-2024-37371
  - CVE-2023-45853

changelog kubeops-dashboard 1.7.2

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog prometheus 1.7.2

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog filebeat 1.7.2

fixes

Addressed the following CVEs:
- CVE-2024-41110
- CVE-2024-45337
- CVE-2023-24538
- CVE-2023-24540
- CVE-2024-24790
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog calico-multus

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

Kubeopsctl: KubeOps 1.7.1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.1 - Release Date 29.07.2025

changelog lima 1.7.1

new

- added support of RHEL 9.6
- added containerd.io-1.7.27
- added container-selinux-2.237.0
- opened firewall ports for 30000-32767 tcp on all nodes

changelog helm 1.7.1

fixed

- updated helm to 3.17.4 for Rhel 9.6 support

changelog rook-ceph 1.7.1

fixed

- added spreading the crushmap across the kubeops zones

changelog kubeopsctl 1.7.1

fixed

- added suport of RHEL 9.6

Kubeopsctl: KubeOps 1.7.0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.0 - Release Date 30.06.2025

changelog lima 1.7.0

new

- added logic for updating from RHEL 8 to RHEL 9

fixed

- fixed issue where auditlog path was not templatable
- fixed issue where nodes could not be joined after update registry was performed

changelog rook-ceph 1.7.0

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog harbor 1.7.0

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opensearch-os 1.7.0

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opensearch-dashboards 1.7.0

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog kubeopsctl 1.7.0

fixed

- fixed issue where auditlog path was not templatable

changelog Velero 1.7.0

fixes

Addressed the following CVEs:
- CVE-2024-24790
- CVE-2024-32002
- CVE-2024-45491
- CVE-2024-45492
- CVE-2024-37371
- CVE-2023-45853

Kubeopsctl: KubeOps 1.7.0_Beta0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.0_Beta0 - Release Date 30.04.2025

Changelog kubeopsctl 1.7.0_Beta0

New

Added Licence verification

Bugfix

Resolved issue with kubeopsctl status for keycloak

Changelog lima 1.7.0_Beta0

Bugfix

Fixed a bug where ssh key support for ed25519 was not possible.

Changelog opensearch 1.7.0_Beta0

Bugfix

Fixed a bug where the secret internal-users-config-secret was not created.
resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog nginx-ingress 1.7.0_Beta0

Updates

Upgraded Helm chart to version 4.11.5 to address CVE-2025-1974.

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

changelog prometheus 1.7.0_Beta0

Bugfix

Updated execution permissions for the executable in the Grafana OpenSearch plugin.
resolved imagePullSecrets warning in Kubernetes events
added ressource limits

changelog kubeops-dashboard 1.7.0_Beta0

Bugfix

Fixed CVE-2024-45337 issue
resolved imagePullSecrets warning in Kubernetes events
added ressource limits

changelog rook-ceph 1.7.0_Beta0

Bugfix

Added missing ImagePullSecret in Rook Operator and Ceph Cluster.
resolved imagePullSecrets warning in Kubernetes events
added ressource limits
removed not templatable values

changelog harbor 1.7.0_Beta0

Bugfix

Added missing ImagePullSecret in Rook Operator and Ceph Cluster.
Resolved the PostgreSQL issue encountered during the Harbor upgrade.

changelog setup 1.7.0_Beta0

Updates

Added rhel9 support for kubedependencies, containerd, containerde-dependencies, nftablese

changelog lima 1.7.0_Beta0

New

Added Licence verification
added new version of calicomultus

Updates

Added rhel9 support for kubedependencies, containerd, containerde-dependencies, nftablese
Lima will now create the file /etc/systemd/system/containerd.service.d/override.conf in order to overwriten LimitNOFILE=Infinity with LimitNOFILE=1048576 for ceph support in Red Hat Enterprise 9
Lima will now add SystemdCgroup = true below [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] in file /etc/containerd/config.toml
Resolved the login issue for non-root users when creating a Lima container
fixed issue with reading clusterstorage.yaml
fixed issue with kubernetes upgrade to 1.32 versions
fixed issue with syntax error in lima update -r

Changelog cert-manager 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog filebeat 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog keycloak 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits
fixed cves, updated version

Changelog logstash 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog opa-gatekeeper 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog opensearch-dashboards 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog velero 1.7.0_Beta0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog kubevirt 1.0.0_Beta0

new

added package

Kubeopsctl: KubeOps 1.7.0_Alpha2

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.0_Alpha2 - Release Date 11.04.2025

Changelog kubeopsctl 1.7.0_Alpha2

Bugfix

Resolved issue with kubeopsctl status for keycloak

Changelog lima 1.7.0_Alpha2

Bugfix

Fixed a bug where ssh key support for ed25519 was not possible.

Changelog opensearch 1.7.0_Alpha2

Bugfix

Fixed a bug where the secret internal-users-config-secret was not created.

Kubeopsctl: KubeOps 1.7.0_Alpha1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.0_Alpha1 - Release Date 08.04.2025

Changelog nginx-ingress 1.7.0_Alpha1

Updates

Upgraded Helm chart to version 4.11.5 to address CVE-2025-1974.

changelog prometheus 1.7.0_Alpha1

Bugfix

Updated execution permissions for the executable in the Grafana OpenSearch plugin.

changelog kubeops-dashboard 1.7.0_Alpha1

Bugfix

Fixed CVE-2024-45337 issue

changelog rook-ceph 1.7.0_Alpha1

Bugfix

Added missing ImagePullSecret in Rook Operator and Ceph Cluster.

changelog harbor 1.7.0_Alpha1

Bugfix

Added missing ImagePullSecret in Rook Operator and Ceph Cluster.

changelog setup 1.7.0_Alpha1

Updates

Added rhel9 support for kubedependencies, containerd, containerde-dependencies, nftablese

changelog lima 1.7.0_Alpha1

Updates

Added rhel9 support for kubedependencies, containerd, containerde-dependencies, nftablese
Lima will now create the file /etc/systemd/system/containerd.service.d/override.conf in order to overwriten LimitNOFILE=Infinity with LimitNOFILE=1048576 for ceph support in Red Hat Enterprise 9
Lima will now add SystemdCgroup = true below [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] in file /etc/containerd/config.toml

Kubeopsctl: KubeOps 1.7.0_Alpha0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.7.0_Alpha0 - Release Date 01.04.2025

Changelog kubeopsctl 1.7.0_Alpha0

New

Added Licence verification

Changelog lima 1.7.0_Alpha0

New

Added Licence verification
added new version of calicomultus

Bugfix

Resolved the login issue for non-root users when creating a Lima container
fixed issue with reading clusterstorage.yaml
fixed issue with kubernetes upgrade to 1.32 versions
fixed issue with syntax error in lima update -r

Changelog cert-manager 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog filebeat 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog nginx-ingress 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog keycloak 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits
fixed cves, updated version

changelog kubeops-dashboard 1.7.0_Alpha0

fixed

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog logstash 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog opa-gatekeeper 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog opensearch 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog opensearch-dashboards 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog prometheus 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog velero 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits

Changelog rook-ceph 1.7.0_Alpha0

Bugfix

resolved imagePullSecrets warning in Kubernetes events
added ressource limits
removed not templatable values

Changelog kubevirt 1.7.0_Alpha0

new

- added package

Kubeopsctl: KubeOps 1.6.11

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.11 - Release Date 30.09.2025

changelog kubeops-dashboard 1.6.11

fixed

Resolved issue with service account

Kubeopsctl: KubeOps 1.6.10

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.10 - Release Date 05.09.2025

changelog kubeopsctl 1.6.10

fixed

Prevented failure by adding fallback creation logic for missing namespace and secret

changelog rook-ceph 1.6.10

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog opensearch-dashboards 1.6.10

fixed

Removed external ingress and deployed using helm chart

changelog kubeops-dashboard 1.6.10

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog keycloak 1.6.10

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog harbor 1.6.10

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog prometheus 1.6.10

fixed

Removed external ingress and deployed using helm chart
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog filebeat 1.6.10

fixes

Addressed the following CVEs:
- CVE-2024-41110
- CVE-2024-45337
- CVE-2023-24538
- CVE-2023-24540
- CVE-2024-24790
Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog velero 1.6.10

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog cert-manager 1.6.10

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog OPA-Gatekeeper 1.6.10

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog nginx-ingress 1.6.10

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

changelog calico-multus

fixes

Updated imagePullPolicy to Always to comply with BSI security requirements.

Kubeopsctl: KubeOps 1.6.9

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.9 - Release Date 30.06.2025

changelog lima 1.6.9

new

- added logic for updating from RHEL 8 to RHEL 9

fixed

- fixed issue where auditlog path was not templatable

changelog rook-ceph 1.6.9

fixed

- fixed issue where the lableing was only after installing platform packages
- added logic for cerating crushmap for rook-ceph

changelog rook-ceph 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog harbor 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opensearch-os 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opensearch-dashboards 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog opa-gatekeeper 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog cert-manager 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog ingress-nginx 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog keycloak 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog velero 1.6.9

fixed

- added topologySpreadConstraints for spreading across the kubeops zones

changelog kubeopsctl 1.6.9

fixed

- fixed issue where auditlog path was not templatable

changelog Velero 1.6.9

fixes

Addressed the following CVEs:
- CVE-2024-24790
- CVE-2024-32002
- CVE-2024-45491
- CVE-2024-45492
- CVE-2024-37371
- CVE-2023-45853

Kubeopsctl: KubeOps 1.6.8

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.8 - Release Date 30.04.2025

changelog lima 1.6.8

new

added logic for creating rook-ceph on rhel 9
added new containerd and containerd packages for rhel9

fixed

fixed issue where setting the auditlog parameter to false did nothing
fixed issue where setting the auditlog parameter to true only made changes on the clustermaster

changelog opensearch 1.6.8

changed

updated opensearch to version: 2.32.0 and appVersion: 2.19.1
Issue with creating and accessing secrets in OpenSearch has been resolved.
Issue with creating namespace for opensearch has been resolved

changelog opensearch dashboards 1.6.8

changed

updated opensearch to version: 2.28.0 and appVersion: 2.19.1

changelog prometheus 1.6.8

fixed

fixed issue where grafana opnsearch datasource hat wrong permissions

changelog ingress nginx 1.6.8

changed

updated helm chart to appVersion: 1.11.5 and version: 4.11.5

fixed

fixed critical cves:
- CVE-2025-1097
- CVE-2025-1098
- CVE-2025-1974
- CVE-2025-24513
- CVE-2025-24514
fixed issue where advanced parameters did not work

changelog kubeopsctl 1.6.8

fixed

fixed issue where clusterstorage was not updated after kubeopsctl commands
fixed issue where kubeopsctl version command did not create a output

Kubeopsctl: KubeOps 1.6.7

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.7 - Release Date 27.03.2025

changelog lima 1.6.7

fixed

fixed issue in kubernetes 1.31.2 where the pause image has the tag 3.10.0 instead of 3.10
fixed issue in kubernetes 1.31.4 where the pause image has the tag 3.10.0 instead of 3.10
fixed issue where the kubernetes upgrade from 1.30 to 1.31 failed with the CreateContainerConfigError
fixed issue where the kubernetes upgrade from 1.30 to 1.31 only applied on the worker nodes
fixed issue where the kubernetes upgrade from 1.30 to 1.31 create a version conflict between the kube-apiserver of an upgraded master and an not upgraded master

changelog kubeopsctl 1.6.7

fixed

fixed issue where the kubernetes upgrade from 1.30 to 1.31 only applied on the worker nodes
fixed issue where the kubernetes upgrade from 1.30 to 1.31 create a version conflict between the kube-apiserver of an upgraded master and an not upgraded master

changelog rook-ceph 1.6.7

fixed

fixed templating of mgr so that it can be modified via the advanced parameters

added

added ressourcenlimits for rook

changelog harbor 1.6.7

fixed

fixed templating of postgres so that it can be modified via the advanced parameters

added

added ressourcenlimits for harbor

changelog helm 1.6.7

added

added ressourcenlimits for helm

changelog cert-manager 1.6.7

added

added ressourcenlimits for cert-manager

changelog filebeat 1.6.7

added

added ressourcenlimits for filebeat

changelog ingress-nginx 1.6.7

added

added ressourcenlimits for ingress.nginx

changelog keycloak 1.6.7

added

added ressourcenlimits for keycloak

changelog kubeops-package 1.6.7

added

added ressourcenlimits for kubeops-package

changelog kubeops-setup 1.6.7

added

added ressourcenlimits for kubeops-setup

changelog logstash 1.6.7

added

added ressourcenlimits for logstash

changelog opa-gtekeeper 1.6.7

added

added ressourcenlimits for opa-gatekeeper

changelog opensearch 1.6.7

added

added ressourcenlimits for opensearch

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

changelog prometheus 1.6.7

added

added ressourcenlimits for prometheus

Kubeopsctl: KubeOps 1.6.6

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.6

changelog lima 1.6.6

fixed

fixed issue where a non exiting cluster storage lead to exceptions
fixed issue where the restart of the proxy in lima update registry lead to errors

changelog opensearch 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog opensearch-dashboards 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

changelog logstash 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog filebeat 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog prometheus 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog prometheus 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog opa-gatekeeper 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog cert-manager 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog ingress-nginx 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog keycloak 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

changelog velero 1.6.6

fixed

fixed issue where package had automatically a imagepullsecret from the local harbor

Kubeopsctl: KubeOps 1.6.5

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.5

changelog lima 1.6.5

added

added kubernetes version 1.29.12
added kubernetes version 1.30.8

changelog kubeops-dashboard 1.6.5

fixed

fixed issue where nodeport was randomly assigned

Changelog opensearch-dashboards 1.6.5

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

Kubeopsctl: KubeOps 1.6.4

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.4

changelog lima 1.6.4

added

added kubernetes version to 1.29.10
added kubernetes version to 1.30.6

changed

updated yq version in lima code for vulnerability fix

fixed

fixed issue where change auditlog only worked on clustermaster
fixed issue where clusterstorage was not complete after cluster creation

changelog rook-ceph 1.6.4

changed

updated app-version version of rook-ceph operator to 1.15.6, helm chart version to 1.15.6
updated helm chart version of rook-ceph cluster to 1.15.6, helm chart version to 1.15.6

changelog kube prometheus stack 1.6.4

changed

updated the ingress configuration to enable HTTPS access for Prometheus and Grafana services
updated to grafana plugin to 11.3.1
updated prometheus image because of critical cves

changelog kubeopsctl 1.6.4

fixed

fixed issue where change auditlog only worked on clustermaster
fixed issue where an warning message was thrown, that the node could be be upgraded, even if no upgrade was applied

changelog opensearch 1.6.4

added

added ressourcenlimits for opensearch

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

Kubeopsctl: KubeOps 1.6.3

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.3

changelog kubeopsctl 1.6.3

bugfixes

resolved issue where only masters in zones was not possible
resolved issue where installation of harbor without password and external url was possible
resolved issue where imagepullsecret was not created correctly for harbor with domain name
resolved issue where harbor password was in logs

changelog lima 1.6.3

bugfixes

Resolved the issue where the kubectl command was not found

added

added kubernetes version to 1.31.2

changelog rook-ceph 1.6.3

fixes

resolved issue with accessing ceph dashboard
fixed issue with updating rook-cephfs storageClass

Changelog Harbor 1.6.3

fixes

hardened images because of critical cves: CVE-2024-41110, CVE-2024-45491, CVE-2024-45492, CVE-2024-37371

Changelog prometheus-stack 1.6.3

fixes

changed scrape port of etcd for prometheus stack to 2381
resolved issues with prometheus crds in prometheus upgrade

Changelog nginx-ingress 1.6.3

fixes

fixed the critical vulnerability CVE-2024-24790

Changelog kubeops-dashboard 1.6.3

fixes

updated helm chart to 0.26.0 and app version 0.26.0

Changelog opensearch-dashboards 1.6.3

updates

upgraded Helm chart version to 2.22.0

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

Kubeopsctl: KubeOps 1.6.2

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.2

changelog kubeopsctl 1.6.2

bugfixes

resolved issue where images were not pinned, so images were deleted by kubernetes
resolved issue where updateRegistry was not executed
resolved issue where lima container did not properly stop after errors
fixed issue with storageClassName in logstashValues

changelog rook-ceph

fixes

resolved issue where parameters useallnodes and usealldevivces, and deviceslist were not templated

Changelog opa gatekeeper 1.6.2

fixes

fixed issue where opa gatekeeper namespace was controlled by helm
updated helm chart of opa gatekeeper, changed curl image

Changelog keycloak 1.6.2

fixes

fixed issue with critical CVEs

Changelog opensearch-dashboards 1.6.2

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

Kubeopsctl: KubeOps 1.6.1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.1

Changelog kubeopsctl 1.6.1

Bugfix

Resolved the issue of missing master nodes details in /usr/local/etc/haproxy/haproxy.cfg
Resolved the issue where no error message was displayed when running kubeopsctl apply -f with a non-existent YAML file

Changelog lima 1.6.1

Bugfix

Resolved issue with upgrading kubenetes versions

Changelog velero 1.6.1

Bugfix

Fixed issue with CVE-2024-24790 vulnerability

Changelog kubeops-dashboard 1.6.1

Bugfix

Fixed issue with CVE-2024-24790 and CVE-2024-41110 vulnerabilities

Changelog cert-manager 1.6.1

Bugfix

Upgraded the cert-manager Helm chart to version 1.15.3

Changelog prometheus 1.6.1

Bugfix

Upgraded the prometheus Helm chart to version 62.7.0

Changelog nginx-ingress 1.6.1

Bugfix

Fixed issue with CVE-2024-24790 vulnerability

Changelog harbor 1.6.1

Bugfix

Upgraded the prometheus Helm chart to version 1.15.1

Changelog opensearch-dashboards 1.6.1

known issue

the installation can create the following error: Failed loading builtin log types from disk! java.nio.file.FileSystemNotFoundException

Kubeopsctl: KubeOps 1.6.0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.0

Changelog kubeopsctl 1.6.0

New

Added apiVersion kubeops/kubeopsctl/alpha/v5
Added a overwrite parameters for platform packages such as rook, harbor, logstash, filebeat etc.

Bugfix

Fixed the issue where the platform was creating Harbor project despite it being set to false.
Resolved the issue with status giving output for invalid argument/s
Resolved the issue with multiline literal that converted “|” to “>”

Changelog lima 1.6.0

Bugfix

Resolved issue with upgrading of clustermaster twice

Changelog keycloak 1.6.0

Bugfix

Issue with CVE-2024-37371 vulnerability resolved.
Resolved issue with nodeport configuration

Changelog prometheus 1.6.0

Bugfix

Fixed the issue with the “ZBP” tag in the Grafana-Alertmanager dashboard.

Changelog kubeops-dashboard 1.6.0

Bugfix

Correct version has been adapted

Kubeopsctl: KubeOps 1.6.0-Beta1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.0-Beta1

Changelog kubeopsctl 1.6.0_Beta1

Bugfix

Resolved issue with log messages are written along with output on the screen
Resolved issue with support for API versions v1, v2, v3, and v4

Changelog lima 1.6.0_Beta1

Bugfix

Resolved issue with ‘NullReferenceException’ for unsupported packages
Resolved issue with logging channels

Kubeopsctl: KubeOps 1.6.0-Beta0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 1.6.0-Beta0

Changelog kubeopsctl 1.6.0_Beta0

New

Added apiVersion kubeops/kubeopsctl/alpha/v5
Added a tutorial for using advanced parameters to overwrite values

Bugfix

Moved advancedParameter under cluster and operator in rookValues
fixed issue where packages could not be upgraded
fixed issue with v4 to v5 model conversion

Known Issues

log messages are written along with output on the screen

Changelog lima 1.6.0_Beta0

Bugfix

added more parallelization in the clustercreation and update registry for more speed
fixed problem where kubernetes upgrade was not possible

Changelog opensearch 1.6.0_Beta0

Bugfix

The initial dashboard password has been set to a neutral value
fixed issue with ’nil pointer evaluating interface
Added a tutorial for using advanced parameters to overwrite values in kubeopsctl.yaml

Changelog rook-ceph 1.6.0_Beta0

Bugfix

Removed the image.tag specification from the values.yaml file in the Helm chart to allow for more dynamic configuration of the image version during deployments

Changlog nginx-ingress 1.6.0_Beta0

Bugfix

number of replicas set to 3

Changelog setup 1.6.0_Beta0

Bugfix

fixed issue with installing ‘kubectl’ on admin node
updated calicomultus version to 0.0.4

Changelog harbor 1.6.0_Beta0

Bugfix

set replicas to 3

Changelog filebeat 1.6.0_Beta0

Bugfix

changed toleration from master to control-plane

Kubeopsctl-2.0: KubeOps 2.1.1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 2.1.1 - Release Date ??.08.2026

Changelog KubeOpsCtl 2.1.1

Fixes

- fixed missing opensearch-dashboards plugins and wrong version

Versions KubeOps 2.1.1

Kubeopsctl-2.0: KubeOps 2.2.0_Beta1

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 2.2.0_Beta1 - Release Date 02.08.2026

Changelog KubeOpsCtl 2.2.0_Beta1

Whats new?

- Added Traefik support for kubeopsctl
- Added Kubevirt-Manager support for kubeopsctl
- Added Kubernetes 1.36 support
- Added Advanced Parameters for static Registry mount path, check out the file format [Cluster Values YAML](/kubeopsctl-2.0/reference/fileformats/#cluster-valuesyaml)

Fixes

- fixed a podman bug affecting pull packages
- fixed a harbor bug setting up cluster
- added creation of nginx ingressclass, when installing traefik for backwards compatabilty
- fixed default secret for kubeops-dashboard
- kubeopsctl pull command now verifies if the specified tool value file exists
- Added kubeops-harbor-secret for pulling images from the internal private harbor project 'kubeops'
- fixed a bug that prevented to create the kubeops harbor project as private 
- fixed a bug that prevented to change the kubeops harbor project to private 
- fixed a bug that prevented the image push of the kubernetes images to local-registry 
- changed the 'localhost' reference to '127.0.0.1' for enterprise-tools in preperation for the kubernetes 1.36 support
- fix image push in local registry
- fixed duplicated coredns in 1.36.1
- updated kosi versions for consistent installation
- updated Harbor ImagePullSecret with 127.0.0.1 instead of master1 IP
- remove hardcoded plugin versions in cluster prepare node
- fixed a bug in rook delete section
- fixed a bug in nginx ingressclass for traefik
- added default secret for grafana
- added role bindings for the kubevirt roles
- fixed delete ingresses in update process 
- OpenSearch users created in the Dashboards Security UI now persist across package updates 
- OpenSearch TLS certificates are no longer regenerated on every install/update, preventing certificate rotation on running clusters 
- Fixed OpenSearch SSO returning 401 Unauthorized by reliably applying the openid security configuration to the backend after deploy 
- Added a Keycloak protocol mapper so realm roles are propagated to OpenSearch for backend role mapping 
- Added Single Sign-On (SSO) with Keycloak (OpenID Connect) for OpenSearch and OpenSearch Dashboards 
- Added username/password fallback authentication for OpenSearch alongside SSO 
- Made the OpenSearch admin password configurable via the new `adminPassword` package value (opensearch-os & opensearch-dashboards) 
- Added `hostname` package value for opensearch-os to point OpenSearch at the Keycloak SSO endpoint 
- Refresh the Keycloak admin token before each step of the OpenSearch Dashboards OIDC join so long-running joins no longer fail with 401 once the 60s token lifespan is exceeded
- fixed an error in OpenSearch when it was not installed in the `logging` namespace
- fixed an issue in kubeopsctl where `tools-values.yaml` and `cluster-values.yaml` were not always stored as secrets in the cluster
- changed the netpols for traefik-nginx and ingress-nginx to make the dashboards available
- helm-plugin now parses the flags correctly in all possible actions (install, upgrade, delete)
- fixed missing kosi update on nodes, if packageRepository: local
- fixed missing opensearch-dashboards plugins and wrong version
- template network policies with default deny rules
- when installing the tools, apply cluster- and tools-values

Known Issues

⚠ Warning In an air-gapped scenario, the Harbor external URL cannot be set to a public domain name. It must remain http://«controlplane1-ip»:30002.

⚠ Warning WireGuard installation may fail when packagerepo: local is used on cluster nodes that are not running Red Hat Enterprise Linux 9.6, due to its dependency on the systemd-resolved version included in the release.

⚠ Warning If Cilium is selected as the CNI, all dashboards are currently unavailable.

Versions KubeOps 2.2.0_Beta1

Tool	App Version	Chart Version	Package Version	SHA256 Checksum
kernelmodules			kubeops/kernelmodules:2.2.0_Beta1	476f5be30871160d6fef47f791f601f7a090a988dc0dd86de3c237b5aa5ef986
sysctl			kubeops/sysctl:2.2.0_Beta1	3361e60dc684be44da3fbf6da355074219c4d03006a77445ec26b524f61ce83e
disable-swap			kubeops/disable-swap:2.2.0_Beta1	e7fffe5cb73f9079c5d591293ba6b324314bd579c14dc4ff9a9866c356d11c09
runtime-setup			kubeops/runtime-setup:2.2.0_Beta1	08c5d65c06522ef7a96f9218f9cced7969baf1a77170c922dd582c515a767c78
kubeadm-init			kubeops/kubeadm-init:2.2.0_Beta1	a42cb150cfd8b722c54736951b1008ef28a52972dd448b03e2b473d8db868f75
local-registry			kubeops/local-registry:2.2.0_Beta1	cd184da0d8963c7afa0226b452d5c90223d6c74a51430141f3545f9c1b13f4c2
load-balancer			kubeops/load-balancer:2.2.0_Beta1	ee9bf297c642b35e64da3a2ff5b6351a784afd4a50289affdeb881b0fca6a614
initial-controlplane			kubeops/initial-controlplane:2.2.0_Beta1	3eeca78df9a2936886a8de3fd34bcd6f13e863a17219e7f971a58f8c1a7bae79
clustersetup			kubeops/clustersetup:2.2.0_Beta1	823088a03fa9a79e1098ca79df305fec2a2f40f65c8727d9d2125b99c4adac9d
kubernetes-cni			kubeops/kubernetes-cni:2.2.0_Beta1	a95ce6f24430a419541a747d48d7e8647eb970c21f73ffc632e676b6577c9288
join-controlplane			kubeops/join-controlplane:2.2.0_Beta1	cce1d2f21ed4fdb5cd5bd8af33a3a17bc63a5b6cf6b2ba8fbaa078f5675c0151
join-worker			kubeops/join-worker:2.2.0_Beta1	8ce7e876cd88c6db2ae189d666c1e1cb6dabad09e1ea7e5131bcc852112b4b20
pull-packages			kubeops/pull-packages:2.2.0_Beta1	2c1ab9b35d0c56a027471ec83da24a6bdcb1184005a03877aff04379b7ecd13c
prepare-node			kubeops/prepare-node:2.2.0_Beta1	9d197abcb9c845f47f6220e44b4be3d7af45e5c06c0ae9734001bd677f338dab
firewall-setup			kubeops/firewall-setup:2.2.0_Beta1	2ec1d78c693956e2bf7617d9b8cf695ecfc41c259e45f03e3fc59039f9428483
kube-vip			kubeops/kube-vip:2.2.0_Beta1	21db077376cb1c69d6ada941aae9174a479fdbd5e397e8757a1034391a3bc0c6
delete-node			kubeops/delete-node:2.2.0_Beta1	6745ba0018a95682a203205313ba5680d23cf26ac2e2b9263eaad2e372c77a4a
auditlog			kubeops/auditlog:2.2.0_Beta1	a513f525362fd69bdd82f006f4cda1498ffb32fdd91c079a9a4a90c27475943a
kubeadm-upgrade-initial-controlplane			kubeops/kubeadm-upgrade-initial-controlplane:2.2.0_Beta1	240d9754f3221347c352cafbde7d84a818c6a384913626825712f94c6bd04b55
kubeadm-upgrade-nodes			kubeops/kubeadm-upgrade-nodes:2.2.0_Beta1	3bfaa2a1142318c3fce2212ef183313dc03d57435e84e4d86f3ede042e32c4a7
kubernetes-tools			kubeops/kubernetes-tools:1.30.8	ee9e8e41552be2d90f1f19700c251fd77fe01da31eabec6a3fcab318b01ceb2c
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.30.8	2677d909cd257c0b75de2d4bd9e9d198148310a9ecbbca384996700a54d3a9df
kubernetes-tools			kubeops/kubernetes-tools:1.31.6	0af110107067c17851375742d301f1da68aac2f732b626f8210ece17e17fd30b
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.31.6	41ac4282be31a9127ca403be3d0f03bab912e16e66f05febb506479574b3c4f1
kubernetes-tools			kubeops/kubernetes-tools:1.32.2	9ddc947d61b69ff6121809d016411ffe50732a78b760e5d307f03dc24294e8da
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.32.2	0b6c07788c320786b5f7cce1c809bd7305e28fecb804c223fc9b1dae6ee7bb1b
kubernetes-tools			kubeops/kubernetes-tools:1.32.9	5587a13d33096c92f52ced4947815c44554a82a49091dd9d78c8eaa065534f7c
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.32.9	e4c34ddf29889d6a8f9075752a61a61fc62b7b7efa5e981d0d4ccbfc5cc1a939
kubernetes-tools			kubeops/kubernetes-tools:1.33.5	2598dd0c50a7f1d38f1929f34bbfe0ce7efccb614b9f48d417c9285cb3e1b949
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.33.5	1f6000a20f57c17cf8c229269cb4c3abf076f68f336234d79dbb24656f4190f2
kubernetes-tools			kubeops/kubernetes-tools:1.34.1	69ea34b7768e8de460f75b6a165a63b8de3e032d584fd43c4da87474d2e046ca
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.34.1	32f294ab19e6a2b3769044a728c0d6a4bebeb1d393aee061d68ebef0a7c83b2b
kubernetes-tools			kubeops/kubernetes-tools:1.35.0	00fdeb8f3816b04661c9d866c89a3fa79dd743c1989709a7d7ff581010b9d7e6
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.35.0	64e3c2fbc81d2973cc28d05cc517687570c07d8b3a9194fe0ee33455669efc58
kubernetes-tools			kubeops/kubernetes-tools:1.36.1	2b4ce27d375937f71b1760e93e19668de5e3c594ab332521cf7c1109560f0e34
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.36.1	079962600f3b361904d4972b6c59c10396630f62da0a33e6d1889142a10a0540
calico	v3.29.1	v3.29.1	kubeops/calico:v3.29.2	3d492518bb31ad078f9e6e091ff2d22e43cac5a8b3f84c127571a1545a891833
calico-airgap			kubeops/calico-airgap:v3.29.2	5923f633ea4136a64f0c0ddf56069b109704b61e542a530378a686292b4d034c
cilium	1.18.2	1.18.2	kubeops/cilium:v1.18.3	5451104fd9703d6039b6d6da6912bca5db62d9942f24b76549314a27774f0459
cilium-airgap			kubeops/cilium-airgap:v1.18.3	a334071a23cdd2330ea2c723993e18822522c900481fabe7f1ee12b4c849f5ab
multus			kubeops/multus:snapshot-thick2	3585956f2b1b1687ba01f2592682fda07dd34005f169666cc815a289b9945130
multus-airgap			kubeops/multus-airgap:snapshot-thick2	746eee2272ab13281d7540a21d696226e806c0d13380fdcce554564c7b84e709
wireguard-tools			kubeops/wireguard-tools:v1.0.20210914	70a49e1fec74e5c6b977e40264e1a084cd2f25d38a9df034141a9e165aea0f8c
helm			kubeops/helm:v3.16.4	8e2def3efd54f2d149235e3ef71116c37514f7dcda81e8c793e081120c16f323
podman			kubeops/podman:5.2.2	faa2ca5552df3cf46da6b0c4afc55220bb751e59319c1157e7f3652479bde4e4
ingress-nginx	1.15.6	4.15.6	kubeops/ingress-nginx:2.2.0_Beta1	d9db2892127d3dd895439854c33d0ee7aa6c8280b134e34c0db34bb292a70128
cert-manager	v1.18.2	v1.18.2	kubeops/cert-manager:2.2.0_Beta1	2f146fdfc04ed39454d606ac0fdbaef792c9c414c8ae27a589b67320b442a528
opa-gatekeeper	v3.17.1	3.17.1	kubeops/opa-gatekeeper:2.2.0_Beta1	ac240161a7e1bcbc7ae62baa9d8af038d01725d6e0a580229730aaa10436a3bf
velero	1.16.2	10.1.0	kubeops/velero:2.2.0_Beta1	b9cf523e5d84b5aba7cf70a106fb38036abfd3182bee0f4b1fbf8450256d3ac9
rook-ceph	v1.15.6	v1.15.6	kubeops/rook-ceph:2.2.0_Beta1	8507a3890439307573305c3412ed8a0605d89502b295c2933f5ff0ab3c691784
rook-ceph-airgap			kubeops/rook-ceph-airgap:2.2.0_Beta1	63ba9218f2947992fb0878afe645d349c6d7c3c22e6f0c91ea9e2fad43f05101
harbor	2.14.0	1.18.0	kubeops/harbor:2.2.0_Beta1	bed605d28b7ab75076aff6e9cdd4c84e694897fa8b4b8537e515042740a04664
harbor-airgap			kubeops/harbor-airgap:2.2.0_Beta1	5f11608c2056e38615655568b53112a3d5436b12c0cf9e6b457f5ea3884d8b70
kube-prometheus-stack	v0.85.0	77.0.2	kubeops/kube-prometheus-stack:2.2.0_Beta1	72690ac02de26e89fd22ab5c87ce4cf16b0d6f1e3bfa9c3abf626eb2f91556e6
keycloak	16.8.0	1.2.0	kubeops/keycloak:2.2.0_Beta1	fc50a1ed40b6ec5c57793f0ae5841565925794c4508a9be9fa7351aec0f506d7
kubeops-dashboard	0.26.0	0.26.0	kubeops/kubeops-dashboard:2.2.0_Beta1	ac2e1f7230adc914ec89c5d0ade57aacf917a2aa388b8897126ff7fcf0add526
filebeat-os	8.5.1	8.5.1	kubeops/filebeat-os:2.2.0_Beta1	28e96f41bb3869a2cc360aed2117dc4a60dba115c62f74ef8016334a50b0c9ce
logstash-os	8.4.0	8.4.0	kubeops/logstash-os:2.2.0_Beta1	d73fbac1760df25e7428cb717378fe3e38d98ac7c8f866e025fa9cb1eb7ec9c4
opensearch-os	3.4.0	3.4.0	kubeops/opensearch-os:2.2.0_Beta1	a11a77bb0e952c5269495720655004887ef939164fbe041fc9970c145b264b2a
opensearch-dashboards	3.4.0	3.4.0	kubeops/opensearch-dashboards:2.2.0_Beta1	316d7f0b3a5afa70e4822f1bddb0b14f579e4cd1fd54cb779ea9c0627f7489b9
traefik	v3.6.7	39.0.0	kubeops/traefik:2.2.0_Beta1	b53231c94b4b72bdeee41f3dc44effb15937eb4ca78c864611b7fbca7413ed4b
kubevirt-manager			kubeops/kubevirt-manager:2.2.0_Beta1	bff3c27b0fa086e1698472af19b5f7dc9f65be01de56c7f9982732418cffbb0c
kubevirt			kubeops/kubevirt:2.2.0_Beta1	f569cd38ddeb894eeac4cf65f9758d1c8effc200ec214dcfe80ce569d5e34011

Kubeopsctl-2.0: KubeOps 2.2.0_Beta0

Mon, 01 Jan 0001 00:00:00 +0000

KubeOps 2.2.0_Beta0 - Release Date 30.07.2026

Changelog KubeOpsCtl 2.2.0_Beta0

Whats new?

- Added Traefik support for kubeopsctl
- Added Kubevirt-Manager support for kubeopsctl
- Added Kubernetes 1.36 support
- Added Advanced Parameters for static Registry mount path, check out the file format [Cluster Values YAML](/kubeopsctl-2.0/reference/fileformats/#cluster-valuesyaml)

Fixes

- fixed a podman bug affecting pull packages
- fixed a harbor bug setting up cluster
- added creation of nginx ingressclass, when installing traefik for backwards compatabilty
- fixed default secret for kubeops-dashboard
- kubeopsctl pull command now verifies if the specified tool value file exists
- Added kubeops-harbor-secret for pulling images from the internal private harbor project 'kubeops'
- fixed a bug that prevented to create the kubeops harbor project as private 
- fixed a bug that prevented to change the kubeops harbor project to private 
- fixed a bug that prevented the image push of the kubernetes images to local-registry 
- changed the 'localhost' reference to '127.0.0.1' for enterprise-tools in preperation for the kubernetes 1.36 support
- fix image push in local registry
- fixed duplicated coredns in 1.36.1
- updated kosi versions for consistent installation
- updated Harbor ImagePullSecret with 127.0.0.1 instead of master1 IP
- remove hardcoded plugin versions in cluster prepare node
- fixed a bug in rook delete section
- fixed a bug in nginx ingressclass for traefik
- added default secret for grafana
- added role bindings for the kubevirt roles
- fixed delete ingresses in update process 
- OpenSearch users created in the Dashboards Security UI now persist across package updates 
- OpenSearch TLS certificates are no longer regenerated on every install/update, preventing certificate rotation on running clusters 
- Fixed OpenSearch SSO returning 401 Unauthorized by reliably applying the openid security configuration to the backend after deploy 
- Added a Keycloak protocol mapper so realm roles are propagated to OpenSearch for backend role mapping 
- Added Single Sign-On (SSO) with Keycloak (OpenID Connect) for OpenSearch and OpenSearch Dashboards 
- Added username/password fallback authentication for OpenSearch alongside SSO 
- Made the OpenSearch admin password configurable via the new `adminPassword` package value (opensearch-os & opensearch-dashboards) 
- Added `hostname` package value for opensearch-os to point OpenSearch at the Keycloak SSO endpoint 
- Refresh the Keycloak admin token before each step of the OpenSearch Dashboards OIDC join so long-running joins no longer fail with 401 once the 60s token lifespan is exceeded

Known Issues

⚠ Warning In an air-gapped scenario, the Harbor external URL cannot be set to a public domain name. It must remain http://«controlplane1-ip»:30002.

⚠ Warning OpenSearch is not installed if a namespace other than logging is used.

⚠ Warning If Cilium is selected as the CNI, all dashboards are currently unavailable.

Versions KubeOps 2.2.0_Beta0

Tool	App Version	Chart Version	Package Version	SHA256 Checksum
kernelmodules			kubeops/kernelmodules:2.2.0_Beta0	68d889dc1af5d7178ba726780002ef6d55632f8983cbde9bbeb6b62226d88715
sysctl			kubeops/sysctl:2.2.0_Beta0	9383df73d75b712ba901a908a9164dcf2ee73c24816bc83ed6551576a67c3d48
disable-swap			kubeops/disable-swap:2.2.0_Beta0	5a1649104ffd6eef6624a58c7f21f1afec0815b5ded7a377e03f0cb222cec6a9
runtime-setup			kubeops/runtime-setup:2.2.0_Beta0	ebb23938d9adba0291efdd452df0a609495a62687d38564639e033d8af6a02ab
kubeadm-init			kubeops/kubeadm-init:2.2.0_Beta0	e0cb5a6f290bfcef9d44af0777f2f5f7889973192dcca90af5591bc12915b3c0
local-registry			kubeops/local-registry:2.2.0_Beta0	d8be4d7857acb6d4a934ed78f4d189006d7196dbe6b20c9c5641bdcf92c05ac0
load-balancer			kubeops/load-balancer:2.2.0_Beta0	1b325d8da7f18dc31baff8f2be372e8c5e425dd30135238e791bfc21c416070f
initial-controlplane			kubeops/initial-controlplane:2.2.0_Beta0	fd1f442342336d2d6c728dfe6b0da4e939701014f883042c370d7772df9eeeb8
clustersetup			kubeops/clustersetup:2.2.0_Beta0	279fb5152212362d25d1a342d1f277bd231fb9a02f4726b14bcba1cbb0fc864f
kubernetes-cni			kubeops/kubernetes-cni:2.2.0_Beta0	95925111bd30f34e620763567012991ade1bdb5f29636ca8591e8e5723f4c864
join-controlplane			kubeops/join-controlplane:2.2.0_Beta0	6a4bdd593b15bcc1ca2d8a0283ae860287d17ebc7e83ca802847aa89f59c97b2
join-worker			kubeops/join-worker:2.2.0_Beta0	2247094d87204722f2891db7599ebcd1bbdf581467b19d5ed2c2e22a31185086
pull-packages			kubeops/pull-packages:2.2.0_Beta0	b4823f7c72cf684f442f9bc2363997f58cb582f36ac9ab0081dfc3842bedca57
prepare-node			kubeops/prepare-node:2.2.0_Beta0	f02718afb5ce35561384bc8cda3d541dbf45dc2adfc8a87237813299c1f3adb9
firewall-setup			kubeops/firewall-setup:2.2.0_Beta0	f23fb5a14ca317ad6b432c5331048e83176592fb648352df4c8739d73e2f80f1
kube-vip			kubeops/kube-vip:2.2.0_Beta0	8c2076a66c589839c1c25da3311248da167a372abdea68d79e3b028691a27851
delete-node			kubeops/delete-node:2.2.0_Beta0	f874d4d02b72a3e6d5bce8669df1aa203cb9a9f8caaabe96d67974e8bf115315
auditlog			kubeops/auditlog:2.2.0_Beta0	c848ce79a3ab528a853efbcb671eaf1ff200db2319bb05fd167139b1ac31e9ba
kubeadm-upgrade-initial-controlplane			kubeops/kubeadm-upgrade-initial-controlplane:2.2.0_Beta0	4506d301c855428cc929156715e3af1c61cba0a33e6d084472b011cf81a812e4
kubeadm-upgrade-nodes			kubeops/kubeadm-upgrade-nodes:2.2.0_Beta0	736023f7a1924a40a82f01fc18db3688d114e4329c454970e8c90fd339b29c46
kubernetes-tools			kubeops/kubernetes-tools:1.30.8	ee9e8e41552be2d90f1f19700c251fd77fe01da31eabec6a3fcab318b01ceb2c
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.30.8	2677d909cd257c0b75de2d4bd9e9d198148310a9ecbbca384996700a54d3a9df
kubernetes-tools			kubeops/kubernetes-tools:1.31.6	0af110107067c17851375742d301f1da68aac2f732b626f8210ece17e17fd30b
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.31.6	41ac4282be31a9127ca403be3d0f03bab912e16e66f05febb506479574b3c4f1
kubernetes-tools			kubeops/kubernetes-tools:1.32.2	f7d5191a269dacfa12ad867b28b01bb7a80f506b6b01950c8a34c358bb22c6e3
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.32.2	954093914536cb21f6e8b407bd568210992d59db7ce80464ce12dd49679f4c35
kubernetes-tools			kubeops/kubernetes-tools:1.32.9	6baff46de7291b425f42d64d7563a3e3e11acf9abb27b1b8b467bab0fa3b0485
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.32.9	d421afe96680942236f1576f453a9685e99e93496c0ac9cc4d7a61c0dfcf2037
kubernetes-tools			kubeops/kubernetes-tools:1.33.5	8b3e5ebbbfb8bdd4ce30990ab80f2a7ffadabeb0890f8fcf7f9365d9a3d2ef20
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.33.5	43960d74ba78ced9112aa5d46f58ffc053764a194e79631560b8ae79e17978ea
kubernetes-tools			kubeops/kubernetes-tools:1.34.1	be69bc3240dda743b185b8922e42ef2a535b24d4a01e83162e24c702d2214fea
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.34.1	20bec465d22a385cf72de606a1a7ee599cc4b7e5cc81a7e7b7c4aecbb89606a0
kubernetes-tools			kubeops/kubernetes-tools:1.35.0	ff2f7135ca0d3d1ecaa67d8463db4af75b06542024105c2b8fb6d0f02ddefc10
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.35.0	9d4b6dd4c9f9090ab8d20f1614f009b68c3479b07cdb347af57c3b5829932f05
kubernetes-tools			kubeops/kubernetes-tools:1.36.1	ed52d7537780edd3c434f9233f875e1a59c73cc2b2b9426bc6f3d43a2ee2d079
kubernetes-tools-airgap			kubeops/kubernetes-tools-airgap:1.36.1	e3d89e76f9924d27b7073353fb597442a23aeed8f8d797f5b76537acefb2e8da
calico	v3.29.1	v3.29.1	kubeops/calico:v3.29.2	64c1c281c2a87b2a158fd182e14301a8bf6f19f9ea5b07aa8a5d5d7fe94d9698
calico-airgap			kubeops/calico-airgap:v3.29.2	5923f633ea4136a64f0c0ddf56069b109704b61e542a530378a686292b4d034c
cilium	1.18.2	1.18.2	kubeops/cilium:v1.18.3	c4913943eaf2bea8e5fc75c57792866a28bbd64e577e6e2b7b049dda69d067f7
cilium-airgap			kubeops/cilium-airgap:v1.18.3	a334071a23cdd2330ea2c723993e18822522c900481fabe7f1ee12b4c849f5ab
multus			kubeops/multus:snapshot-thick2	186439bca4db89a9e23364bb77d000ccf906262d1db8d06716801563f9f9a72e
multus-airgap			kubeops/multus-airgap:snapshot-thick2	5a7f258eca3d534f37547de3c8126fba2c3e5e885fb6710abd62f08f22ede6de
wireguard-tools			kubeops/wireguard-tools:v1.0.20210914	a6a521dfef6a550a658a5626cb15ab935cb1bfff99f82a2ced7b21317ab12fed
helm			kubeops/helm:v3.16.4	8e2def3efd54f2d149235e3ef71116c37514f7dcda81e8c793e081120c16f323
podman			kubeops/podman:5.2.2	309d2cbae83687ef2bf151ad6486824348e807348bcd7cdadbecc1bc40db4f86
ingress-nginx	1.15.6	4.15.6	kubeops/ingress-nginx:2.2.0_Beta0	57640157dbe06f169ab3e6a770a97bc83fb35269096bff37db498c5309e60bd3
cert-manager	v1.18.2	v1.18.2	kubeops/cert-manager:2.2.0_Beta0	80870509663ad76eca9d2ec7147f43d1a89e21f1e31cb273d8d534c48621a47c
opa-gatekeeper	v3.17.1	3.17.1	kubeops/opa-gatekeeper:2.2.0_Beta0	dae0d519d56b4044c771105047df68c76db61df4e2660309714a118b220b88a8
velero	1.16.2	10.1.0	kubeops/velero:2.2.0_Beta0	0769402efb9aae7f0d3c404aa5123f01d2324b1db167717147da324978a4d199
rook-ceph	v1.15.6	v1.15.6	kubeops/rook-ceph:2.2.0_Beta0	0d98891d643524020f7062db66b6142b56d95933e8c118c456c4f24fc74e888d
rook-ceph-airgap			kubeops/rook-ceph-airgap:2.2.0_Beta0	de2379884bcc658660706aef4729917f5fe41e093c8d9f5098d17a3dc2dd48b3
harbor	2.14.0	1.18.0	kubeops/harbor:2.2.0_Beta0	34d3b20326be1be7e69d5242730394d4750c2b908378e543bc69b4d7a049db86
harbor-airgap			kubeops/harbor-airgap:2.2.0_Beta0	406e9fe9e8a77d900f3126e93c215f82d8dbbb680554f7ce003ecb5dec5f5c4f
kube-prometheus-stack	v0.85.0	77.0.2	kubeops/kube-prometheus-stack:2.2.0_Beta0	bc2c5da217a58fa67040535dfbdd50933ee6f84411e7162e6c3de366275aa3c7
keycloak	16.8.0	1.2.0	kubeops/keycloak:2.2.0_Beta0	a18016f2079eab2153a594f17bd0e91706eac6a264bbfabf4bd016d2e7148734
kubeops-dashboard	0.26.0	0.26.0	kubeops/kubeops-dashboard:2.2.0_Beta0	d4922db9d47c5569ae563a96d72524fa8a0e54b5e2578f1c71b91a5e214b484d
filebeat-os	8.5.1	8.5.1	kubeops/filebeat-os:2.2.0_Beta0	c98242a2488b0cd28782ba5435ca8bed6db6dc5a1d78625291192ad06a7db705
logstash-os	8.4.0	8.5.1	kubeops/logstash-os:2.2.0_Beta0	3e07746ce9b02d08ddcc65e111f64435c0603819ae197c14fa0516bf237826c7
opensearch-os	3.4.0	3.4.0	kubeops/opensearch-os:2.2.0_Beta0	755d96edd69433bbb2027833cea3768efc70ae717382b8dc89d2e290a18be3cf
opensearch-dashboards	3.4.0	3.4.0	kubeops/opensearch-dashboards:2.2.0_Beta0	c7dcf941550bd41945cb589721a37f98f2e4593eb4d47d29499e12fd4f9a38ca
traefik	v3.6.7	39.0.0	kubeops/traefik:2.2.0_Beta0	a03d24eb8d153edcf4deefd8dd9a605be544e368018b9e9a0be06dcabe7579bf
kubevirt-manager			kubeops/kubevirt-manager:2.2.0_Beta0	22b89ccbd8b15fafd8ce25e7ba1b3c4818b81785e8d07d7414f022a1d52757a7
kubevirt			kubeops/kubevirt:2.2.0_Beta0	c41a51da0fa7cc21d1fa0f10914acb62e6473fb27c0e34b68d6f4e435c50f361

KubeOpsDoku – documentation

Kubeopsvm: About KubeOpsVM

What Is KubeOpsVM?

Why KubeOpsVM Matters

Key Features Driving Innovation

A Vision for the Future

Kubeopsvm: Detailed Explanation for Configuration File for Custom VMs

Kubeopsvm: How to install KubeOps Virtualization (kubevirt)

Requirements

Step 1 - Download KubeOpsVM

Step 2 - Remove the Taint of the Master Node (only necessary on single node cluster)

Step 3 - Create Kubevirt Configuration File

Step 4 - Install KubeOps-VM

Step 5 - Pull and Install KubeVirt Manager Package

Next Steps

Kubeopsvm: Release Notes

Kubeopsvm: Troubleshooting

Troubleshooting

virt-handler pods only 2/3 ready and status is CrashLoopBackOff

Solution

Kubeopsvm: Commandline Operations

Create a new virtual machine

Start the virtual machine

Status of a virtual machine

Status of a virtual machine instance

Stop the virtual machine instance

Restart the virtual machine

Cascade delete (implicit)

Cascade delete (explicit)

Delete orphans

Kubeopsvm: General Concepts of KubeOpsVM

What is the purpose of a Virtual Machine?

How does KubeOpsVM work?

Kubeopsvm: Getting Started

Kubeopsvm: How to create and deploy a Custom Virtual Machine using KubeOps

Requirements

Additional Information

How to create and deploy a Custom Virtual Machine

Step 1 - Create a Configuration File for the Virtual Machine

Step 2 - Transfer the VM Image to your Local Registry

Step 3 - Deploy the Virtual Machine

Step 4 - Access the Custom VM

References

Kubeopsvm: How to access a virtual machine in a KubeOps Cluster

Requirements

Step 1 - Identify your Virtual Machine

Step 2 - Access the Virtual Machine

Step 1 - Create a Service forwarding the SSH port of the VM

Step 2 - Access the Custom VM

Kubeopsvm: How To Guides

Kubeopsvm: Install KubeOps virtualization on a Dedicated Machine

Requirements

System Requirements

Step 1 - Download the KubeOps-VM ISO File

Step 2 - Install the KubeOps-VM ISO

Step 3 - Login to the Control-plane Node on the Cluster

Step 4 - Install the example VM

Step 5 - Access the example virtual machine

Next Steps

Kubeopsctl-2.0: KubeOps Compliance 2.0

Kubeopsvm: KubeOpsVM

Kubeopsvm: Reference

Kubeopsvm: How to create a ContainerDisk from a local image

Kubeopsvm: How to import a VM from vSphere and deploy it in a Cluster using KubeVirt

Requirements

Step 1 - Export OVF Template from vSphere

Step 2 - Convert the VMDK File to QCOW2 Format

Step 3 - Create a Container Image from the QCOW2 File

Step 4 - Deploy the VM using KubeVirt

Step 5 - Verify Deployment

Kubeopsvm: How to Monitor KubeVirt in Grafana

Monitor KubeVirt VMIs with Grafana and Prometheus

Prerequisites

Step 1: Ensure Prometheus is Collecting KubeVirt Metrics

Step 2: Import the KubeVirt Grafana Dashboard

Step 3: Customize the Dashboard (Optional)

Conclusion

Kubeopsvm: How to create L3 networking for VMs

Prerequisites

Configuring Node Networking using Network Manager State (NMState)