Originally published at: Comparing for Clarity: K8ssandra v1.x and the K8ssandra-operator - K8ssandra, Apache Cassandra® on Kubernetes
We’ve recently released K8ssandra-operator v1.0, which created some confusion as to how this new project relates to the original K8ssandra v1.x releases and which project should be picked for new deployments. While we expected things wouldn’t be crystal clear for everyone, there were some good reasons for that choice.
In this blog post, we will try to answer some questions about how both projects relate to each other, how they differ and which one should be used for upgrades and new projects.
What are K8ssandra and K8ssandra-operator?
We created K8ssandra-operator as the next step for K8ssandra, but as a separate project: the former is a Kubernetes operator while the latter is a set of Helm charts.
K8ssandra v1 was created by stacking together several tools that are widely used in the community such as cass-operator, Reaper and Medusa, among others. Helm makes the installation easy, but the lack of an umbrella operator quickly limited our ability to build new features as well as making it harder to create strong and safe code: templates are both hard to write and maintain since there is no compiler, and we rely heavily on integration tests to make sure we’re not breaking anything. It’s also much harder to deal with the lifecycle of objects at runtime as the templates can’t interact directly with the Kubernetes API server. Helm generates the manifests, applies them and then exits. An operator runs permanently, reconciling the desired state of the cluster based on events throughout its lifetime, bringing capabilities to a whole new level. The following diagram shows the maturity model of operators:
We can see that while Helm provides install and upgrade capabilities, Go based operators cover the whole range up to auto-pilot in level 5.
See part 1, part 2 and part 3 of the “Why we decided to build a K8ssandra operator” blog post series for more information.
How do the projects differ operationally?
In K8ssandra v1.x, everything is done through Helm: both deploying the operators and creating custom resources. Creating a K8ssandra v.1x cluster with all the required components will be done through a single Helm command, using a customized values file:
helm install k8ssandra k8ssandra/k8ssandra -f values_file.yaml -n k8ssandra -create-namespace
Modifying a deployment is done through Helm as well using helm upgrade… and creating backups for example is done by installing the appropriate chart:
helm install test k8ssandra/backup --set name=test,cassandraDatacenter.name=dc1
In K8ssandra-operator, Helm can still be used to deploy the operator itself and cass-operator, but it can also be done through Kustomize.
We no longer publish Helm charts for creating custom resources, which should be done through kubectl directly. Helm is a package manager for Kubernetes which does a great job of configuring, installing and/or upgrading packages in the form of Helm charts. It does not do anything with respect to lifecycle management of custom or even native Kubernetes resources. With custom resources in particular, a lot of configuration and even installation, in the form of additional resources, happens post-installation which is where operators come in. We rely on Helm to configure and install K8ssandra-operator. Creating a K8ssandraCluster custom resource however can simply be done with kubectl.
The first step is then to install the operators (see the full procedure here):
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager --create-namespace --set installCRDs=true
helm install k8ssandra-operator k8ssandra/k8ssandra-operator -n k8ssandra-operator --create-namespace
Followed by the creation of a K8ssandraCluster object, using a customized manifest:
kubectl apply -n k8ssandra-operator -f k8c_manifest.yml
All operations from modifying an existing deployment as well as creating backups or triggering restores is done through kubectl apply, using the right manifests.
Is K8ssandra-operator the v2 of K8ssandra then?
We created K8ssandra-operator as the next step for K8ssandra, but as a separate project: the former is a Kubernetes operator, the latter is a Helm chart repository. Versioning it as v2 would not make sense if we have no v1 for its repository. The K8ssandra repository will live on and continue evolving for at least the following reasons:
- We will publish fixes for the v1.5 branch if needed
- We are publishing all of our Helm charts there, including the K8ssandra-operator ones
- K8ssandra might upgrade to v2 if we can come up with a way of upgrading K8ssandra v1.x clusters in place. Currently, v1.x clusters require a (no downtime) migration to upgrade to K8ssandra-operator, so we’re sticking with a distinction between K8ssandra v1.x and K8ssandra-operator… v1.0.
Which project should I choose for new deployments?
K8ssandra-operator is what we’re focusing on for the foreseeable future. K8ssandra v1.5 will only receive critical security updates.
The numerous improvements the operator brings over the v1.x releases include:
- Multi-DC support across a single or multiple k8s clusters, including improved seed management
- Improved Stargate integration
- Improved Reaper integration
- Improved Medusa integration
New deployments should go straight to K8ssandra-operator, installing it through Helm or Kustomize.
Should I upgrade my K8ssandra v1.x cluster to K8ssandra-operator?
Yes, we strongly encourage K8ssandra v1.x users to migrate to K8ssandra-operator. Follow the migration procedure to avoid downtime during the operation.
Let us know what you think of the K8ssandra v1.x vs K8ssandra-operator comparison by joining us on the K8ssandra Discord or K8ssandra Forum today. For exclusive posts on all things data, follow DataStax on Medium.