K8ssandra Forum

Does k8ssandra support creation of multi data center Apache Cassandra in single kubernetes namespace

Hi,

I am trying to setup an multi-datacenter Apache Cassandra cluster in Kubernetes in a single namespace , when I run helm install for creating the second datacenter I am getting the below error.

Error: rendered manifests contain a resource that already exists. Unable to continue with install: Secret “testcluster-reaper-jmx” in namespace “test” exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key “meta.helm.sh/release-name” must equal “test-datacenter-2”: current value is “test-datacenter-1”
helm.go:88: [debug] Secret “testcluster-reaper-jmx” in namespace “test” exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key “meta.helm.sh/release-name” must equal “test-datacenter-2”: current value is “test-datacenter-1”
rendered manifests contain a resource that already exists. Unable to continue with install
helm.sh/helm/v3/pkg/action.(*Install).Run
helm.sh/helm/v3/pkg/action/install.go:283
main.runInstall
helm.sh/helm/v3/cmd/helm/install.go:242
main.newInstallCmd.func2
helm.sh/helm/v3/cmd/helm/install.go:120
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.1.3/command.go:852
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.1.3/command.go:960
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.1.3/command.go:897
main.main
helm.sh/helm/v3/cmd/helm/helm.go:87
runtime.main

Is this supported or are there any alternate ways to achieve this ?

1 Like

Thanks for the question! Have look at this post from @jeffreyscarpenterDeploy a multi-datacenter Apache Cassandra cluster in Kubernetes.

It will take you through the process step-by-step. Let us know how you go. Cheers! :beers:

Hi @ErickRamirezAU ,

Thanks for the quick response.

I did try the link which you mentioned and it works perfectly fine, but that uses two namespaces in Kubernetes cluster.
What I was trying to achieve is using single namespace . When I try using a same namespace for both the datacenters it wont work, even if I use a different release name in helm install

You are correct @Preetam - K8ssandra does not currently support creating a Cassandra cluster with mulitple datacenters in the same Kubernetes namespace, but cass-operator does support this. The post referenced above was intended in part as a workaround. Is there a reason that using separate namespaces doesn’t work for you?

Hi @Preetam

K8ssandra does not support installing multiple releases in the same namespace so what you are trying to do won’t work. You can however create a multi-DC cluster across multiple namespaces. You will need to set the additionalSeeds property in the second DC.

additionalSeeds is an array of either pod IPs or hostnames. Since the second DC is in the same k8s cluster, you have DNS available so you can simply use the seed service of the first DC.

Lastly, I want to point out that active work for building a k8ssandra operator in GitHub - k8ssandra/k8ssandra-operator: The Kubernetes operator for K8ssandra. The operator will provide multi-DC as well as multi-cluster support.

Cheers

John

Hi @jeffreyscarpenter ,

Thanks, I did not explore the cass-operator. Will check on that.

One of the reason I was trying to use single namespace was to avoid duplicating Stargate, Prometheus, Grafana, and Reaper and use the existing instances for the second datacenter as well.

You should be able to reuse Prometheus, Grafana, and Stargate across namespaces.

You might have trouble trying to reuse Reaper though due to how it does JMX authentication. If you configure k8ssandra releases to use the same JMX credentials then I think you might be able to reuse Reaper as well.

Cheers

John