I am trying to setup an multi-datacenter Apache Cassandra cluster in Kubernetes in a single namespace , when I run helm install for creating the second datacenter I am getting the below error.
Error: rendered manifests contain a resource that already exists. Unable to continue with install: Secret “testcluster-reaper-jmx” in namespace “test” exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key “meta.helm.sh/release-name” must equal “test-datacenter-2”: current value is “test-datacenter-1”
helm.go:88: [debug] Secret “testcluster-reaper-jmx” in namespace “test” exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key “meta.helm.sh/release-name” must equal “test-datacenter-2”: current value is “test-datacenter-1”
rendered manifests contain a resource that already exists. Unable to continue with install
helm.sh/helm/v3/pkg/action.(*Install).Run
helm.sh/helm/v3/pkg/action/install.go:283
main.runInstall
helm.sh/helm/v3/cmd/helm/install.go:242
main.newInstallCmd.func2
helm.sh/helm/v3/cmd/helm/install.go:120 github.com/spf13/cobra.(*Command).execute github.com/spf13/cobra@v1.1.3/command.go:852 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/cobra@v1.1.3/command.go:960 github.com/spf13/cobra.(*Command).Execute github.com/spf13/cobra@v1.1.3/command.go:897
main.main
helm.sh/helm/v3/cmd/helm/helm.go:87
runtime.main
Is this supported or are there any alternate ways to achieve this ?
I did try the link which you mentioned and it works perfectly fine, but that uses two namespaces in Kubernetes cluster.
What I was trying to achieve is using single namespace . When I try using a same namespace for both the datacenters it wont work, even if I use a different release name in helm install
You are correct @Preetam - K8ssandra does not currently support creating a Cassandra cluster with mulitple datacenters in the same Kubernetes namespace, but cass-operator does support this. The post referenced above was intended in part as a workaround. Is there a reason that using separate namespaces doesn’t work for you?
K8ssandra does not support installing multiple releases in the same namespace so what you are trying to do won’t work. You can however create a multi-DC cluster across multiple namespaces. You will need to set the additionalSeeds property in the second DC.
additionalSeeds is an array of either pod IPs or hostnames. Since the second DC is in the same k8s cluster, you have DNS available so you can simply use the seed service of the first DC.
Thanks, I did not explore the cass-operator. Will check on that.
One of the reason I was trying to use single namespace was to avoid duplicating Stargate, Prometheus, Grafana, and Reaper and use the existing instances for the second datacenter as well.
You should be able to reuse Prometheus, Grafana, and Stargate across namespaces.
You might have trouble trying to reuse Reaper though due to how it does JMX authentication. If you configure k8ssandra releases to use the same JMX credentials then I think you might be able to reuse Reaper as well.
