Multi-cluster deployment using customized namespace issue

K8ssandra help
1

I was trying to deploy a multi-cluster deployment, because we only have 2 clusters, I have to deploy one of the data-plane and control-plane on the same cluster, and deploy another data-plane on another cluster.

The namespace I used are “k8ssandra-control-plane” for control plane and “k8ssandra-data-plane” for data plane.

deployment file is

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: k8ss-multi-cluster1
  namespace: k8ssandra-control-plane
spec:
  cassandra:
    serverVersion: "4.0.1"
    storageConfig:
      cassandraDataVolumeClaimSpec:
        storageClassName: vsphere-thin
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
    config:
      jvmOptions:
        heapSize: 512M
    networking:
      hostNetwork: true
    datacenters:
      - metadata:
          name: dc1
        k8sContext: cluster1
        namespace: k8ssandra-data-plane
        size: 3
      - metadata:
          name: dc2
        k8sContext: cluster2
        namespace: k8ssandra-data-plane
        size: 3
  stargate:
     size: 1
     heapSize: 512M
  reaper:
    heapSize: 512Mi
    autoScheduling:
      enabled: false
'''

I got below error when try to install the yaml file, I have already set namespace in yaml file and in kubectl context, why is it still trying to access service in “k8ssandra-operator” namespace?

Thanks!

“Error from server (InternalError): error when creating “k8ss_cluster_deployment.yaml”: Internal error occurred: failed calling webhook “vk8ssandracluster.kb.io”: Post “https://k8ssandra-operator-webhook-service.k8ssandra-operator.svc:443/validate-k8ssandra-io-v1alpha1-k8ssandracluster?timeout=10s”: dial tcp 10.43.121.252:443: connect: connection refused”

2

what is the namespace in which the k8ssandra-operator is installed?

3

Hi @Ajith_Palani

Is it required to have the datacenter in the same namespace?

I want to deploy dc to the other namespace, so here is my config.

metadata:
  namespace: k8ssandra-operator
spec:
  cassandra:
    datacenters:
        metadata:
          name: 'dc1'
          namespace: foo
        size: 1

But I got an error like this, "configmaps is forbidden: User “system:serviceaccount:k8ssandra-operator:k8ssandra-operator” cannot create resource “configmaps” in API group “” in the namespace “foo”.

I noticed that there is a role, but not a cluster role in “k8ssandra-operator” namespace. Maybe it is the cause why it fails?

4

Is it required to have the datacenter in the same namespace?

Only if you deployed the operator with namespace scope. In cluster scope deployments the operator will monitor all namespaces and have a ClusterRole that allows it to create resources in all of them.
Cluster scope is the default deployment mode btw, so you must have explicitly deployed it with namespace scope.

1 Like
5

Hello, @alexander. Thank you for the quick reply. I have checked my operator installation. I did install it with namespace scope (Single-cluster install with kustomize | K8ssandra, Apache Cassandra on Kubernetes).

Now it works with cluster scope. Thanks again.