I did some test to setup the multi-cluster with k8ssandra helm release 1.3.0.
If stargate is disabled, after following the whole steps in this document, I am able to setup the inter-cluster HA.
If stargate is enabled, I met the following error when execute the “alter keyspace” commands on dc1.
Error:
ALTER KEYSPACE system_auth
WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1': 3, 'dc2': 3};
ConfigurationException: Cannot alter RF while some endpoints
are not in normal state (no range movements):
[/10.151.29.188:7000, /10.151.29.107:7000, /10.151.25.144:7000, /10.151.29.129:7000, /10.151.25.166:7000]
The 10.151.25.x is the node IPs of stargate pods in primary cluster of dc1.
Does that mean stargate still can not support customized rack name? In reaper UI, I can see the the stargate endpoint is not available. Is it caused by the wrong rack name “dc2/default”?
In my k8ssandra.yaml, the are three racks defined as:
cassandra:
# Version of Apache Cassandra to deploy
version: 4.0.0
# Configuration for the /var/lib/cassandra mount point
cassandraLibDirVolume:
# Azure provides this storage class on EKS clusters out of the box. Note we
# are using `managed-premium` here as it has `volumeBindingMode:
# WaitForFirstConsumer` which is important during scheduling.
storageClass: managed-premium
# The recommended live data size is 1 - 1.5 TB. A 2 TB volume supports this
# much data along with room for compactions.
size: 2048Gi
additionalSeeds: []
auth:
superuser:
secret: ${k8ssandra_superuser_secret}
heap:
size: 8G
newGenSize: 8G
resources:
requests:
cpu: 4000m
memory: 32Gi
limits:
cpu: 4000m
memory: 32Gi
# This key defines the logical topology of your cluster. The rack names and
# labels should be updated to reflect the Availability Zones where your GKE
# cluster is deployed.
datacenters:
- name: dc2
size: 3
racks:
- name: eastus2-1
affinityLabels:
agentpool: casspool
topology.kubernetes.io/zone: eastus2-1
- name: eastus2-2
affinityLabels:
agentpool: casspool
topology.kubernetes.io/zone: eastus2-2
- name: eastus2-3
affinityLabels:
agentpool: casspool
topology.kubernetes.io/zone: eastus2-3
reaper:
enabled: true
autoschedule: true
autoschedule_properties:
initialDelayPeriod: PT15S
periodBetweenPolls: PT10M
timeBeforeFirstSchedule: PT5M
scheduleSpreadPeriod: PT6H
excludedKeyspaces: []
excludedClusters: []
stargate:
enabled: true
replicas: 3
heapMB: 512
cpuReqMillicores: 1000
cpuLimMillicores: 1000
medusa:
enabled: true
multiTenant: true
storage: azure_blobs
bucketName: ${backup_blob_name}
storageSecret: ${medusa_bucket_key}
kube-prometheus-stack:
grafana:
adminUser: ${k8ssandra_grafana_username}
adminPassword: ${k8ssandra_grafana_pwd}
