Istio-side car injection is not working in k8ssandra

K8ssandra help
1

Hi Team,
k8ssandra is working fine without enabling side car injection but after enabling istio side car injection, cass operator pod is running with two containers(operator container and proxy container) but cassandra-dc1-default-sts-0 pod is running with one container only(cassandra container is failing, proxy container is running).

cassandra container is failing with below logs.

"
INFO [nioEventLoopGroup-2-1] 2021-10-14 06:14:00,480 Cli.java:617 - address=/127.0.0.6:55325 url=/api/v0/probes/liveness status=200 OK INFO [epollEventLoopGroup-673-1] 2021-10-14 06:14:04,735 Clock.java:47 - Using native clock for microsecond precision
WARN [epollEventLoopGroup-673-2] 2021-10-14 06:14:04,736 AbstractBootstrap.java:452 - Unknown channel option ‘TCP_NODELAY’ for channel ‘[id: 0xee3ca913]’
WARN [epollEventLoopGroup-673-2] 2021-10-14 06:14:04,737 Loggers.java:39 - [s668] Error connecting to Node(endPoint=/tmp/cassandra.sock, hostId=null, hashCode=6e83d65f), trying next node (FileNotFoundException: null)
INFO [nioEventLoopGroup-2-2] 2021-10-14 06:14:04,739 Cli.java:617 - address=/127.0.0.6:36323 url=/api/v0/probes/readiness status=500 Internal Server Error
INFO [epollEventLoopGroup-674-1] 2021-10-14 06:14:14,707 Clock.java:47 - Using native clock for microsecond precision
WARN [epollEventLoopGroup-674-2] 2021-10-14 06:14:14,709 AbstractBootstrap.java:452 - Unknown channel option ‘TCP_NODELAY’ for channel ‘[id: 0x4226e4cc]’
WARN [epollEventLoopGroup-674-2] 2021-10-14 06:14:14,710 Loggers.java:39 - [s669] Error connecting to Node(endPoint=/tmp/cassandra.sock, hostId=null, hashCode=702dc8a), trying next node (FileNotFoundException: null)
"

Can anyone support me in resolving this issue

2

Hello, @Shubham_Srivastava . Sorry for the delay in replying to you. I was aware that you posted a comment in another topic but I got sidetracked. I’ll bring it to the attention of the engineers. Cheers! :beers:

3

Thanks @ErickRamirezAU

4

@ErickRamirezAU Any update on this issue

5

Shubham_Srivastava are you trying to deploy the k8ssandra helm charts with Istio? How many nodes are you trying to stand up, and how long have you waited for the cluster to come up?

I can point you to some resources:

  • An example of deploying cass-operator with Istio is provided here, I have not tested this and have only skim read it.
  • When reading through that, bear in mind that Istio’s statefulset behaviour changed in 1.10 and make the necessary adjustments.

I’d potentially start with a single node cass-operator deployment, figure out how to make it work, then look at what you need to do to shim the k8ssandra helm services together with Istio.

Also, can you please share your manifests or some reproducible example? We’ll need versions for cass-operator, management API and Istio, manifests will be the easiest way to ascertain these. I’ve been meaning to write a blog post on this, so I’m happy to provide you some assistance.