MedusaRestoreJob can't find backup when prefix is used

K8ssandra help
1

Hi,

My restore jobs can’t find the backups I made. I suspect it’s because of the prefix I’ve given.

I’ve have a backup schedule job making backups to this location:
cassandra/fdon-dev/db-datacenter1-default-sts-0/medusa-backup-schedule-1698197400

Medusa:
Storage Properties:
Bucket Name: cassandra
Concurrent Transfers: 1
Host: redacted
Max Backup Age: 0
Max Backup Count: 7
Multi Part Upload Threshold: 104857600
Port: 9000
Prefix: fdon-dev
Secure: false
Storage Provider: s3_compatible
Storage Secret Ref:
Name: medusa-bucket-key
Transfer Max Bandwidth: 50MB/s

My restore job is shown here:

apiVersion: medusa.k8ssandra.io/v1alpha1
kind: MedusaRestoreJob
metadata:
  name: restore-backup1
  namespace: cassandra
spec:
  cassandraDatacenter: datacenter1
  backup: medusa-backup-schedule-1698197400

Here is the error shown in the operator stating that it can’t find the backup:

2023-10-25T12:36:24.564Z	ERROR	Failed to get MedusaBackup	{"controller": "medusarestorejob", "controllerGroup": "medusa.k8ssandra.io", "controllerKind": "MedusaRestoreJob", "MedusaRestoreJob": {"name":"restore-backup1","namespace":"cassandra"}, "namespace": "cassandra", "name": "restore-backup1", "reconcileID": "f9046b07-5d6f-4639-b9cb-246110301227", "medusarestorejob": "cassandra/restore-backup1", "MedusaBackup": {"namespace": "cassandra", "name": "medusa-backup-schedule-1698197400"}, "error": "MedusaBackup.medusa.k8ssandra.io \"medusa-backup-schedule-1698197400\" not found"}
github.com/k8ssandra/k8ssandra-operator/pkg/medusa.(*factory).NewMedusaRestoreRequest
	/workspace/pkg/medusa/requests.go:73
github.com/k8ssandra/k8ssandra-operator/controllers/medusa.(*MedusaRestoreJobReconciler).Reconcile
	/workspace/controllers/medusa/medusarestorejob_controller.go:72
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:122
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:323
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:235
2023-10-25T12:36:24.564Z	ERROR	Reconciler error	{"controller": "medusarestorejob", "controllerGroup": "medusa.k8ssandra.io", "controllerKind": "MedusaRestoreJob", "MedusaRestoreJob": {"name":"restore-backup1","namespace":"cassandra"}, "namespace": "cassandra", "name": "restore-backup1", "reconcileID": "f9046b07-5d6f-4639-b9cb-246110301227", "error": "MedusaBackup.medusa.k8ssandra.io \"medusa-backup-schedule-1698197400\" not found"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:235

I suspect it’s looking for the backup under
cassandra/db-datacenter1-default-sts-0/medusa-backup-schedule-1698197400
and not under
cassandra/fdon-dev/db-datacenter1-default-sts-0/medusa-backup-schedule-1698197400
where it is in reality.

Can someone help?

Thanks
Vladimir

2

What it’s not finding currently is the MedusaBackup object corresponding to the backup you’re restoring. This custom resource has to exist in the same namespace as the MedusaRestoreJob you’re creating.
If you’re trying to restore the backup to a different cluster or in a different namespace, you’ll need to run a sync MedusaTask first to recreate the MedusaBackup objects from what exists in the bucket.
Check the docs for more info.

1 Like