Skip to main content
Version: 0.7

Backup and Restore

Backing Up Kubernetes#

Kore has been tested against a backup and restore strategy using Velero; for complete documentation please go to their documetation site. Velero allows you to create scheduled backups of all Kubernetes resources inside the cluster it is installed into, archived into cloud storage (such as an S3 or GCP Cloud Storage bucket).

Example Installation#

info

Note, the following demostrates manual steps for illustrative purposes - in production, this should be automated via a CI pipeline.

Create a S3 bucket to hold the backups#

export AWS_REGION=<aws region>export BUCKET_NAME=<name of s3 bucket>
# Create a bucket to hold the backups$ aws s3api create-bucket \  --bucket ${BUCKET_NAME} \  --acl private \
# Ensure the bucket contents is encrypted with KMS or AES256$ aws s3api put-bucket-encryption \  --bucket ${BUCKET_NAME} \  --server-side-encryption-configuration '{  "Rules": [    {      "ApplyServerSideEncryptionByDefault": {        "SSEAlgorithm": "AES256"      }    }  ]}'

Create a policy capable of reading and writing to the bucket.#

cat <<EOF > policy.json{   "Version":"2012-10-17",   "Statement":[      {         "Effect":"Allow",         "Action": "s3:ListAllMyBuckets",         "Resource":"arn:aws:s3:::*"      },      {         "Effect":"Allow",         "Action":["s3:ListBucket","s3:GetBucketLocation"],         "Resource":"arn:aws:s3:::${BUCKET_NAME}"      },      {         "Effect":"Allow",         "Action":[            "s3:PutObject",            "s3:PutObjectAcl",            "s3:GetObject",            "s3:GetObjectAcl",            "s3:DeleteObject"         ],         "Resource":"arn:aws:s3:::${BUCKET_NAME}/*"      }   ]}EOF
# create the iam policy$ aws iam create-policy --policy-name kore-backup --policy-document file://policy.json

Create an IAM user and attach the policy#

# create a iam group to apply the policy$ aws iam create-group --name kore-backups
# attach the kore backup policy to the group$ aws iam attach-group-policy --name kore-backups --policy-arn <arn-from-before>
# create the user used for backups$ aws iam create-user --user-name kore-backups$ aws iam create-access-key --user-name kore-backups
# attach the user to the group$ aws iam add-user-to-group kore-backups --user-name kore-backups

Create the credentials.aws containing the AWS access keys#

[default]aws_access_key_id = <access_key>aws_secret_access_key = <access_secret>

Install Velero into the Kore management cluster#

$ velero install --provider aws \  --plugins velero/velero-plugin-for-aws:v1.1.0 \  --bucket ${BUCKET_NAME} \  --secret-file ./credentials.aws \  --backup-location-config region=${AWS_REGION} \  --snapshot-location-config region=${AWS_REGION}

The above installs Velero into the velero namespace, creating the various apigroups, backup location and deployment.

c) Create a scheduled backup for Kore cluster

$ velero schedule create kore-backup \  --schedule="0 */1 * * *" \ # The frequency of the backups  --ttl=120h               \ # How long before the backup can be garbage collected

At this point Velero has been installed and performing backups on the Kore management cluster on the hour.

Restoring from backups#

The following is assuming we are restoring the backup to a newly built cluster, though the process should be the same regardless. We need to install Velero once again into the cluster (see here). Once we have the Velero service up and running check the backups via aws s3 ls s3://${BUCKET_NAME}

Ensure the backup location has been marked as readonly for safety measures.
$ kubectl patch backupstoragelocation default \  --namespace velero \  --type merge \  --patch '{"spec":{"accessMode":"ReadOnly"}}'

Once you have the backup name you can restore via:

$ velero create restore --from-backup <name>
# And can watch the restoration status via the$ velero get restore

Ensure the kore namespace has been restored in the cluster and check the status of the pods: kubectl -n kore get po

Last updated on May 24, 2021