Some of the projects I'm working on require importing MySQL databases from production into staging or dev environments on daily basis.Usually I'm using systemd timers, Jenkins jobs or Airbnb's Airflow for more complex workflows.
A particular case is where each environment it's a separate Kubernetes cluster inside distinct AWS VPCs. After upgrading to Kubernetes 1.4 I started to use Scheduled Jobs
aka Cron Jobs in 1.5, to backup, copy and restore MySQL databases between clusters.
I set up a Github repository containing a simple example for this.
There are 3 Cron Jobs
running daily:
- a backup job using
mysqldump
in production - a job that copies the backups to the staging cluster
- a job that restores the databases to staging
At the moment I'm using a nodeSelector
to restrict the nodes where the jobs are running, but tweaking the jobs a little bit you can achieve HA Cron Jobs
.