Some of the projects I'm working on require importing MySQL databases from production into staging or dev environments on daily basis.Usually I'm using systemd timers, Jenkins jobs or Airbnb's Airflow for more complex workflows.
A particular case is where each environment it's a separate Kubernetes cluster inside distinct AWS VPCs. After upgrading to Kubernetes 1.4 I started to use Scheduled Jobs aka Cron Jobs in 1.5, to backup, copy and restore MySQL databases between clusters.
I set up a Github repository containing a simple example for this.
There are 3 Cron Jobs running daily:
- a backup job using
mysqldumpin production - a job that copies the backups to the staging cluster
- a job that restores the databases to staging
At the moment I'm using a nodeSelectorto restrict the nodes where the jobs are running, but tweaking the jobs a little bit you can achieve HA Cron Jobs.