Scheduled Backups with Clockwork and Backup Gem

Today I am here to demonstrate a solution I have used to backup a Rails application to Amazon S3.

A popular library to perform backups in Ruby world, is Backup, it has a lot of options (for storage, compression, notification..) and seemed great to me.

Having our backup gem chosen, it's time to pick a gem to deal with scheduled jobs in order we can schedule automatic backups. As Clockwork already was in project's Gemfile, I used it.

Setting up Environment

It isn't recommended include Backup gem in project's main Gemfile, a solution is to create an isolated environment to this gem.

Firstly, let's install the gem:

gem install backup -v '~> 4.0.1'

Now, you will need to create the below structure:

-my_rails_app
 |-vendor
   |-backup
     |-Gemfile

Inside vendor/backup/Gemfile you drop:

source 'https://rubygems.org'
gem 'backup', '~> 4.0.1'

This Gemfile is just to Bundler recognize the folder as an isolated env.

Generating Backup Model

In my case, I generated a simple backup model in order to backup a PostgreSQL database, compress the dump and send to Amazon S3:

bundle exec backup generate:model --trigger my_bkp_name --databases="postgresql" --storages="s3" --compressor="gzip" --config-file="./config.rb"

Fill in your credentials (taken from your .yml files) in the recently created file inside models folder:

Model.new(:my_bkp_name, 'Description for my_bkp_name') do
##
# PostgreSQL [Database]
#
database PostgreSQL do |db|
DB = YAML.load_file("#{DIR}/config/database.yml")
# try to get RAILS_ENV variable,
# if it is not set, use 'production'
RAILS_ENV = ENV.fetch('RAILS_ENV'){'production'}

# To dump all databases,
# set `db.name = :all` (or leave blank)
db.name = DB[RAILS_ENV]['database']
db.username = DB[RAILS_ENV]['username']
db.password = DB[RAILS_ENV]['password']
db.host = DB[RAILS_ENV]['host']
db.additional_options = ["-xc", "-E=utf8"]
end

##
# Amazon Simple Storage Service [Storage]
#
store_with S3 do |s3|
S3 = YAML.load_file("#{DIR}/config/s3.yml")

# AWS Credentials
s3.access_key_id = S3['access_key_id']
s3.secret_access_key = S3['secret_access_key']

s3.region = "us-east-1"
s3.bucket = "bucket-name"
s3.path = "path/to/backups"
# max 7 files
s3.keep = 7
end

##
# Gzip [Compressor]
#
compress_with Gzip
end

We will need set our DIR variable in config.rb file also:

DIR = File.expand_path('../../../', __FILE__)

And after, run it:

bundle exec backup perform --trigger my_bkp_name --config-file="./config.rb"

It should works, access your bucket and check out your dump..

Scheduling Backups

Finally, I scheduled backups in app/clock.rb file:

if Rails.env.production?
every(1.day, 'database_backup', at: '03:30') do
dir = File.expand_path('../../', __FILE__)
Bundler.with_clean_env do
system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
end
end
end

Here, we run the perform command inside our isolated env using with_clean_env, that ensure we are not in application's env.

You can also create a rake task like this in lib/tasks/db.rake:

task :backup do
dir = File.expand_path('../../../', __FILE__)
Bundler.with_clean_env do
system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
end
end

However, you will have to create a method inside app/clock.rb to invoke the rake:

def execute_rake(file,task)
require 'rake'
rake = Rake::Application.new
Rake.application = rake
Rake::Task.define_task(:environment)
load "#{Rails.root}/lib/tasks/#{file}"
rake[task].invoke
end

if Rails.env.production?
every(1.day, 'database_backup', at: '03:30') { execute_rake "db.rake", 'db:backup' }
end

After all, you must have your automatic backup scheduled successfully.

See you.

Written on February 11, 2014

Share: