Scheduled Backups with Clockwork and Backup Gem

Today I am here to demonstrate a solution I have used to backup a Rails application to Amazon S3.

A popular library to perform backups in Ruby world, is Backup, it has a lot of options (for storage, compression, notification..) and seemed great to me.

Having our backup gem chosen, it's time to pick a gem to deal with scheduled jobs in order we can schedule automatic backups. As Clockwork already was in project's Gemfile, I used it.

Setting up Environment

It isn't recommended include Backup gem in project's main Gemfile, a solution is to create an isolated environment to this gem.

Firstly, let's install the gem:

gem install backup -v '~> 4.0.1'

Now, you will need to create the below structure:

-my_rails_app
 |-vendor
   |-backup
     |-Gemfile

Inside vendor/backup/Gemfile you drop:

source 'https://rubygems.org'
gem 'backup', '~> 4.0.1'

This Gemfile is just to Bundler recognize the folder as an isolated env.

Generating Backup Model

In my case, I generated a simple backup model in order to backup a PostgreSQL database, compress the dump and send to Amazon S3:

bundle exec backup generate:model --trigger my_bkp_name --databases="postgresql" --storages="s3" --compressor="gzip" --config-file="./config.rb"

Fill in your credentials (taken from your .yml files) in the recently created file inside models folder:

Model.new(:my_bkp_name, 'Description for my_bkp_name') do
  ##
  # PostgreSQL [Database]
  #
  database PostgreSQL do |db|
    DB = YAML.load_file("#{DIR}/config/database.yml")
    # try to get RAILS_ENV variable,
    # if it is not set, use 'production'
    RAILS_ENV = ENV.fetch('RAILS_ENV'){'production'}

    # To dump all databases,
    # set `db.name = :all` (or leave blank)
    db.name               = DB[RAILS_ENV]['database']
    db.username           = DB[RAILS_ENV]['username']
    db.password           = DB[RAILS_ENV]['password']
    db.host               = DB[RAILS_ENV]['host']
    db.additional_options = ["-xc", "-E=utf8"]
  end

  ##
  # Amazon Simple Storage Service [Storage]
  #
  store_with S3 do |s3|
    S3 = YAML.load_file("#{DIR}/config/s3.yml")

    # AWS Credentials
    s3.access_key_id     = S3['access_key_id']
    s3.secret_access_key = S3['secret_access_key']

    s3.region            = "us-east-1"
    s3.bucket            = "bucket-name"
    s3.path              = "path/to/backups"
    # max 7 files
    s3.keep              = 7
  end

  ##
  # Gzip [Compressor]
  #
  compress_with Gzip
end

We will need set our DIR variable in config.rb file also:

DIR = File.expand_path('../../../', __FILE__)

And after, run it:

bundle exec backup perform --trigger my_bkp_name --config-file="./config.rb"

It should works, access your bucket and check out your dump..

Scheduling Backups

Finally, I scheduled backups in app/clock.rb file:

if Rails.env.production?
  every(1.day, 'database_backup', at: '03:30') do
    dir = File.expand_path('../../', __FILE__)
    Bundler.with_clean_env do
      system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
    end
  end
end

Here, we run the perform command inside our isolated env using with_clean_env, that ensure we are not in application's env.

You can also create a rake task like this in lib/tasks/db.rake:

task :backup do
  dir = File.expand_path('../../../', __FILE__)
  Bundler.with_clean_env do
    system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
  end
end

However, you will have to create a method inside app/clock.rb to invoke the rake:

def execute_rake(file,task)
  require 'rake'
  rake = Rake::Application.new
  Rake.application = rake
  Rake::Task.define_task(:environment)
  load "#{Rails.root}/lib/tasks/#{file}"
  rake[task].invoke
end

if Rails.env.production?
  every(1.day, 'database_backup', at: '03:30') { execute_rake "db.rake", 'db:backup' }
end

After all, you must have your automatic backup scheduled successfully.

See you.

Written on February 11, 2014

Share: