Scheduled Backups with Clockwork and Backup Gem
Today I am here to demonstrate a solution I have used to backup a Rails application to Amazon S3.
A popular library to perform backups in Ruby world, is Backup, it has a lot of options (for storage, compression, notification..) and seemed great to me.
Having our backup gem chosen, it's time to pick a gem to deal with scheduled jobs in order we can schedule automatic backups. As Clockwork already was in project's Gemfile
, I used it.
Setting up Environment
It isn't recommended include Backup gem in project's main Gemfile
, a solution is to create an isolated environment to this gem.
Firstly, let's install the gem:
gem install backup -v '~> 4.0.1'
Now, you will need to create the below structure:
-my_rails_app |-vendor |-backup |-Gemfile
Inside vendor/backup/Gemfile
you drop:
source 'https://rubygems.org'
gem 'backup', '~> 4.0.1'
This Gemfile
is just to Bundler recognize the folder as an isolated env.
Generating Backup Model
In my case, I generated a simple backup model in order to backup a PostgreSQL database, compress the dump and send to Amazon S3:
bundle exec backup generate:model --trigger my_bkp_name --databases="postgresql" --storages="s3" --compressor="gzip" --config-file="./config.rb"
Fill in your credentials (taken from your .yml
files) in the recently created file inside models
folder:
Model.new(:my_bkp_name, 'Description for my_bkp_name') do
##
# PostgreSQL [Database]
#
database PostgreSQL do |db|
DB = YAML.load_file("#{DIR}/config/database.yml")
# try to get RAILS_ENV variable,
# if it is not set, use 'production'
RAILS_ENV = ENV.fetch('RAILS_ENV'){'production'}
# To dump all databases,
# set `db.name = :all` (or leave blank)
db.name = DB[RAILS_ENV]['database']
db.username = DB[RAILS_ENV]['username']
db.password = DB[RAILS_ENV]['password']
db.host = DB[RAILS_ENV]['host']
db.additional_options = ["-xc", "-E=utf8"]
end
##
# Amazon Simple Storage Service [Storage]
#
store_with S3 do |s3|
S3 = YAML.load_file("#{DIR}/config/s3.yml")
# AWS Credentials
s3.access_key_id = S3['access_key_id']
s3.secret_access_key = S3['secret_access_key']
s3.region = "us-east-1"
s3.bucket = "bucket-name"
s3.path = "path/to/backups"
# max 7 files
s3.keep = 7
end
##
# Gzip [Compressor]
#
compress_with Gzip
end
We will need set our DIR
variable in config.rb
file also:
DIR = File.expand_path('../../../', __FILE__)
And after, run it:
bundle exec backup perform --trigger my_bkp_name --config-file="./config.rb"
It should works, access your bucket and check out your dump..
Scheduling Backups
Finally, I scheduled backups in app/clock.rb
file:
if Rails.env.production?
every(1.day, 'database_backup', at: '03:30') do
dir = File.expand_path('../../', __FILE__)
Bundler.with_clean_env do
system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
end
end
end
Here, we run the perform
command inside our isolated env using with_clean_env
, that ensure we are not in application's env.
You can also create a rake task like this in lib/tasks/db.rake
:
task :backup do
dir = File.expand_path('../../../', __FILE__)
Bundler.with_clean_env do
system "cd #{dir}/vendor/backup/; bundle exec backup perform --trigger my_bkp_name --config-file ./config.rb"
end
end
However, you will have to create a method inside app/clock.rb
to invoke the rake:
def execute_rake(file,task)
require 'rake'
rake = Rake::Application.new
Rake.application = rake
Rake::Task.define_task(:environment)
load "#{Rails.root}/lib/tasks/#{file}"
rake[task].invoke
end
if Rails.env.production?
every(1.day, 'database_backup', at: '03:30') { execute_rake "db.rake", 'db:backup' }
end
After all, you must have your automatic backup scheduled successfully.
See you.