PgHaMigrations

We've documented our learned best practices for applying schema changes without downtime in the post PostgreSQL at Scale: Database Schema Changes Without Downtime on the Braintree Product and Technology Blog. Many of the approaches we take and choices we've made are explained in much greater depth there than in this README.

Internally we apply those best practices to our Rails applications through this gem which updates ActiveRecord migrations to clearly delineate safe and unsafe DDL as well as provide safe alternatives where possible.

Some projects attempt to hide complexity by having code determine the intent and magically do the right series of operations. But we (and by extension this gem) take the approach that it's better to understand exactly what the database is doing so that (particularly long running) operations are not a surprise during your deploy cycle.

Provided functionality:

• Migrations
• Utilities
• Rake Tasks

Installation

Add this line to your application's Gemfile:

gem 'pg_ha_migrations'

And then execute:

$ bundle

Or install it yourself as:

$ gem install pg_ha_migrations

Usage

Rollback

Because we require that "Rollback strategies do not involve reverting the database schema to its previous version", PgHaMigrations does not support ActiveRecord's automatic migration rollback capability.

Instead we write all of our migrations with only an def up method like:

def up
  safe_add_column :table, :column
end

and never use def change. We believe that this is the only safe approach in production environments. For development environments we iterate by recreating the database from scratch every time we make a change.

Migrations

In general, existing migrations are prefixed with unsafe_ and safer alternatives are provided prefixed with safe_.

Migrations prefixed with unsafe_ will warn when invoked. The API is designed to be explicit yet remain flexible. There may be situations where invoking the unsafe migration is preferred.

Migrations prefixed with safe_ prefer concurrent operations where available, set low lock timeouts where appropriate, and decompose operations into multiple safe steps.

Running multiple DDL statements inside a transaction acquires exclusive locks on all of the modified objects. For that reason, this gem disables DDL transactions by default. You can change this by resetting ActiveRecord::Migration.disable_ddl_transaction in your application.

The following functionality is currently unsupported:

• Rollbacks
• Generators
• schema.rb

safe_create_table

Safely creates a new table.

safe_create_table :table do |t|
  t.type :column
end

safe_create_enum_type

Safely create a new enum without values.

safe_create_enum_type :enum

Or, safely create the enum with values.

safe_create_enum_type :enum, ["value1", "value2"]

safe_add_enum_value

Safely add a new enum value.

safe_add_enum_value :enum, "value"

safe_add_column

Safely add a column.

safe_add_column :table, :column, :type

unsafe_add_column

Unsafely add a column, but do so with a lock that is safely acquired.

unsafe_add_column :table, :column, :type

safe_change_column_default

Safely change the default value for a column.

safe_change_column_default :table, :column, "value"

safe_make_column_nullable

Safely make the column nullable.

safe_make_column_nullable :table, :column

unsafe_make_column_not_nullable

Unsafely make a column not nullable.

unsafe_make_column_not_nullable :table, :column

safe_add_concurrent_index

Add an index concurrently.

safe_add_concurrent_index :table, :column

Add a composite btree index.

safe_add_concurrent_index :table, [:column1, :column2], name: "index_name", using: :btree

safe_remove_concurrent_index

Safely remove an index. Migrations that contain this statement must also include disable_ddl_transaction!.

safe_remove_concurrent_index :table, :name => :index_name

Utilities

safely_acquire_lock_for_table

Safely acquire a lock for a table.

safely_acquire_lock_for_table(:table) do
  ...
end

adjust_lock_timeout

Adjust lock timeout.

adjust_lock_timeout(seconds) do
  ...
end

adjust_statement_timeout

Adjust statement timeout.

adjust_statement_timeout(seconds) do
  ...
end

safe_set_maintenance_work_mem_gb

Set maintenance work mem.

safe_set_maintenance_work_mem_gb 1

Configuration

The gem can be configured in an initializer.

PgHaMigrations.configure do |config|
  # ...
end

Available options

• disable_default_migration_methods: If true, the default implementations of DDL changes in ActiveRecord::Migration and the PostgreSQL adapter will be overridden by implementations that raise a PgHaMigrations::UnsafeMigrationError. Default: true

Rake Tasks

Use this to check for blocking transactions before migrating.

$ bundle exec rake pg_ha_migrations:check_blocking_database_transactions

This rake task expects that you already have a connection open to your database. We suggest that you add another rake task to open the connection and then add that as a prerequisite for pg_ha_migrations:check_blocking_database_transactions.

namespace :db do
  desc "Establish a database connection"
  task :establish_connection do
    ActiveRecord::Base.establish_connection
  end
end

Rake::Task["pg_ha_migrations:check_blocking_database_transactions"].enhance ["db:establish_connection"]

Development

After checking out the repo, run bin/setup to install dependencies and start a postgres docker container. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/braintreeps/pg_ha_migrations. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the PgHaMigrations project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.