Archive for August 2008

Our Git deployment workflow

At weplay, we recently switched to pure git version control from git-svn. Now that we've had a couple weeks for the dust to settle, I'd like to share our workflow for managing deployments to our staging and production clusters.

We started by outlining the goals of our system:

  • All code that's pushed to our staging and production environments must be in GitHub. Nothing goes straight from a local repository to our servers.

  • Any developer can deploy our most recent work to staging.

  • Any developer can deploy the code on staging into production. We (try to) avoid deploying anything to production that hasn't been pushed to staging first.

  • Any developer can see a diff between "What is deployed" and "What I'm about to deploy."

  • Any developer can branch from the production codebase for time sensitive tweaks and fixes. These need to be staged before they're deployed to production too.

  • That functionality is available in an automated, safe, easy to use form. (Hint: rake)

We ruled out having "production" and "staging" tags because updating tags across a tree of remotes doesn't seem to work smoothly. We also ruled out doing git merges from master into our staging branch. Our staging server/codebase jumps around from the latest changes to the production code (up to a week old) based on what we need to test, so we really just want to replace it.

We settled on using remote git branches for production and staging on origin and hard resetting them to other branches to simulate a copy. We treat these branches more like tags and never commit to them directly. This has worked great so far.

Here's what our rake tasks look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
class GitCommands

  def diff_staging
    `git fetch`
    puts `git diff origin/production origin/staging`
  end
  
  def tag_staging(branch_name)
    verify_working_directory_clean
    
    `git fetch`
    `git branch -f staging origin/staging`
    `git checkout staging`
    `git reset --hard origin/#{branch_name}`
    `git push -f origin staging`
    `git checkout master`
    `git branch -D staging`
  end
  
  def tag_production
    verify_working_directory_clean
    
    `git fetch`
    `git branch -f production origin/production`
    `git checkout production`
    `git reset --hard origin/staging`
    `git push -f origin production`
    `git checkout master`
    `git branch -D production`
  end

  def branch_production(branch_name)
    verify_working_directory_clean
    
    `git fetch`
    `git branch -f production origin/production`
    `git checkout production`
    `git branch #{branch_name}`
    `git checkout #{branch_name}`
    `git push origin #{branch_name}`
  end
  
protected

  def verify_working_directory_clean
    return if `git status` =~ /working directory clean/
    raise "Must have clean working directory"
  end
end


namespace :tag do
  desc <<-DESC
    Update the staging branch to prepare for a staging deploy.
    Defaults to master. Optionally specify a BRANCH=name
  DESC
  
  task :staging do
    branch_name = ENV['BRANCH'] || "master"
    GitCommands.new.tag_staging(branch_name)
  end

  desc "Update the remove production branch to prepare for a release"
  task :production => ['diff:staging'] do
    GitCommands.new.tag_production
  end
end

namespace :diff do
  desc "Show the differences between the staging branch and the production branch"
  task :staging do
    GitCommands.new.diff_staging
  end
end

namespace :branch do
  desc "Branch from production for tweaks or bug fixs. Specify BRANCH=name"
  task :production do
    branch_name = ENV['BRANCH']
    raise "You must specify a branch name using BRANCH=name" unless branch_name
    GitCommands.new.branch_production
  end
end

namespace :deploy do
  desc "Tag and deploy staging"
  task :staging => "tag:staging" do
    `cap staging deploy:long`
  end
end

The last one (rake deploy:staging) simply wraps up the common task of tagging our latest code to be pushed to staging and initiating a staging deploy.

Note: To use most of these commands, your local working directory must be clean. If we have outstanding changes in our tree when we need to run them, we use git-stash to temporarily move them out of the way.

Thanks to Scott Chacon for helping us work this out. Be sure to check out his GitCasts and Git Internals PDF.