Git for Game Development

I’ve used a few different version control systems for film and game production over the years: CVS, Perforce, even a few in-house tools. They always struck me as clunky-but-necessary evils–invaluable for tracking work on a big project, but also an ongoing source of confusion, bottlenecks, and production inflexibility.

Git first popped up on my radar about six months ago. It looked amazingly powerful and flexible. The Massively Fun developers were already using it for code development when I joined the team. I was eager to see if we could make it work for art production. This post tells the story of what happened.

Background: What is Git?

Git is a distributed version control system. Most everyone in game development will already be familiar with at least one other version control system like Subversion or Perforce. Like these other systems, Git lets us keep track of the many different versions of all the files that go into a project. But Subversion and Perforce are centralized systems–all files are checked into or out of a repository that lives on a central server. Git is different. It doesn’t depend on a central server. Every user gets a full-fledged copy of the whole repository, with all its history and revisions.

This design has a number of important implications, most of which are positive. But what makes git stand out most in my eyes is how it makes branch and merge operations almost trivial. You can create a branch to work on some experimental idea, another one to fix some unrelated problem, still another for a new feature. You can move quickly between these different branches, and you can merge them back together as needed. It’s an incredibly useful feature.

Stumbling Blocks

Despite its many strengths, off-the-shelf git has two obvious flaws for art production:

• It’s not artist-friendly. Git’s default interface is the command line. There are a lot of commands and options to learn, along with underlying concepts that differ sufficiently from other version control systems to invite confusion. Documentation is plentiful but often difficult to understand. It all adds up to a package that’s not very approachable and somewhat intimidating.
• It’s not built to manage the kind of big binary files that artists make. The fact that each local repository contains all history and revisions means that a lot of rapidly changing artwork can eat a lot of disk space quickly.

We came up with solutions for both these problems.

Interface Matters

There are several good graphic interfaces to git. Artists still need to understand basic git concepts, but a well-designed front end makes it considerably easier to pick up and use. We found Atlassian’s SourceTree worked great in our Mac-based environment: it’s powerful, intuitive, and free (for now)–an important consideration for a scrappy startup. The SourceTree team also responded promptly (< 12 hours) to our tweeted support queries.

A Slick Git Extension

Git-media addresses git’s shortcomings with respect to big binary files. It’s a git extension that lets you store files outside of git. Git still tracks the file versions; it just stores the file contents somewhere outside the repository. We planned to store our big binaries in Amazon’s S3 cloud.

Not So Fast

Using SourceTree with Git-media does present a gotcha, however: transferring files between our repositories and Amazon S3 requires us to execute a shell command; i.e.,

    $ git media sync

A workflow that obliges artists to leave the relative comfort of a graphic application to type shell commands is not artist-friendly. SourceTree does give you a way to execute this command as a custom action, but that wasn’t good enough. Artists would still have to remember to do it at appropriate times. Sooner or later they’d forget and somebody would waste valuable time figuring out what went wrong and fixing it.

How could we get SourceTree and git-media to play nicely together?

Git Wrap

We wrote a generic wrapper script that optionally executes some other commands before and/or after the git command itself. We set it up to automatically call ‘git media sync’ before or after commands that interact with a remote repository, as appropriate. We configured SourceTree to invoke the wrapper instead of out-of-the-box git. Source code for the wrapper is available here on Github.

The Adventure Continues

I’d like to tell you that was the end of the story, that SourceTree and git-media worked together seamlessly and we all lived happily ever after. They did in fact work, up to a point. Alas, it wasn’t good enough.

SourceTree normally does a good job of previewing image files. This is crucial for artists when they’re committing or merging changes to such files. But using git-media broke SourceTree’s ability to preview these files because it meant we were no longer storing the images in git; we were only storing version information about the images. We couldn’t find an acceptable way around this; it was a deal breaker.

As of this writing we’re making another attempt with git’s submodule feature. Storing our art assets in a submodule is another way to minimize the size of the main project repository. SourceTree understands and supports submodules fairly well, and its ability to preview images remains intact. Submodules do have some idiosyncracies, but nothing that looks too disruptive to our artist work flow.

Watch this space for updates; we’ll let you know how things turn out.