Git: migration and the first 5½ years

Mar 30, 2020 | By Ben Cohen

Two decades later…

Razor served us well for many years but it lacks features offered by newer version control systems so the development team were increasingly unhappy with it. In particular, branching in Razor is intended for temporary concurrent development and it isn’t very practical for longer-lived branches affecting multiple files.

We decided to switch away from Razor in Summer 2014. The replacement would have to retain integration between a version control system and bug tracker, and let us choose bug fixes and backport them to release branches easily.

We chose Git, as the emerging leading version control system, and Bugzilla, a mature bug tracking system, and finally changed over on 3 October 2014.

Migration to Git

Razor uses RCS for underlying storage on Linux. I copied the RCS files out of the Razor repository and made a few adjustments:

Set the RCS revision titles to the issue id and title because people had generally not given check-in titles for the revisions.
Added a tag for each released version to the appropriate revision in each RCS file.
Checked binary files such as icons into RCS manually because Razor stores these separately.

I converted the RCS files via CVS to a git repo using cvs2git. Files in RCS and CVS are versioned independently so a nice feature of cvs2git is that it matches up RCS revisions made at about the same time across multiple files into a single git commit, and also migrates the tags added above.

I then added branches for each release version and manually committed changes for the patch releases that were maintained manually outside Razor.

Integrating Bugzilla and friends

Git and Bugzilla are not actually integrated so we had to do some work to achieve what we wanted.

Bugzilla: bug tracking

A colleague extracted the issues details from the Razor repository and imported them into Bugzilla. She also wrote a script to create a patch build for a release branch by applying any bugs that we want to backport indicated by a custom Bugzilla bug field called “Take”.

We added various git hooks to provide some convenience features for the user and validation when a commit is pushed to the server:

Ensure that the git commit has a bugzilla bug id in the commit message title and that the bug is open and assigned to the correct user (pre-receive hook).
If the commit title is “12345: *” replace the star with the title from bug 12345 (commit-msg hook).
If the commit title starts with “AUTO” then create a bug with the title that follows and put the bug number in the commit title (commit-msg hook).
Change the bug status to ready for testing if the commit body contains “BZ:DONE” (post-receive hook).
Add the git commit id and message to its Bugzilla bug if the push succeeds (post-receive hook).

CGit: git repository browser

CGit is a lightweight web frontend for git repositories, so we installed it on our Git and Bugzilla server. We made bug ids and commit ids clickable links where they appear in commit messages in CGit and Bugzilla bugs (using a commit-filter script and the bug_format_comment hook). These clickable links make cross-referencing between bugs, commits and source code in a web browser very easy.

Jenkins: continuous integration

We use Jenkins to do builds and some minimal testing automatically when changes are pushed to certain repos. If a build or test fails then the details are emailed to the person who made the last change (who is usually, but not necessarily the culprit!)

Gitolite: finer-grained access control for the git server

We originally used the basic git with ssh authentication on the server but later we wanted anyone to be able to add new repos. Gitolite makes this easy using its “wild repos“. We now have over twenty other repos for documentation, useful scripts, test engines and so on.

Teething problems and annoyances

We learnt a few things along the way:

Make sure that everyone’s ssh key is on the git server and that they remember to forward the ssh agent, to avoid the dreaded “Permission denied (publickey); Could not read from remote repository”.
Recommend that everyone sets the git config option “merge.conflictstyle diff3“, which is unfortunately not the default, to show the common parent when doing manual merges.
A colleague accidentally reverted some changes after “git pull --rebase“, because Vim running in a terminal (unlike the Vim GUI and Emacs) didn’t immediately warn him that the file had changed and he didn’t notice the warning later when saving it. He mitigated this with a vim setting.
You can’t amend comments in Bugzilla, which is annoying if you make a typo in a test case or explanation because you can’t fix it and a later correction might not be noticed. (The forthcoming version 6 will finally make this possible.)

Conclusion

Git gives us more choice in how we work and has enabled us to do things we couldn’t have done before, particularly with its branching and merging features. We have developed several features using long-term branches that we would have found difficult previously.

The server we use for git and bugzilla was originally hosted on-site as a VMWare client but we moved it into AWS when we did a cloud migration in 2017.

Kognitio has also published some repos on GitHub including my collection of tips for Git.

Blogs