.py in the sky

How to conduct a full code review on GitHub

2013-04-10T13:38:00+02:00

Why we might want to do it

I think it’s fair to say I’m addicted to using GitHub. I’ve used it so much in the last couple of years that I don’t understand/remember how we got any serious collaborative coding done before. In particular, the ability to comment on code line-by-line, having conversations, updating the pull requests, and merging them with a single click is in my mind so much more rewarding and productive than having to comment on a patch in an email discussion.

However, I occasionally want to do a full review of a package that someone else has written, and comment on various parts of the code. While it is possible to leave line-by-line comments on a commit-by-commit basis, GitHub does not provide an official way to review the latest full version of a file or package.

There are a few ways to conduct a full code review that I can think of:

Browse through the files, on GitHub or locally, and open new issues for anything we would like to comment on, copying and pasting the relevant code. Not ideal if we want to comment on 20-30 chunks of code or more!
Browse through the files on GitHub, and if we see a line we want to comment on, we can go to the Blame tab, and then find the last commit that modified that line, and comment on it. The issue with this is that we might want to comment on a chunk of code that was the result of several commits in which case this method breaks down.
Leverage the pull request interface, with a little git-fu, to conduct a proper full code review. This is in my opinion the best approach, and in this post, I describe one way to do this. There may be more elegant ways, so please let me know if you have any suggestions!

How to do it

Ideally, one could simply create an empty branch on GitHub, then set up a pull request from master (or whatever branch you want to review) onto the empty branch. However, as far as I can tell, you can’t create completely empty branches on GitHub - instead, we need our empty branch to have at least one commit, which needs to match the first commit of the branch we want to review (otherwise GitHub will complain that there is no common history).

So how we proceed depends on whether the first commit contains code that needs to be reviewed, or if it is unimportant (for example, a lot of repositories start with the addition of an empty README file).

If the first commit is unimportant…

… then the situation is fairly easy. You first need to find out the commit hash for the first commit in the repository, which you can do with:

$ git rev-list –all | tail -1
ec2287e5837386c54fbd082021530aa18c0dcf18

In the example above the hash is ec2287e5837386c54fbd082021530aa18c0dcf18, but this will be different for you. Now, create an empty branch containing only that commit:

$ git branch empty ec2287e5837386c54fbd082021530aa18c0dcf18

This will create, but not switch to, the empty branch. Next push your empty branch to GitHub:

$ git push origin empty

Go to your repository on GitHub and click on the ‘Pull Request’ button at the top right of the window:

Then set it up so that you are pulling the changes from master into empty, as follows:

You can now enter a title and message for the pull request, and invite other people to comment on the code. If you make changes to master, you can simply push the changes to GitHub as usual:

$ git push origin master

which should cause the new commits to appear in the pull request. Once the review is complete, you can just close the pull request (without merging), and keep the empty branch for future reviews (or delete it).

If the first commit is important…

… this makes things a little more complicated. The approach we’ll take here is to create two new branches - review, containing the code to review, and empty, containing no files - both of which contain a common and empty first commit (which we will add). In this way, the two branches have a common history, even though the empty branch has no files. We can set then set up a pull request from review to empty.

Important disclaimer: make sure that you make a backup of your repository, and that there are no unsaved changes! If you follow these instructions, any files that are not already in the repository will get deleted, as well as any uncommitted changes! In fact, it might be safest to do this in a clean clone of your repository, so that if anything goes wrong, you haven’t affected your usual work repository.

With that disclaimer in mind, go to the repository you want to do a review for, and then create an empty branch that we will call review

$ git checkout –orphan review

This branch has no history, but the files should still be there and would be added to the branch if we were to commit. However, you don’t want to do this, so remove all the files in the repository in the current branch by first unstaging all the files:

$ git rm -r –cached *

then removing them all:

$ git clean -fxd

Note that any file that was not previously part of the repository will be deleted for good, not just from this branch!

You should now have a nice and empty branch:

$ git log
fatal: bad default revision ‘HEAD’

$ git status
# On branch review
#
# Initial commit
#
nothing to commit (create/copy files and use "git add" to track)

You are now ready to set up the review. You should first add a dummy commit that contains no files:

$ git commit –allow-empty -m "Start of the review"

Then create a new branch called empty that will contain only this commit:

$ git branch empty

This will create a branch with the same empty commit, but will keep on the review branch. You can now merge in the changes from the branch we want to actually review, say master, into review:

$ git merge master

You will be asked to provide a merge commit message, and you can just leave the default. Next push your review and empty branches to GitHub:

$ git push origin review
$ git push origin empty

Go to your repository on GitHub and click on the ‘Pull Request’ button at the top right of the window:

Then set it up so that you are pulling the changes from review into empty, as follows:

You can now enter a title and message for the pull request, and invite other people to comment on the code. Make sure that you switch back to your master (or other) branch to implement the changes, and if you then want to update the review pull request, you can switch back to review and merge the latest changes from master:

$ git checkout review
$ git merge master
$ git push origin review

which should cause the new commits to appear in the pull request.

Epilogue

As you can see, if the first commit in your repository is unimportant, things are actually pretty straightforward. I’d love to hear if anyone has a better way to deal with the case where we want to review all commits, including the first. Finally, if any GitHub employees are reading this - please make it easier for people to conduct full reviews! :)

What Python installations are scientists using?

2013-01-13T10:10:00+01:00

Back in November 2012, I asked Python users in Science to fill out a survey to find out what Python, Numpy, and Scipy versions they were using, and how they maintain their installation. My motivation for this was to collect quantitative information to inform discussions amongst developers regarding which versions to support, because those discussions are usually based only on guessing and personal experience. In particular, there has been some discussion in the Astropy project regarding whether we should drop support for Numpy 1.4, but we had no quantitative information about whether this would affect many users (which motivated this study).

In this post, I’ll give an overview of the results, as well as access to the (anonymized) raw data. First, I should mention that given my area of research and networks, the only community I obtained significant data are Astronomers, so the results I present here only include these (though I also provide the raw data for the remaining users for anyone interested).

Before I show the results, I just want to make it clear that I am not claiming that the results are a true sampling of Python user levels. I advertised the poll via Twitter, a couple of Python mailing lists, and the Facebook group for Astronomers. The survey was announced on different days on Twitter and Facebook, so there may be some useful information about the typical Python installations of Twitter vs Facebook users buried in the data that I won’t cover here. If anyone is interested about when the announcements were made, to correlate with response peaks in the data, please let me know!

With that out of the way… let’s look at the results!

Overview

First, some general stats - there were 313 responses in total, of which 244 were related to Astronomy (where I use the term in the broadest sense, including solar physics, planetary science, astrophysics, and cosmology). The responses were recorded between November 17th 2012 and December 2nd 2012 (at which point the rate of responses had gone down to less than one a day).

Python Versions

As shown above, an overwhelming 80% of Astronomers use Python 2.7, and almost 15% use Python 2.6. Almost no-one uses Python 3.x for production work yet, which is not surprising, given that at the time of the poll there were not stable versions for all the crucial packages in a scientific Python stack (in particular, Matplotlib only released their first Python 3.x compatible release in December). It will be interesting to see how this fraction changes over the next year (more on that in future blog posts).

Numpy Versions

In the above plot, dev includes anything that is a developer version more recent than the 1.6.2 release (which was the latest stable release at the time of the poll). The distribution is again significantly peaked, with almost 80% of respondents using Numpy 1.6.x. There is more of a spread in the remaining versions compared with the Python versions, but the vast majority of people are using Numpy 1.5.x or more recent.

Scipy Versions

In the above plot, dev includes anything that is a developer version more recent than the stable 0.11 release (which was the latest stable release at the time of the poll). Unlike the Python and Numpy versions, which are almost exclusively dominated by two versions, the Scipy versions show a larger spread, with the most popular version, 0.10.x, representing less than 45% of users.

I originally thought that Scipy released more often than Numpy, and this would explain the difference, but it seems that both projects have been releasing at a reasonably similar rate (see here and here). Therefore, this might be to do with package managers, or simply to the fact that Numpy is used more often than Scipy, and users are therefore more likely to run into bugs and update to the latest stable version? I have to admit that I would not even be able to tell without checking what Scipy version I am using, whereas I know I’m using Numpy 1.6.2 for production work.

Installation

We now get to some very interesting statistics - how users install Python and dependencies. While Python is awesome in many respects, installation is probably the biggest hurdle that users have to jump to get started.

I’m not sure if anyone’s quantitatively looked at this before, but this was the first time that I really got a sense for all the different ways that one can maintain a Python installation, and which methods are the most popular. The options shown above are described below:

Linux Manager means linux package managers (apt-get, yum, etc.) Source means an installation from the source code. This means either downloading the source code and running python setup.py install, or using pip install or easy_install. EPD stands for the Enthought Python Distribution, which is a scientific Python bundle that includes e.g. Numpy, Scipy, Matplotlib, and many other packages. It is free for users at academic institutions. MacPorts is one of the most widely used package managers on Mac, and I have provided instructions for getting set up with Python and MacPorts here. Official Installers refers to the MacOS X disk images, Linux RPMs, and Windows installers that are provided by some projects (including Python itself, Numpy, and Scipy). Admins means that Python and the packages were installed by System Administrators. SciSoft and STScI Python are two Astronomy-specific software bundles. And ActivePython is similar to EPD, but where binary packages are downloaded on-the-fly as needed.

Of course, some of these are not orthogonal, because for example easy_install can be used to install additional packages not in EPD. But the responses from the survey refer to how the main packages (Python, Numpy, and Scipy) were installed.

What can we take away from the results?

If we combine the Linux Package Managers and MacPorts (one of the Mac Package Managers) into a more general Package Managers category, this amounts to around 40% of users, the single largest group.
Only a small fraction of people use the official binary installers, with many more people installing from source. This was surprising to me, given how quick/easy it is to install Python, Numpy, Scipy, and Matplotlib using the official installers. I think this is down to the fact that this is not a well-documented installation procedure, and is platform dependent.
Astronomy-specific bundles (SciSoft and STScI Python) are not as widely used, which indicates that more effort should be put in getting packages in existing package managers than building new software bundles.
A small fraction (around 7%) have no idea how they installed Python and other packages, so they may run into issues when they try and upgrade in future. If you install Python for someone, please explain to them what you are doing and how they can update packages in future!

I personally feel that we should encourage users to install Python and whatever dependencies are available from package managers. Of course, in some cases users don’t have root access, but this generally means that they have sysadmins, so in those cases, the best option is still for the sysadmins to install the main Python packages via package managers.

Summary

To me, one of the most interesting results is that a large number of people have a reasonably up-to-date installation, with Python 2.7 and Numpy 1.6.x, and I imagine that the Python 2.7 peak is here to stay, given that the transition to Python 3 will be slow.

For developers, supporting only Python 2.6 and above seems like a sensible choice at this stage (a decision we made within Astropy), and given the imminent release of Numpy 1.7.0, I think that developers can start thinking about dropping support for Numpy 1.4 in the near future. For Scipy, things are a little more difficult, given the broad spread of versions, so developers should ensure that they know what versions they are implicitly supporting, and to check what version users have installed.

In terms of installation method, I think it’s very important to ensure that packages are included in package managers. Even if it is easy to install packages via pip or easy_install in some cases, putting packages in package managers ensures that users will more likely stay up-to-date with the most recent versions.

There is more information still contained in the data than I covered here (for example, some of the above points can be correlated - do the people who do not know how they installed Python correlate with the older versions?). For anyone who is interested in looking at the data, I’ve placed the files and the scripts I used to make the above plots in a GitHub repository here.

If you have any thoughts about the results, or find anything interesting in the raw data, please leave a comment!