So which python version you want?

June 1, 2010

For pkgcore, we run a pretty comprehensive set of buildslaves targets for testing pkgcore. Specifically

  • python 2.4
  • python 2.5
  • python 2.6
  • unladen swallow; python 2.6 based
  • python 2.7 snapshot (20100523)
  • python 3.1
  • python 3.2 VCS snapshot (20100523)

Originally, I’d ran this as separate KVM instances. This is nonoptimal however, since each instance is basically the exact same OS just w/ a differing python version overlaid, and w/ buildbot’s buildslave running within each. So via bastardizing some LXC work from diego, we now run a single kvm instance w/ each python version (and it’s buildslave) tucked away into their own container. Each container (including the raw parent) is intentionally externally addressable so that developers can reach into the container and tinker as needed, or experiment w/ that particular version of python.

Couple of folk have poked me for access to a copy of the vm image, so the puppy was stripped down (buildslave machinery removed among other things) and was posted to gentoo mirrors (and here is a direct link to allpython-amd64-qemu-20100531.qcow2.xz); still is propagating in full, but hit whatever your favorite local mirror is and raid it from there.

Few things to note about this vm image:

  1. it’s configured for, and expects to get access to it’s block device as virtio; this is tweakable, but really not recommended (virtio performance is quite nice)
  2. the containers are currently named buildbot-py$VER; this is a hold over from stripping down pkgcore’s buildslave vm… and my own lazyness in not changing the names
  3. each container root is actually an AUFS2 union of the raw parent FS. This was done so that the container could still share dentry cache for it’s libs w/ the parent, and to keep each containers footprint minimal.
  4. these are *full* containers, intentionally so to keep them from screwing up the parent/eachother in any fashion.
  5. the root password is ‘python’. Strongly suggest you change that if you expose this puppy publically.
  6. this is a bit of a custom setup- this is a patched version of lxc (backport adding init shutdown support), and a nasty little trick in the lxc init scripts to allow the parent to cleanly tell the guest container to shutdown (lxc-ps –lxc auxf # is a good command to look at- note that init is not the first process in each container).
  7. if you’re just running this for your own usage in a non deployed manner, feel free to remove acpid. acpid runs by default since the buildslave VM this was derived from is ran via init scripts, so there needed to be a way to tell it to shutdown (monitor shutdown events trigger acpi events, thus acpid).
  8. This was an oversight on my part during the scrub/releasing, but the default python in each container is still set to python-2.6. Feel free to run `eselect python set` to change the default version. In our buildslave usage, we leave the default python as 2.6 and force the target python via the buildbot step’s themselves; this was done to avoid having to modify buildbot bits hardcoding the python version into it’s shebang.
  9. Bugger is running a snapshot of pkgcore/snakeoil; mainly wanted a couple of unreleased fixes in there. VM and containers have been maintained/created via pkgcore in addition (issues/bug reports welcome).

Finally, there is *zero* support for this. I’m interested in bugs mind you, but I’m not supporting this- I’m just putting it out there since people have asked for it. Also if you’re interested in building your own buildslaves setup targeting multiple python versions based on this, feel free to either find me in irc or email me. I’ve been quite happy w/ this setup, including it’s minimal resource usage.

Hope it’s useful.