Over the past few months I’ve been working on using Puppet to automate my server setup. At the moment I only have one server, but it is still a pain to rebuild every few years whenever an old version of Ubuntu is discontinued. You can see what I’ve got so far on GitHub. Puppet seems like a great idea but documentation is absolutely appalling.
The tool itself has changed quite quickly, so a lot of old documentation isn’t relavent anymore and others (on the main site) has “not yet been written”. Hmm. It is also quite clear (not trying to flame, but anyway…) that the tool is built for sys admins not developers. This isn’t necessarily a bad thing as they are going to be the ones mainly using it, but if you just want to Get Shit Done™, you are in for one hell of a ride.
My previous attempts to use Puppet (see my GitHub repo for examples of both!) have been using Puppet under Vagrant (really nice, except I couldn’t find an Ubuntu 11.10 image… so I had to build my own) and using some custom magical single server setup (again, see the repo). I’m actually beginning to use Puppet across more than one server now though, so I wanted to setup Puppetmaster. This is basically a central repository to Puppet scripts, which shared them out to connected clients. As expected there was no documentation, so I wrote this!
Hostnames
Puppet uses SSL certificates to ensure that the client and server are who they say they are. This prevents your clients from receiving bad commands, and from your server revealing private data (MySQL root password) to untrusted hosts. For this to happen the client first talks to the server, and the server keeps it in a pending state. You then need to go onto the server and run a command to mark the client as safe. After that it will receive commands.
For this to happen firstly the time needs to be in sync between the boxes, and also the hostnames need to be the same on each (e.g. the master is master.home on both the master and client). As such you might need some /etc/hostname and /etc/hosts trickery.
Installation
Next up install the Puppet packages. Ubuntu versions lower than 11.10 have out of date versions, so don’t even bother installing those. If your host only provides images for older versions either move somewhere else or upgrade the installation once you’ve got the box setup.
All commands should be run on root, those denoted master should be run on the master, and those denoted client should be run on any clients (simple eh?).
master# apt-get update
master# apt-get install puppetmaster
client# apt-get update
client# apt-get install puppet
Start the master
When the mater is started it’ll generate it’s own certificate so make sure you have hostnames sorted by this point. Don’t worry if the directory you are deleting doesn’t exist, that it just to clear previously generated certificates.
master# rm -Rf /var/lib/puppet ssl
master# puppetmasterd --verbose --logdest console --no-daemonize
Start the client
The same notes apply to the master as the client.
client# rm -Rf /var/lib/puppet ssl
client# puppet agent --verbose --logdest console --no-daemonize --server=master.home
After that, you should see the following on screen (as well as other stuff):
Client:
info: Creating a new SSL certificate request for client.home
info: Certificate Request fingerprint (md5): 0A:7B:EB:BF:63:E5:CC:92:03:96:28:43:65:59:08:3C
Master:
notice: client.home has a waiting certificate request
Verify client certificate
Next you need to verfiy the clients certificate:
master# puppet cert list
client.home (0A:7B:EB:BF:63:E5:CC:92:03:96:28:43:65:59:08:3C)
master# puppet cert sign client.home
notice: Signed certificate request for client.home
notice: Removing file Puppet::SSL::CertificateRequest client.home at '/var/lib/puppet/ssl/ca/requests/client.home.pem'
After a while…
The client automatically retries to connect to the master. You can either wait, or restart the process on the client. Then you should see this:
Client:
info: Caching certificate for client.home
info: Caching certificate_revocation_list for ca
info: Caching catalog for client.home
info: Applying configuration version '1327959635'
info: Creating state file /var/lib/puppet/state/state.yaml
notice: Finished catalog run in 0.03 seconds
Master:
notice: Compiled catalog for client.home in environment production in 0.02 seconds
Success
After that you’ll want to daemonize the processes, which should just involve some /etc/init.d trickery. After that you are done! Puppet scripts go on /etc/puppet on the master.
]]>
Le sigh. I’m quite surprised that the Javascript works fine, but I’m using Backbone and jQuery for practically everything rather than rolling it myself, so that is probably why. The CSS though is a different story… I am also rather surprised about that as I thought Internet Explorer 8 was supposed to do a half decent job, I guess not. Below is what I discovered.
“IE doesn’t believe in HTML 5 elements”
Some of the CSS selectors applied to header tags, which IE doesn’t know about. As such it just ignores them. Adding the Javascript from html5 doctor solves that. This fixed the message date and hover-menu styling.
Menu Highlighting
This was actually my own fault rather than IE’s. I used the Ultimate CSS Gradient Generator to generate the code for the gradients, and for certain elements wanted to override it. This was easy, I just needed background: none (e.g. to remove it). However I missed that it also generated IE specific filter code. I also needed filter: none. Next…
Menu Dates
Unlike the message dates, the menu dates weren’t fixed with the teaching-IE-about-HTML5 Javascript file. Again the CSS rules being ignored were on header tags, however for some reason these were still being ignored. I couldn’t figure this one out, so I just moved the rules to other tags. Meh.
Box Shadow
CSS3 Generator, which I used for the other CSS3 effects didn’t supply a filter tag for the box shadow which IE supports, so this needed to be added:
]]>
White City.
Victoria.
Southwark.
]]>
Next up, it now separates different types of links, so articles are separated from pictures and music. This is probably the biggest new feature for me in this release, as before you were inundated with links and didn’t really know too well what they were. Previews are shown for pictures as well which I quite like. I’m looking to extend this for other media types as well, for example checkins and videos. If you have any other requests let me know!
There have also been a couple of changes behind the scenes, link expanding for example is a lot more robust than it was before (old links have been reprocessed), so more articles should be picked up. I’m currently working on rewriting the Tweet fetcher to use the streaming API, so that new links will appear near enough at the same time as they are posted to Twitter. I am also going to add pagination (of some sort, I’m thinking infinite scrolling) to the list of links, as some people have quite a lot of links. My list of images, for example, takes quite a while to load and for the browser to render.
I’m still wondering where to take this in the future, so I am open to any suggestions or requests you may have! One thing I was thinking of doing was making the link expander and detail fetcher thingy (I need someone to explain that in a marketing way) publicly accessible, if something like this had existed in the first place I wouldn’t really have to have built Tweeted Links!
]]>
ext4
Using the default options, on the face of it, ext4 doesn’t provide any significant advantages over ext3, or even ext2 when used for a media drive. However there are some nice features behind the scenes:
Optimising
Next up is to pass some options to tune2fs to enable the advanced features. These options could have been passed to the previous command, however to easily explain they have been listed separated. You can also run this command to upgrade an existing ext3 file system. A lot of these are enabled by default, however they are listed here for completeness sake.
