Planet GLLUG

Dean Wilson: By Puppet or Package

Fri, 03 Jul 2009 22:12:00 +0000

At work we both build our own packages and use puppet to manage our servers. While the developers package up their work in the systems team we've moved more to deploying programs and their dependencies via Puppet.

While it seems easier, and quicker, to do the pushing that way, at least for scripts, you lose the ability to track what's responsible for putting each file on the system. I'm probably already modelling the more complex parts of what would be in a package (such as services and cronjobs) in the module and thanks to Puppet I'm probably doing it in quite a portable way. Is this actually better than using packages? It's certainly easier than building complex packages but it quickly gets awkward when you start needing to deploy compiled binaries or apps with lots of moving parts.

For now my rule seems to be - use puppet for small, non-compiled, apps and package up anything with lots of dependencies or that needs to be compiled. How do you deploy your infrastructure scripts and supporting artifacts?

Dean Wilson: EuroPython 2009 - Wrap up Post

Thu, 02 Jul 2009 21:13:00 +0000

Over the last week I've been up in Birmingham catching up with some old friends and attending some talks at the little get together of around 450 Pythonistas that was EuroPython 2009.

This was my second Python conference. The first was PyCon 2008, which was so well organised (by many of the same team as this years EuroPython) that I was inspired to come back. And I wasn't disappointed. There were a lot of very good talks, some that have planted seeds that I'll have to come back and try to find the time to look at and some that showed me things I plan on using in the very near future (such as py.test).

The atmosphere was topnotch. Everyone seemed friendly, the speakers were approachable and after spending the evening with so many people working on so many things it was a pleasure to get back to the room and make sure I actually did something technical before bed.

It's a wonderful feeling to come away from a conference feeling motivated to try new technologies and all I need to do now is actually schedule some time actually write some Python code...

The organisers did a great job and I'll be back next year.

Dean Wilson: dstat - a window to your system

Wed, 01 Jul 2009 21:32:00 +0000

When it comes to Unix diagnostics I was raised the old fashion way, with iostat, vmstat and similar tools. However times change and tools evolve. dstat, while not as comprehensive as using all the tools one by one, provides a wide range of system performance details in an easy to use package.

While it's useful enough in its default state there is even more functionality lurking just below the surface. To see which other modules are available (but are not enabled by default) run dstat -M list. To add an extra module to the output use a command like this one: dstat -a -M topmem -M topcpu

As part of my growing use of the tool I've started to write my own little dstat plugins. I was pleasantly surprised at how easy they were to write and deploy even with my basic python skills. While the memcached plugin was a proof of concept I've not needed much I've found the process count plugin to be very handy.

dstat is becoming one of the overview tools I use when investigating performance issues and it's worthy of a place in your toolbox too.

Karanbir Singh: Connection from UDP log messages

Wed, 01 Jul 2009 10:20:29 +0000

Plenty of people seem to have an issue with snmpd logging connection information for each poll on CentOS 5, like this:

Jul  1 09:50:04 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:59768
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:56329
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:42126
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:47950
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:36634
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:52677
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:44864
Jul  1 09:50:05 doghouse snmpd[4159]: Connection from UDP: [10.0.1.10]:54498

This isnt necessarily a bad thing, however if you want to turn that off, on a fully updated CentOS-5 machine you can change /etc/sysconfig/snmpd.options to something like this :

# snmpd command line options
OPTIONS="-LS4d -Lf /dev/null -p /var/run/snmpd.pid"

Thats about all there is to it. Also, the reason why one finds so many different ways or achieving the same thing on the internet, isn't because everyone is wrong. Its mostly since the -L options syntax and use has been changing over the last few years. And while I am sure there is a good reason for this change but the developers really should consider keeping some backward compatibility in place.

- KB

Original post.

davblog - Dave Cross: Hard Rock Calling

Tue, 30 Jun 2009 13:18:11 +0000

On Saturday I was at the Hard Rock Calling festival in Hyde Park. It's a pretty good idea to have a three-day festival in London over the same period as the Glastonbury festival as it's easy for the acts to play both festivals over the weekend and those of us whose camping days are long behind us get to enjoy some of the Glastonbury bands and sleep in a comfortable bed.

My main reason for going was to see Neil Young, but there were a number of other acts (across three stages) to see first.

We arrived just as the first act started on the main stage. They were called The Original Sinners and my friend said he was interested to see them. It turns out that there must be more than one band of that name as these really weren't the band he was expected. They were pretty dreadful all in all. They were followed by Rudy Vaughn who had apparently won a competition in the US to come over and play at the festival. They were even worse. And instantly forgettable (I just had to look up their name). The only thing I can remember about them was that they finished with a cover of Michael Jackson's "Beat It".

By now I was desperate for some decent music. The next band on were The Pretenders and they didn't disappoint. To be honest I was surprised that they were still going. Haven't most of them died? But Chrissy Hynde still has it and their set was head and shoulders above what had gone before. They were followed by Seasick Steve. I've seen him on TV a few times and I've never really understood what the fuss is all about. But live, it's a completely different story. He was brilliant. It's just him and a drummer, but they had the huge festival audience hanging on every note.

The next act on was Ben Harper with his new band. I'd never heard of him and nothing I'd read about him filled me with any enthusiasm so I took the opportunity to explore the other stages. I didn't get very far as on the first stage I walked past I came across Alessi's Ark. I was completely entranced by their modern folky tunes and I sat and listened to all of their short set. After that I wandered back to the main stage where I was unfortunate enough to catch the end of Ben Harper's set. Which really wasn't very good.

The sun had be shining all afternoon, but as Harper left the stage it was becoming obvious that a storm was on the way. And sure enough, within fifteen minutes we were all rather wet. There was even some thunder and lightning.

The next band on were the Fleet Foxes. It was still raining when they came on, but it stopped soon afterwards. I love their music, but I'm afraid the rain rather dampened my enthusiasm. When they were playing and singing it was fabulous. But they take their music all very seriously and that means that inbetween songs there is interminable retuning of instruments which rather destroys the atmosphere. I'd love to see them again in a more appropriate venue. It sounds like I didn't enjoy them, but I should point out that even after two hours of Neil Young, it was still Fleet Foxes tunes that were stuck in my head on the way home.

Some time later Neil Young took the stage. I've been a Neil Young fan for years (we all had copies of "After The Goldrush" when I was a teenager) and I'm appalled that I've never seen him before. It's not like he rarely tours or anything.

Young opened with "Hey Hey, My My" (or perhaps it was "My My, Hey Hey" - I get those two confused) and the next couple of hours is a joyful blur of music and feedback. He played pretty much everything I wanted to hear (the full setlist is online). Oh, ok, there were a couple of things missing, but with a back catalogue the size of Young's you'd need to play all day to fit everything in. In amongst the grunge and feedback he found time for a more folky section where he played things like "Heart of Gold" and "The Needle and the Damage Done". If I have one tiny criticism, it's that if he learned to finish his songs a bit sooner he could fit in twice as many. For example, he finished with "Rockin' in the Free World" which had at least four false endings.

The encore was the Beatles' "A Day in the Life". I understand he's be playing that a lot on this tour. But when he got to "Woke up, fell out of bed..." Paul McCartney joined him on stage. I can't really see Paul McCartney and Neil Young as a double act so it all seemed a bit strange to me. I suppose I can say that I've now seen McCartney live. And I won't have to go to one of his (increasing rare and, I expect, overpriced) gigs.

This has to be one of the best gigs I've ever seen (and I said that after seeing Magazine earlier this year too - been a good year for gigs so far). I think this was the last night of the tour. But he'll be back. And if you get the chance to see him play, I strongly recommend that you take it.

Huw Lynes: Europython - Day 2 - Tutorials

Mon, 29 Jun 2009 21:34:22 +0000

Today’s notable achievements were that I managed to stay on power and network for most of the day. Mostly due to the fact that I lucked out to get a seat next to a power bar in the lecture theatre holding Luke Leighton’s Pyjamas tutorial. I was interested in Pyjamas for a web project I may have to get up and running quite quickly over the summer. Although there were some rocky patches due to SVN mismatches I mostly managed to get a handle on how Pyjamas works. As a note to future tutors: if you need your tutees to download the trunk from SVN it’s probably best to specify the revision that works. This avoids everyone turning up with a version of your code that won’t run the examples. Also, I still don’t understand decorators.

Today’s buffet lunch was nice. Props to the conference organisers.

The day was nicely rounded off by dinner at a fine indian restaurant and a pint of very nice beer in the Wellington. Looking forward to the start of the conference proper.

Huw Lynes: Europython - Day 1 - Tutorials

Sun, 28 Jun 2009 18:56:26 +0000

I have to admit to a certain amount of trepidation when I signed up for EuroPython 2009. As primarily a sysadmin rather than a developer I was worried that I might not have the requisite knowledge to get the benefit of a week-long developer conference. After today’s experience I’m beginning to relax about that.

Today and tomorrow are the tutorial sessions before the conference proper starts. Having never been to a Python conference before I wasn’t sure what form the tutorials would take. From the outcome of the day I would have to say “much less programming than you might expect”.

The day started off with Michael Spark’s giving an introduction to Kamaelia the simple concurrency system designed by BBC Research. We started off by building a brain-dead simple version of Kamaelia to outline the principles by which it operates. This took us on to writing a bulletin-board system by chaining together simple Kamaelia components. This was, needless to say, pretty intense for a Sunday morning.

Having expected to be doing a lot of coding I dutifully spent Friday evening makeing sure that I had the suggested software installed and working on my netbook. As it turned out I only wrote about 20 lines of code during the whole tutorial. I was ever so slighlty miffed by this. This is the first time this tutorial has been given and in my opinion would benefit from being all-day with time for coding exercises between explanations.

Despite these minor problems I felt that the tutorial left me with enough of a grasp of Kamaelia’s basics that I could go away and write something simple in it without too much trouble. One other good point of the this session was the handout printed from lulu.com which was really nice. So nice in fact that I think we should spring for these next time we run a training course at work.

After lunch I was in Jonathan Fine’s JavaScript for Python Programmers tutorial. Which was in a room that was too small for the audience and much, much too hot. It also appeared to have a grand total of two power outlets. Fine started off with a horrifying list of the ways basic constructs in JS behave in ways that Pythonistas will find completely illogical. After the break he delved into the nitty-gritty of OO and Inheritance. As the tutorial progressed and Fine got further from his slides the session transformed into something more like a seminar rather that a tutorial. Overall I found this session enjoyable and informative, although my brain was beginning to melt by the end of the day.

I suspect that Wifi and power are what most people will grumble about, but knowing how hard it is to sort these out for events at my home institution I won’t carp too much.

Now for some time with the Django tutorial in preparations for tomorrow’s Pyjamas session.

Rev. Simon Rumble: Announcing: Swedish Chef wave robot

Sun, 28 Jun 2009 05:09:09 +0000

I managed to score myself a Google Wave account by promising to write a Swedish Chef robot. So this afternoon I wrote one, in a language I've never used before (Python) and to a target platform I've never explored (Google App Engine). Mostly I ripped off code from other sources, especially Dive Into Python.

So if you have a Google Wave sandbox account, add borkforceone@appspot.com to a wave and your text will be translated into cod-Swedish.

Contact me

davblog - Dave Cross: Checking Copyright

Thu, 25 Jun 2009 15:46:20 +0000

There's a lot of material out there on the internet. And the nature of the internet means that it's easy to reuse that material without paying any attention to copyright. If my browser can display an image, then I can save that image to my local disk and then, perhaps, use it on my own web site or in some other publication.

But just because it's easy from a practical perspective, that doesn't mean that it's legal to do it. Much of the material on the web is subject to various copyright restrictions. And if you're going to be a responsible internet citizen then you're going to ensure that you are careful not to use any material in ways that are contrary to the copyright.

If you are, say, a national newspaper then you're going to want to be really sure that you're being careful about copyright. I'm sure that someone like (to pick a paper at random) the Daily Mail would get very upset if they found someone using one of their photos without permission or without giving correct attribution. It's therefore reasonable to expect them to offer the same courtesy to others.

Take a look at this story about Philip Schofield and Twitter. Don't bother to read it. It's the usual Mail nonsense. They're complaining that Schofield shares too many details of his life on Twitter. But they do it (ironically, I'm sure) by poring over every detail of a meal in the Fat Duck. No, don't read the words. Take a look at the pictures. Schofield has illustrated his evening by posting photos to TwitPic. TwitPic is a Twitter "add-on" that allows you to share photos as easily as Twitter allows you share text.

Notice that the Mail have put a copyright attribution on each of Schofield's photos. They all say "© Twitpic", implying that that TwitPic own the copyright on the photos. But if you take a few seconds to read TwitPic's terms and conditions, you find that they say:

All images uploaded are copyright © their respective owners

TwitPic lay no claim at all to copyright on the pictures, so the Daily Mail are attributing copyright to the wrong people. It's not at all hard to find this out (it's a link labelled "terms" at the bottom of the page - exactly the same, in fact, as it is on the Mail site), but the lazy Daily Mail picture editor couldn't be bothered to do that and just guessed at the copyright situation.

And whilst we're talking about the Mail not understanding copyright, it's worth remidning ourselves of the nonsense in their terms and conditions.

3.2. You agree not to:
3.2.1. use any part of the materials on this Site for commercial purposes without obtaining a licence to do so from us or our licensors;
3.2.2. copy, reproduce, distribute, republish, download, display, post or transmit in any form or by any means any content of this Site, except as permitted above;
3.2.3. provide a link to this Site from any other website without obtaining our prior written consent.

Under clause 3.2.3, I've broken their terms at least twice in this article. But clause 3.2.2 is the really interesting one. You're not allowed to download or display the content of the site. Which makes it rather hard to view it in a browser. Idiots.

Update: They have now changed the copyright on the photos to "© Philip Schofield/Twitter". So that's one less piece of stupidity in the world. The struggle continues.

Richard WM Jones: Use a snapshot to test libguestfs changes

Wed, 24 Jun 2009 15:23:51 +0000

If you want to test out some experimental libguestfs changes to a virtual machine, you can use a qemu snapshot to test changes before committing them. Here’s how. First, use qemu-img to create a snapshot. In this case I have a base file called “RHEL52PV32-20090213.img” and my temporary snapshot will just be called “snapshot”. [...]

davblog - Dave Cross: Banging Your Head Against a Brick Wall

Tue, 23 Jun 2009 14:29:55 +0000

Long-time readers will know that I am not averse to contacting companies to complain about bad service that I receive. This isn't a particularly fulfilling hobby as you very rarely get any kind of satisfaction. But recently it's becoming even less satisfying than before. I've noticed that email conversations with customer service reps are becoming more and more drawn out as many of them seem less and less capable of understanding the issues that I am raising. It can often take a couple of rounds of email before they are clear what I'm talking about. And I'm pretty sure it's not me describing things badly.

Here's an example.

Last week the BBC showed an abbreviated version of Leonard Cohen: Live in London. I missed it as it was first broadcast, so over the weekend I tried to catch up with it on iPlayer. We have Virgin Media, so we can watch iPlayer content through our V+ box. I found the programme and started to watch. I didn't last long though as the aspect ratio of the programme was wrong. The programme had been filmed in 4:3, but the iPlayer has stretched it to 16:9[1]. This meant that everyone everyone appeared fatter than they should be. I know that many people are used to watching television like this, but to me it renders a programme unwatchable.

I wrote to the iPlayer support team explaining the problem. Here's what I wrote:

The version of "Leonard Cohen - Live in London" which is currently available on iPlayer on Virgin Media is in the wrong aspect ratio. It appears to be a 4:3 broadcast which has been stretched to 16:9. Everyone therefore seems to be far too fat and the programme is unwatchable.

I think that's clear.

This morning I got a reply from them. Here's what they said:

I understand you're unhappy with the size of 'Leonard Cohen' on BBC iPlayer.

The bit rate varies per programme and is dependent on the amount of changes per video frame. For example, a programme such as 'Doctor Who' or 'Leonard Cohen' will be at a higher rate than a Current Affairs programme (where a presenter is fairly static in the frame).

Considering the above information:

The average file size for a 30 minute streamed programme is around 110MB. The average file size for a 1 hour streamed programme is around 215MB.

I appreciate you may feel differently on this matter and I'd like to assure you that I have registered your comments on our log. This is the internal report of audience feedback which we compile daily for BBC iPlayer and commissioning executives within the BBC, and their senior management. It ensures that your points, and all other comments we receive, are circulated and considered across the BBC.

Nothing in there at all about aspect ratios. They seem to have assumed that I'm talking about the size of downloaded programmes. Which is strange as I specifically mentioned the Virgin Media version of iPlayer which doesn't support programme downloads.

I can see three explanations:

The person genuinely misunderstood what I was asking about and answered the question to the best of her ability. In which case she needs better training in the products that she is supposed to be supporting.
The person didn't read my mail carefully and just sent a reply that looked like it might address the issues I was talking about. In which case she needs to read email more closely. Perhaps the iPlayer support team needs more people so they have time to read messages and write replies carefully.
No-one read my email and some automated system sent a canned reply based on some (obviously flawed) keyword matching.

I know I shouldn't waste my time, but I've had another go at explaining exactly what the problem is and why this reply didn't address any of my issues. Let's wait and see what happens.

But it shouldn't be like this. Speaking to customer service shouldn't be like banging your head against a brick wall. People should know the products they are supporting and they should want to give the best service they can. It's becoming far too common that customer service replies appear to be dashed off as quickly as possible in the hope that no-one will actually bother to read the reply. Either the support team don't have the training to properly support their products or they are overworked and don't have time to do a proper job.

Either way, it's all very frustrating.

The BBC showed the Leonard Cohen concert again over the weekend. I recorded it on my V+ box. It was broadcast in the correct aspect ratio. I enjoyed watching it very much.

[1] I like to call this "Dixonsvision" in memory of the sadly-missed shop which used to insist on demonstrating all of its widescreen TVs this way.

Martin A. Brooks: Game review: Velvet Assassin

Sun, 21 Jun 2009 22:32:00 +0000

This game should have been the one that made me forget about the likes of the early Splinter Cells and Thief III. This game was born in the era of the expectation of hideously overpowered graphics cards, tons of memory, multiple cores and physics processing units.

Graphically, it is quite good. And what is used to distract you from merely "quite good" graphics is piss-poor linear game play, poor controls, sterile environments and, in fairness, excellent voice acting.

This game has everything wrong with it that you would expect when a moderately successful console games is crammed onto the PC platform by people who've never played any of the sneak'em'up greats. Such developers should be forced to complete Metal Gear Solid, all of the Splinter Cells and all of the Thief series before getting their hands on what should have been the best stealth FPS ever to grace a personal computer.

"Execute over 50 different brutal maneuvers to deliver a quick and silent death to enemy soldiers" the marketing says. What that says to me is I have 50 different brutal maneuvers at my fingertips, choice mine to dispatch an enemy. Bollocks. What in practice happens is you sneak up behind your unsuspecting target and click the left mouse button. That's it.

At this point it can go two ways. 1) You take out the enemy and have time to drag to corpse into the shadows. Jolly good. However what happens depressingly frequently is option 2. Our heroine turns into Miss Stabby and goes to town. Definitely killing the target kraut but taking so long about it that you get caught by the next chap walking along. Repeat.

Then you get moments of sheer comedy:

Kraut #1) Deary me, look at this puddle with the electric cable running through it!
Kraut #2) Mein Fuhrer! Someone could electrocute themselves!
Kraut #1) Ja! Let us hope no-one throws that big switch over there while we are walking through it.
Kraut #2) Nein! That would be awful!

Krauts 1 & 2 proceed to splash about in the water.

Don't buy this game, I did, and it's rubbish.

davblog - Dave Cross: The War of the Worlds

Sun, 21 Jun 2009 12:39:54 +0000

Last night I was at the O2 Arena to see one of the 30th anniversary performances of Jeff Wayne's War of the Worlds. For some reason it was a year late - the 30th anniversary was actually last year.

I wasn't entirely sure what I was expecting. I wouldn't have been surprised if it had been Jeff Wayne with a group of unknowns performing the album, I was therefore pleasantly surprised to find that with a couple of exceptions each part was played by the person who played it on the original album.

One of the exceptions was, of course, Richard Burton who narrated the album. But he was there in a way. A giant animated head was projected onto the side of the stage and recited Burton's lines. The animation was impressive but it was slightly strange that it was based on a version of Burton's face which was far younger than he was in 1978.

There was an added treat even before Burton's opening narration. The show began with an animation of the martians plotting their attack. Some research this morning reveals that it was taken from a computer game based on the album which was released in 1998. This piece is also the source of the sample "the problem is, of course, the humans" which is used in one of the Ulladubulla remixes and which has been bugging me for years.

Following this short film, Burton comes in ("No-one would have believed ...") and "The Eve of the War" starts up. The animation continues telling the story of the invasion and in the middle of the song, Justin Hayward walks onto stage and tells exactly what the odds are against anything coming from Mars.

The performance continues in the same fashion. The band (and orchestra) play almost note-perfect renditions of the music, the video screen plays out the story and occasional guest stars come onto the stage to sing. It's all really rather impressive.

The first missing guest star is David Essex as the artilleryman. I haven't heard why why he isn't involved but, to be honest, he isn't really missed. Alexis James (no, neither have I) takes the part and does it very well.

During "Thunder Child" another surprise is revealed as a large martian fighting machine is lowered to the stage and started shining bright lights over the audience. My companion pointed out that "it all went a bit Pink Floyd" at that point, but in my opinion that's no bad thing. As the Thunder Child was destroyed, Burton told us that "Earth belonged to the martians" and the first half of the show (the first album in the double album set for those of us old enough to remember such things) came to an end. We had a twenty minute break.

The second half starts in a lower key than the first half. "The Red Weed" is possibly the weakest musical link in the whole piece. Not that it's bad by any means, it's just that it doesn't (in my opinion) hold up to the standards of the rest of the album. The animation was very nice at this point though.

I was expecting the pace to pick up again with "The Spirit of Man" which is my favourite part of the album. On the original recording this is a duet between Phil Lynott and Julie Covington. I know, of course, why Lynott wasn't there but Covington was another unexplained absence. And these two were really missed. Lynott's place was taken by another unknown called Shannon Noll and Covington's by Jennifer Ellison. I really really tried to ignore my prejudice against Ellison, but she really isn't the equal of Covington. Maybe her presence drew in some people who wouldn't have come otherwise (I doubt it) but I was distinctly unimpressed by her performance.

The Alexis James returned to give us some more of his David Essex impression whilst singing "Brave New World" and the evening was back on track. Except that there wasn't really very much track left. If you were writing the piece for stage then you'd make sure that there was a big finale. The War of the Worlds doesn't have a big rousing finale. It just kind of fades out when the narrator realises that the martians have all been killed by terrestrial bacteria (sorry if that's a spoiler for anyone).

Most of the audience were people who knew the album well and knew that there wasn't much of interest likely to happen after the end of "Brave New World". Combine that knowledge with the fact that the transport links to North Greenwich were severely curtailed last night (no Jubilee Line) and you end up with a large proportion of the audience starting to leave before the show was over. I have never seen so many people streaming out of an auditorium whilst the band were still playing.

We stayed until the end. It was good, but not as good as the end of the first half. There's nothing they can do about that. The first half is just better.

All in all, it was a great show though. If you're a fan of the original recording then I highly recommend it. If you're not a fan of the original recording, then you really should be.

Now, who's going to buy me a copy of the 7 disk collectors edition?

Rev. Simon Rumble: Acer Aspire One D150 and Ubuntu netbook remix

Sat, 20 Jun 2009 01:59:59 +0000

Earlier this week I bought an Acer Aspire One D150 to use as my new portable and, possibly, desktop. Most of these new netbooks are pretty much the same inside, so a few things won me over to this one:

Ten inch screen, substantially bigger than the seven inch netbooks.
Very good reviews of the keyboard, and I concur it works well with my fat fingers, and dedicated Page Up/Down keys are very handy
Built-in Bluetooth, which means tethering to my phone for mobile broadband is trivial
Built-in SD card (it does other formats too) reader, makes it trivial to upload my photos
VGA-out plug, meaning I can potentially use it as a desktop with two screens

Downsides include the Microsoft tax (which I'll attempt to recoup, after Simon Hackett's encouragement), a hard drive that I don't really need and probably is an unnecessary drain on battery, and a touchpad that has been fairly strongly vilified.

I'd tend to agree that the touchpad is pretty poor. The buttons require so much force that you really have to use two hands to do anything like click-drag. That said, I mostly don't use these things anyway and carry around a little retractible mouse anyway.

The install from Ubuntu Netbook Remix was trivial. Change the BIOS settings to boot from USB (F2 at boot to access BIOS) and boot. It was done in about fifteen minutes and most things just worked, including wireless, suspend and hibernate.

I've found a few issues with sound. Playback from Rhythmbox, the default Ubuntu music app, can be a bit choppy. I suspect this is just the application, and I might just change apps. Sound after suspend doesn't seem to work, which is a bit annoying. I haven't worked out how to reliably get the sound to work with Skype either. I'll keep playing with that.

Settings to change

I've made a couple of changes to the default Ubuntu install for this system.

I disable Caps Lock on all machines I use, because it's a completely useless key and my fat fingers often hit it accidentally. The Caps Lock key on the Aspire One is no exception, and I have to wonder why they would include one when a dedicated NumLock key would be more helpful. Add the following to ~/.bashrc:

 if [ "$PS1" ]; then
    # Disables the bloody CapsLock button
    xmodmap -e "remove lock = Caps_Lock"
 fi

Inexplicably, Ubuntu disables laptop_mode by default, which means it doesn't do useful things when running on battery power that will extend battery life. It also makes it hard to work out why it isn't running, putting the setting in a seemingly unrelated file, and returning nothing when you try to run the init script. Change ENABLE_LAPTOP_MODE=false to ENABLE_LAPTOP_MODE=true in /etc/default/acpi-support to enable it.

The touchpad is overly sensitive and when you're in the middle of frenzied typing, often moves the cursor on you. Most annoying. I get around this by disabling clicks from the trackpad, given I don't use it anyway.

Firefox, by default, takes up a lot of useless vertical screen space. I've reduced this by removing the Bookmarks Toolbar and moving it up next to the menu. I also installed the Littlefox theme, which uses much smaller icons. This gives you a bit more of the critical vertical screen space.

The included soft slip case, made out of wetsuit material, is alright but has no space for my little mouse and a pair of headphones, which I think are essential portable accessories. I might try sewing on a couple of little pockets to make it perfect.

Conclusion

It's early days just yet, but I'm pretty damn happy with my new little netbook. It's suiting my needs pretty well, and looks rather fine too.

Contact me

davblog - Dave Cross: Guerrilla vs Gorilla

Fri, 19 Jun 2009 19:01:18 +0000

On the train home I was pondering the difference between guerrilla marketing and gorilla marketing.

Antonio points out that the Cadbury's Phil Collins advert might be an example of gorilla marketing, but I think it's probably deeper than that.

According to Wikpedia:

Guerrilla marketing is an unconventional system of promotions that relies on time, energy and imagination rather than a big marketing budget. Typically, guerrilla marketing tactics are unexpected and unconventional; consumers are targeted in unexpected places, which can make the idea that's being marketed memorable, generate buzz, and even spread virally.

In comparison, gorilla marketing is probably all about alpha males beating their chests and flinging faeces.

I know which type of marketing I'd rather use.

davblog - Dave Cross: Literacy and Professionalism

Fri, 19 Jun 2009 16:58:50 +0000

I remember a time, not very long ago, when people assumed a link between literacy and professionalism. When producing text for public consumption people would always take the time to ensure that their spelling and grammar were correct. Obvious errors in copy would be seen as a lack of attention to detail and would throw grave doubts on your level of professionalism.

Those days are long over. Here are a couple of good examples from today.

Firstly, I got an email from an agent who was asking if I'd be interested in a requirement that she was trying to fill. Her email began like this:

I hope your well?

That is, of course, a perfectly reasonable sentiment to start an email with. It's just a shame that her grammar let her down so badly. As well as confusing "you're" and "your", she has also tried to turn a sentence into a question. All in all it gives a bad impression of her company. If they can't be bothered to spend the time getting the grammar right in an email, then can they be trusted to check a contract carefully?

Then, this afternoon, on Twitter, Chris Applegate made this observation:

My, there are a *lot* of "gorilla marketing" experts on LinkedIn

Would you consider taking marketing advice from someone who didn't know the difference between "gorilla" and "guerilla"? I suspect that gorillas have a rather different marketing style to guerillas.

LinkedIn acts as a combination of a CV and an advertising hoarding. Seeing how you describe yourself there will often be the first impression that people have of your work. Making fundamental errors in your description can't be a good idea. Are there really so many people out there looking for marketing people that they don't care if you're a gorilla or a guerilla? Or are the gorilla marketing experts aiming their services at clients who don't know the difference either.

I strongly suspect that it's the latter option that is closer to the truth. There are so many people out there who have a no real understanding of how the English language works that it really doesn't matter whether or not you use it correctly. It's only a rapidly shrinking group of curmudgeons like me who will ever notice.

I suppose that before I publish this, I should really go over my LinkedIn profile with a fine-toothed comb. I don't think there are any errors, but I'm sure that my eagle-eyed readers will be able to spot one or two.

Will Jessop: NWRUG Code Surgery and an introduction to Zsh – Tonight

Thu, 18 Jun 2009 12:20:19 +0000

the June NWRUG is tonight. There will be a code surgery and introduction to Zsh. Plus free pizza!

Email me or leave a comment to sign up.

davblog - Dave Cross: More on uSwitch

Wed, 17 Jun 2009 08:29:53 +0000

The people at uSwitch saw yesterday's blog post and were rather pleased that I was so happy with their service. To show their thanks, they've written to me with an offer for my readers.

If you use this link to visit their site and change energy supplier before the end of June 2009, they will send you an Amazon voucher for £10. You'll also (obviously) need to give them a valid email address so they have somewhere to send the voucher.

I don't yet have a policy for promoting products on this site. Perhaps I should think about that if offers like this become more common. I should, however, point out firstly that I have no connection to uSwitch (I'm just a satisified customer) and secondly that I'm getting no payment for mentioning them again - the offer is just for my readers.

So, once again, here's the link to use. The offer is valid until the end of this month. Which gives you about two weeks to make use of it.

davblog - Dave Cross: Buying Power

Wed, 17 Jun 2009 07:59:54 +0000

How do you buy your power? Or, more specifically, how do you decide which company to buy your power from?

Here's how I do it. Ever six months or so I go to uSwitch and spend five minutes researching who does the cheapest gas and electricity for our usage. If I find something that is much cheaper than our existing supplier then I'll change. Most of the time I can make the change over the internet by simply following a link from the uSwitch site. Most of the time the differences are so small that it's not worth changing.

One thing that I will never do is to sign on for a new power supplier from people who knock on my door and try to tell me a new supplier without giving me time to investigate their offers. Firstly, I think it's incredibly rude to disturb people whilst they are at home, but mostly it's the hard-sell tactics that I object to.

We had one last week. He was from npower, but they are all as bad. I noticed him as I was walking home, but realised that he was walking away from my house. When I got in, my wife said that he had knocked at the door but she had ignored him. But an hour or so later he came back. I answered the door and he immediately launched into his nonsense. Apparently, most of my neighbours had realised that he could save them money so they had all signed up. He didn't have price data for my current supplier but he knew that he could save me money. I tried to explain to him how uSwitch works, but it seemed to just confuse him.

In the end I told him that I refused to do business with any company who called on me uninvited as I considered it rude. He started to argue that it wasn't rude, but realised that he was wasting his time and retreated.

Ususally I just let these things go, but on this occasion I decided to take it further. Firstly I checked with Uswitch and, as I suspected, my supplier (Scottish Power in case you're interested) we still the cheaper than npower by about a tenner a year. Then I emailed npower customer services to complain about them sending out uninformed and unethical sales people.

Today I got a reply from them. They were sorry to hear that I felt their sales representative was attempting to mislead me. They were also sure that normally "the standard of service offered by our Sales Team is professional and of the highest standard". They also pointed out that if I sent them my address they would pass my details to their "Marketing Supression Team" which would stop me getting further visits.

That last item got me thinking. Of course, it'll be an improvement if I get no more sales visits from npower. But there are many other power companies who delight in trying to mis-sell their services in this way. The absense of the npower team will scarely be noticed. Wouldn't it be good if there was a industry-wide "Marketing Supression" list that all of the power companies signed up to. I can't be the only person who gets annoyed by this.

I have no idea how effective these sales calls are. It must work to some extent as it's a relatively sales method. I know those people won't be paid much (and most of it will be commission), but there are a lot of them. I'm surprised that people listen to their nonsense, and I worry that they are preying on people who don't have the presence of mind to think about what they are being told and therefore take it at face value. I suppose that if you're told that "most of your neighbours are switching" and you're not given a chance to check that out before signing, then some people will just accept it.

Until we can get an industry-wide marketing suppression list (or, better, banning this kind of selling) can any one recommend any good tactics for annoying these people?

And please, make use of uSwitch.

Martin A. Brooks: Twitter

Tue, 16 Jun 2009 05:14:00 +0000

I'm known for being an early adopter of new Internet fads. You can now stalk me on Twitter, mart_brooks.

Will Jessop: Blocking Internet Explorer 5.5

Mon, 15 Jun 2009 21:08:49 +0000

IE 5.5 is dead. The number of people using it are tiny, at least according to the stats on the most popular sites I run. Here are the numbers.

Jokes-o-matic

Has a larger proportion of Firefox users than Internet Explorer users due to the high level of traffic from some social bookmarking sites, meaning a more web-savvy crowd who are more likely to be using alternative browsers:

jokes-o-matic.com browser breakdown

The level of IE usage is still pretty high though, but even so only a single hit from IE 5.5 recently:

jokes-o-matic.com IE versions

Recycling Group Finder

A much more broad and ‘average’ user base with Internet Explorer dominant:

Recycling Group Finder browser breakdown

Still, only 6 Internet Explorer hits recently, a tiny proportion of overall visits:

Recycling Group Finder IE versions

What does this mean?

Yahoo.com as seen in Internet Explorer 5.5

These statistics, and others I have seen suggest that almost no-one uses Internet Explorer 5.5 or below any more. The small minority that do must be used to the internet breaking in unique and interesting ways due to the lack of support for more recent developments in CSS and really bad Javascript support (see right for yahoo.com as seen in Internet Explorer 5.5). I predict that a fair number of sites just don’t work in any usable way on anything less than Internet Explorer 6.

Time is money

Designing for multiple browsers takes time. The more browsers you need to support the more time you need to spend supporting them.

A recent project I worked on required a redesign of the site from the ground-up (aswell as a bunch of other stuff) and making the site work in any version of Internet Explorer lower than 6 just wasn’t worth the time, and therefore money, put into it. The return just wasn’t there and I decided that IE 5.5 was gone, dead, obsolete and I wasn’t going to try and mangle the design to work in it. Rather than just display a really crappy looking site to the user I decided I would let them know via a short and to-the-point message that their browser was not up to displaying the site and directing them to upgrade to a new version of their browser.

Give them the good news

Message displayed to all users visiting project in anything below IE 6

But how to do this? The site was developed in Ruby on Rails and I’d heard of a neat gem called rack-noie6 that provides a piece of Rack middleware that blocks Internet Explorer 6 and below, but that isn’t going to work in most cases. Hard as it may be to make your site look good in Internet Explorer 6 you can see from the stats above there’s still a large proportion of users using it and to turn them away would be crazy, at least while the pool of IE 6 users out there is still so large.

So a quick fork and patch later (github we <3 you so much) and the new noie6 was accepting an option for a minimum version of Internet Explorer it should accept instead of just rigidly blocking version 6 and below. Configuration is really simple:

Now any users visiting the site in anything below Internet Explorer 6 gets the message above helpfully directing them to update their browser. Not bad for a two line config update.

Karanbir Singh: Gpg signed spam

Mon, 15 Jun 2009 13:10:38 +0000

I've just had my first ever gpg signed spam email. Yes it is gpg signed, and its signed correctly with a key that seems to not be mentioned anywhere ( or atleast papa google does not know about it ). These guys are still getting better and wiser. Whats next ?

Going to see how this shapes up over the next few days before dropping my gpg-header -> whitelist rule :/

Original post.

www.DavidPashley.com/blog: Copying files with netcat

Mon, 15 Jun 2009 12:51:51 +0000

When you want to copy files from one machine to another, you might think about using scp to copy them. You might think about using rsync. If, however, you're trying to copy a large amount of data between two machines, here's a better, quicker, way to do it is using netcat.

On the receiving machine, run:

# cd /dest/dir && nc -l -p 12345 | tar -xf -

On the sending machine you can now run:

# cd /src/dr && tar -xf - | nc -q 0 remote-server 12345

You should find that everything works nicely, and a lot quicker. If bandwidth is more constrained than CPU, then you can add "z" or "j" to the tar options ("tar -xzf -" etc) to compress the data before it sends it over the network. If you're on gigabit, I wouldn't bother with the compression. If it dies, you'll have to start from the beginning, but then you might find you can get away with using rsync if you've copied enough. It's also worth pointing out that the recieving netcat will die as soon as the connection closes, so you'll need to restart it if you want to copy the data again using this method.

It's worth pointing out that this does not have the security that scp or rsync-over-ssh has, so make sure you trust the end points and everything in between if you don't want anyone else to see the data.

Why not use scp? because it's incredibly slow in comparison. God knows what scp is doing, but it doesn't copy data at wire speed. It aint the encyption and decryption, because that'd just use CPU and when I've done it it's hasn't been CPU bound. I can only assume that the scp process has a lot of handshaking and ssh protocol overhead.

Why not rsync? Rsync doesn't really buy you that much on the first copy. It's only the subsequent runs where rsync really shines. However, rsync requires the source to send a complete file list to the destination before it starts copying any data. If you've got a filesystem with a large number of files, that's an awfully large overhead, especially as the destination host has to hold it in memory.

Read Comments (16)

Karanbir Singh: CPU fans for MS 9620

Sun, 14 Jun 2009 00:10:19 +0000

At work, my desktop machine is a dual opteron 250 machine built on a MS-9620 mother board ( http://www.msicomputer.com/product/p_spec.asp?model=K8N_Master2-FAR ) and in the last few months the machine has become really loud - source of this tracked back to the cpu coolers. There is another case fan ( as exhaust, at the back to the case ) which is very quiet, and turning it off seems to make no difference to the overall sound. The PSU also seems to be relatively quiet.

Anyway, the plan is to replace those CPU coolers with something quieter, and we got some Noctua coolers ( http://www.quietpc.com/gb-en-gbp/products/amdcooling/nh-do ). But, inspite of them saying that it will work with a Socket 940 based Mobo, these ones dont fit! the problem being that the MS9620 seems to have some sort of a custom bracket for the fans already on there, and there seems to be no 'usual' hinged/clip type mounting for the cpu coolers. Whereas the Noctua coolers only have that usual hinge/clip mounting. I've taken some pictures of mounting for the existing cpu coolers, you can see them here : http://www.karan.org/pics/ms9620/

Anyone know what replacement coolers I might be able to get for this MoBo ?

- KB

Original post.

Karanbir Singh: Thread summaries in thunderbird 3

Sat, 13 Jun 2009 23:52:05 +0000

One new feature I really like in the upcoming Thunderbird 3 tree : thread summaries in the message view pane. Here is an example:

Click the image above for a full size view.

What makes it even more cool, is that since Thunderbird can already recognise and mark the quoted portion in followup replies, it conveniently ignores this quoted text in the summaries.

Original post.

Karanbir Singh: dstat and disk device names

Fri, 12 Jun 2009 15:04:37 +0000

Running dstat normally gives you something like this :

# dstat --nocolor
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw 
  3   0  97   0   0   0| 820k  456k|   0     0 | 800B  866B|1054   255

And you can split the disk metrics up based on devices using something like this :

# dstat -D sda,sdb,total --nocolor
----total-cpu-usage---- --dsk/sda-- --dsk/sdb-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ: read  writ: read  writ| recv  send|  in   out | int   csw 
  1   1  97   0   0   0| 176k   77k:  30k  162k: 411k  478k|   0     0 |3608B 4005B|1447  1458 
 33   2  63   0   1   2|   0     0 :   0   216k:   0   432k|2470k 1611k|   0     0 |2915  6967 
 31   2  65   0   1   1|   0     0 :   0     0 :   0     0 |2210k 1338k|   0     0 |2866  6051

But you can also get deeper level device names / nodes to measure - which is required in many cases if you have drivers for storage creating nodes further down the tree than /dev. Eg. I have a setup where there are four mysql instances running, each with its own dedicated storage :

# mount | grep srv
/dev/cciss/c0d1 on /var/lib/mysql/node1 type ext3 (rw)
/dev/cciss/c0d2 on /var/lib/mysql/node2 type ext3 (rw)
/dev/cciss/c0d3 on /var/lib/mysql/node3 type ext3 (rw)
/dev/cciss/c0d4 on /var/lib/mysql/node4 type ext3 (rw)
/dev/cciss/c0d5 on /srv/wal type ext3 (rw)

so in this case, to get dstat reporting working you need to mention just the component level, like this :

# dstat -D cciss/c0d1,cciss/c0d2,cciss/c0d3 --nocolor
----total-cpu-usage---- dsk/cciss/c dsk/cciss/c dsk/cciss/c -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ: read  writ: read  writ| recv  send|  in   out | int   csw 
  3   0  97   0   0   0| 209k   89k: 201k   86k: 204k   85k|   0     0 | 799B  865B|1055   256 
 90   1   8   1   0   0|3196k   68k: 764k    0 : 816k   24k|1067k 1650k|   0     0 |1999   609 
 95   1   3   1   0   0|2548k    0 :   0  4084k:2448k 5700k| 660k  791k|   0     0 |1611   571 
 96   1   2   1   0   0|2628k    0 : 808k    0 :1620k    0 | 352k  798k|   0     0 |1835  1605

And you get the details, for each block device.

- KB

Original post.

davblog - Dave Cross: Thunderbird and Exchange

Fri, 12 Jun 2009 12:50:51 +0000

This morning a friend was complaining on Twitter about using Evolution on Linux to talk to his company's Exchange server. Evolution is the default answer to the question "how do I talk to Exchange from Linux", but my past experience has been much like my friend's - it's really not a very good answer.

I suggested Thunderbird to him. In my opinion, Thunderbird is the best email program out there. It's been my email program of choice for several years. It will happily work with Exchange to receive and send email. Googling for "thunderbird exchange" will bring back lots of useful results.

But Exchange isn't just about email. There's a whole calendaring system there too. Evolution supports that, but (by default) Thunderbird doesn't. There is, however, a way to get your Exchange calendar into Thunderbird using a rather circuitous route. Here's how I do it.

Firstly, you need the Lightning add-on for Thunderbird. Lightning adds calendar features to Thunderbird. You can create events and get alerts when they are about to happen. You can even subscribe to external calendars as long as they are in a standard format. Unfortunately, Exchange calendars aren't in standard formats. So we need some kind of intermediary.

The intermediary I use is Google Calendar. In fact I use Google Calendar as my definitive calendar. Every other calendar application I use reads from or writes to my Google Calendar. And Thunderbird (or, rather, Lightning) is one of the applications that interacts with it. Google Calendar writes calendars in the correct standard format, so Lightning will read a Google Calendar out of the box. But we can get cleverer than that using another Thunderbird add-on called Provider. Once Provider is installed, communication between Lightning and Google Calendar becomes two-way. I can add events either in Lightning or in Google Calendar and they will turn up in both.

There's one final step. We need to synchronise our Exchange calendar with Google Calendar. And Google have a product that does just that. It's called Google Calendar Sync. With this installed, your Exchange calendar is automatically synchronised with Google Calendar regularly. So now we can edit our calendar anywhere and the new or updated events will show up in all of our calendars. I've even noticed that invitations to events from other Exchange users show up in Lightning - but I haven't tried replying from there yet.

There are two things I don't like about Google Calendar Sync. Firstly, it has to be running on a PC running Windows which is connected to your Exchange Server. So it's not a solution that will work whilst you're (for example) out of the office with your office PC switched off. Secondly, it will only sync with your main Google Calendar. I would have liked to have a separate calendar for work events (and it's only work events that come from my Exchange calendar), but that doesn't seem to be supported yet.

And there are a couple of caveats with Lightning and Provider. If you're using a Beta test version of Thunderbird 3 then the standard Lightning and Provider downloads don't work with it. There are nightly builds of them both available, and the version of Lightning that I tried worked fine but Provider still didn't seem to work. I expect that situation to change quickly over the next few weeks as the Thunderbird 3 launch gets closer.

A year ago I was really disorganised. I never knew what I was supposed to be doing. Settling on Google Calendar as a definitive place to plan my life was a really good idea. At least now, I know which meetings I'm missing.

Rev. Simon Rumble: Why display advertising sucks

Thu, 11 Jun 2009 06:20:20 +0000

I've been working on a fair bit of marketing-related stuff over the last few years, and we've been spending pretty big on online campaigns. Much of this money has gone on display advertising, the kind of stuff you'll see on Fairfax and News Corp sites. We've had a whole range of problems with this stuff, and I've come to the conclusion it's more trouble than it's worth. Certainly if you want to measure response based on sales, the rate is pitiful. Here's why it's broken.

When you buy a search ad, you can be pretty certain the person is actually interested in the topic represented by the keyword you're buying. So when someone types "wireless keyboard", it's a good bet they're probably interested in buying a "wireless keyboard" of some sort.

When you buy a display ad, you get pretty loose categorisation. Perhaps your technology product might end up in the technology section of the site, so at least you're being exposed to people with an interest. Or you might be in the "general interest" pool, in which case you're getting exposed to the people who clicked on "celebrity shows boobies" links. Just the people you want. But regardless of the type of category you end up in, you're getting people who are expressly there for something other than finding something about a very specific topic.

So really, you can't measure the results of display advertising by expecting people to buy immediately after clicking, it's more for branding. Or so the salesmen for these mass media properties will tell you. So really you're getting your brand exposed to roughly categorised people.

Except it's completely unmeasurable, and the mass media sites have only themselves to blame for this. You see, if I want to have my brand exposed randomly to roughly categorised people, I need to have some pretty solid statistics on how many have seen it. I'm an Internet advertiser, so I'm used to pretty solid stats, not based on diary entries like TV or circulation surveys like newspapers (yeah, like most copies of the newspaper get read by more than one person, every day).

The problem is the major media in Australia specifically make any "impression" numbers meaningless, by adding a little line of code like this to every page on their site, this from news.com.au:

<meta http-equiv="refresh" content="0300" />

This line of code means the page gets reloaded every 300 seconds, or 5 minutes. Fairfax have recently started doing this in a slightly different way (checking if a video is playing first) but they do the same thing.

By reloading the page every five minutes, they've made their impression numbers completely meaningless. People regularly leave their browsers open with a page: the news, the weather, whatever they were just reading. So they go off for a half hour lunch and three "impressions" tick over, except there's nobody home to be impressed. There's no way to quantify how often this happens, so the impression number isn't just inaccurate, it's some unknowable amount completely wrong.

So you see, measured by the only accurate measurement, click throughs and subsequent sales, display doesn't make sense. And as a branding exercise it's not worth playing around with because the publishers have specifically taken steps to make the numbers meaningless.

And they wonder why newspapers are dying, when they go out of their way to devalue the one thing of value they do produce?

11th June 2009 addition: this "study" by News Digital Media just confirms my suspicions. Surveying less than a thousand people when the ad had 300,000 impressions, yeah there's something you can use to draw conclusions! These being people who had just seen the ad (possibly, given what I said above).

Contact me

www.DavidPashley.com/blog: Table sizes in PostgreSQL

Wed, 10 Jun 2009 09:42:45 +0000

Ever wanted to find out how much diskspace each table was taking in a database? Here's how:

database=# SELECT 
   tablename, 
   pg_size_pretty(pg_relation_size(tablename)) AS table_size, 
   pg_size_pretty(pg_total_relation_size(tablename)) AS total_table_size 
FROM 
   pg_tables 
WHERE 
   schemaname = 'public';
 tablename  | table_size | total_table_size 
------------+------------+------------------
 deferrals  | 205 MB     | 486 MB
 errors     | 58 MB      | 137 MB
 deliveries | 2646 MB    | 10096 MB
 queue      | 7464 kB    | 22 MB
 unknown    | 797 MB     | 2644 MB
 messages   | 1933 MB    | 6100 MB
 rejects    | 25 GB      | 75 GB
(7 rows)

Table size is the size for the current data. Total table size includes indexes and data that is too large to fix in the main table store (things like large BLOB fields). You can find more information in the PostgreSQL manual.

Edit: changed to use pg_size_pretty(), which I thought existed, but couldn't find in the docs. Brett Parker reminded me it did exist after all and I wasn't just imagining it.

Read Comments (1)

Rev. Simon Rumble: Moving into the cloud

Tue, 09 Jun 2009 07:37:37 +0000

For the last five years I've run a virtual server from the excellent Linode which has hosted a number of services including this web site, a Jabber server, some other web sites, and my email. Initially I did spam filtering myself, but between the load this placed on the server and the constant tweaking required, it annoyed the hell out of me, so I outsourced that to JunkEmailFilter.com, which while imperfect was passable.

I've used mutt as my email client since some time in the later 90s, after upgrading from pine. The advantages of a text-mode mail client were speed, accessibility from nearly anywhere, integrated tools (gpg, lbbdb) and the fact that a clueless boss looking over your shoulder will think "techie stuff, must be work" and hence not require me to justify my work throughput.

More recently I've noticed myself regularly bouncing messages to my secret gmail account. Partly this is because Google Reader produces broken text attachments, and a few of my correspondents insist on using that, but mainly it's because more and more emails included stuff I couldn't use in a text-mode client, like links to complicated web site, pictures, or links to videos.

So I thought I'd try out moving all my email to Google excellent web client. The process of moving the whole domain was surprisingly easy. Just point the MX at Google's servers, set up the appropriate accounts and define the pass-through mail server for anything that doesn't have an account on the Google servers. This means the mailing lists and weird aliases I have on my server continue working as before.

The hardest part was uploading all my old mail. I've been saving mails into their own Maildirs based on the big before the @ in the email address, so john@smith.com ends up saved as john/ and if there's any doubling up of "john" it's still all in there. Gmail has a pretty good (and very fast) search, so filing becomes much less important.

There's a bunch of tools you can find that claim to be able to upload Maildirs to Gmail. I couldn't get any of them to reliably work. Eventually I downloaded the corpus of my Maildirs to a desktop and used Thunderbird to upload them all to the new location. It took ages, but it worked.

So far I'm pretty damn happy with this. I haven't had a single spam get through, and even the ones that make it to the Spam directory are pretty minimal. The add-on Postini spam filtering service has done two false positives on spam so far, which I consider pretty good and I presume it'll learn.

Now that I'm paying US$50/year for each email account (one for me, one for Holly) it's costing about the same as my former JunkEmailFilter.com and rsync.net accounts. I'm also going to downsize my Linode which will also save me some money. More importantly, I'm doing even less maintenance, and the search engine on my email is vastly superior to mutt. It's also a graphical email client, which is nice, and the mobile clients are really quite good.

Quite happy with it.

Contact me