Hit Him Again, Luke

I am but a few discs away from completing my once-every-couple-of-years rewatching Gilmore Girls. It’s at the point where I just want to throw rocks at Christopher and Logan whilst scratching my head at the personality transplant they gave Marty. Oh, Season 7, you were such a mistake. But at least you ended properly.

Next week, I’ll be heading back to RDU once again. But this time, heading back to LA, not for Santa Monica-related shenanigans, but just as a stopover on the way for a week in Hawaii with the family!

It's 3am, Is Your EventMachine Selecting?

A tip for those of you using EventMachine in Ruby. As you may know, by default EventMachine uses the select() system call during its run through the event loop to check for new inputs on file descriptors (which it then uses to hand off to callbacks you may have registered). This is useful, as you’ll find select() on pretty much any UNIX-based system you can care to name. However, there are a few drawbacks:

  • select() is often limited to FD_SETSIZE file handles, which is normally 1024.
  • Because of the way _select()_ runs, EventMachine needs to loop through all the file descriptors twice, once for passing into select(), and once again to see if it has marked a descriptor ready for reading or writing.

This is often fine during developing, and perhaps even testing (especially if you’re not load-testing properly), but you may be in for a surprise when you start getting serious traffic in production.

Thankfully, EventMachine comes with a few strategies to get around this issue. Sadly, it’s not quite as simple as select(), but not too taxing. Both Linux and BSD-derived systems have taken different approaches - Linux provides a system called epoll, while BSD systems have kqueue. Both implementations eliminate the idea of having to read through all file descriptors twice on every call, and are easier to scale past the 1024-descriptor limit. All you have to do is call EM.epoll or EM.kqueue before you start the Reactor.

Having said that, you’ll see a lot of code around the net that looks a bit like this:


EM.run do 
    EM.epoll
    ...
    ...
end

There’s a slight problem here. It should look like this:


EM.epoll
EM.run do
    …
    …
end

There’s not much difference there, but the ordering of the lines is incredibly important - because EM.epoll in the first example is inside the Reactor, it will do nothing and will instead fall back to select(). And your code will blow up when it hits a big traffic spike. So be careful out there with rogue function snippets, and always set up with kqueue or epoll before you make the fateful call to EventMachine.run.

The Phantom Bedbugs

Most of this week was spent lying awake at night feeling all sorts of bugs crawling over me. These are the annoying phantom bugs that you somehow know are there, but can’t actually see or feel. And while they don’t exist, it actually doesn’t help at 2am when you turn on the light and rifle through the bedclothes trying to find any evidence of the things you just know are there.

It started when I saw the doctor on Tuesday. I have psoriasis, and before/during my trip to Boston, it seemed to be flaring up pretty badly, and even spreading to my legs, though spreading by drying out spot-like things instead of the normal scales like I find on the back of my head. I thought a trip to the doctor’s would be useful.

She ruled out spreading psoriasis pretty early on, and instead suggested that they could be bedbug bites. My knowledge of bedbugs is really just limited to seeing BBC reports on the infestations in places like New York, so I was somewhat horrified, as I had dim recollections of people having to throw out their mattresses, beds, clothes, and all sorts to other things to try and stem the relentless hordes of bugs.

Needless to say, I itched all the way back to work.

When I got back home, armed with Google Image search, I stripped the bed down to the mattress and went bug-hunting. And of course, I couldn’t find anything, but that didn’t mean they weren’t there. So sleep was somewhat harder Tuesday night.

As I couldn’t find anything, but I did want to sleep again sometime, I called Triangle Pest Control, who offer a free bedbug inspection, as well as impressive heat treatments where they heat your house up to 60˚C and KILL EVERYTHING IN SIGHT. The head of the company came around Friday afternoon, told me stories about their bedbug-hunting dogs, horror stories about a certain fast-food franchise in the Durham area, and then pulled up the mattress and the bed to confirm that he couldn’t find anything either.

So, it looks like it isn’t bedbugs. So, not entirely sure why my legs are the way that they are, but we can at least eliminate them. Hurrah!

EVERYTHING IS AWESOME!

It’s been a very tough week, for reasons that I can’t go into here right now. However, you should go see this, as it brightens up even a very dark week:

dd/mm/yyyy

It has begun. Slowly, insidiously, and completely unexpectedly. And I was betrayed by Kinder Eggs, of all things. I have a pack sitting by my table which are waiting to be picked up by a friend. As I’ve had them since Christmas, I wanted to make sure that they were still good, so I flipped the box over and looked at the expiry date:

01/09/2014

For a moment, I thought ‘oh no, they’re out of date’. For a moment. One second later, my brain reasserted itself and I realised it was saying September, not January. A small slip, but a worrying one. I think I over-compensated and refused to give out any temperatures in Fahrenheit for the rest of the day. And had three cups of tea. You can’t be too careful.

I accidentally walked 7.5 miles yesterday. It was sunny! And I hadn’t been about Durham for ages! The blisters I have on my big toe today were totally worth it! Well, maybe not that last one. I did a loop from my house down to Whole Foods and 9th Street and came back home via Trinity, Geer, and Gilbert. You can argue against the evils of gentrification until the cows go home and make tea for two (no beef, please), but it’s amazing how active Geer & Rigsbee is these days, not just with the bars and restaurants there, but the park just behind Cocoa Cinnamon, which almost always has children playing football at the weekend (and yes, I’m talking about real football here, so bonus points allocated), people milling about enjoying the sun, and the feeling that the pedestrian is safe to walk wherever (despite it actually being a road…I guess that could potentially be a problem!).

And then on the way back, I fell in love with another building. I can’t believe I’d never seen it before, but down on Gilbert is the former Hosiery Mills Dye House. From the outside, it looks amazing, a 1920s warehouse with great fronts and features. Once you look at the OpenDurham link, though, the inside is even better - look at the wooden trusses! LOOK AT THEM! OpenDurham says that the Durham Food Bank is currently situated there, but it looked empty yesterday, with a ‘for lease’ sign out front, and the Durham Food Bank website says that they’re moving. If anybody fancies a renovation on the lines of the Haw River Ballroom but in East Durham, they should come this way.

The Apples Were Quite Cold

I did have grand plans. I was going to go into Boston, visit all sorts of hipster places (there’s a tapas restaurant that is also a record and book shop, for crying out loud), take lots of pictures of my favourite building style, and generally enjoy my time in the area.

Except: the snow (and other, non-weather related concerns that meant that I didn’t feel much like going on big adventures during the night). Trying to venture through three feet or so of snow really wasn’t my idea of fun, so in the end I only made it into Boston on one day. A taxi ride to Alewife Station, and then the Red Line into Harvard Square, where it took me an embarrassing amount of time to rediscover Newbury Comics. Which seemed much smaller than I remember (admittedly, it is almost twenty years since I first went there pause for feeling old). There I wandered around trying to warm up, getting in people’s way, awkwardly responding to the conversation of the woman behind the counter, and looking at the glitter. So aside from the size, it was almost exactly like the time I went in 1997. Oho!

(I walked along thinking I wasn’t going to get anything. Then I saw the Criterion section and thought to myself “well, if they have a copy of Hard Boiled, I’ll get it, knowing that it’s fairly rare these days. And lo, they had one. And yes, it then went to the UK, then back to Boston, and only now back to Durham. A frequent flyer DVD)

I went to books, finding myself coming out of Park Street Station and knowing exactly where I was, looking at a CEX shop front that once used to be HMV in one direction, and Boston Common in the other. I walked past the giant Citgo sign as I walked from the centre of the city to Mei Mei. It was…further than I expected. And colder, despite the gloves and the hat and the scarf.

Finally, I did my Good Will Hunting bit, at the Au Bon Pain at Harvard Square. Except the snow covered all of the outside seating where the film scenes were shot, so I had tomato soup while watching two girls play Settlers of Catan.

It was an odd few weeks; I spent most of it trapped inside an office building, the (actually quite nice) Holiday Inn Express at Waltham, or on a plane somewhere. Hopefully when I go back in May I’ll have a chance for a little more exploring. And that the snow will have finally melted.

Alewife Station

dd/mm/yyyy

Goodbye Wall

The End of the Green Wall

The Green Wall has felt like it has been part of Durham forever. Forever only stretches back to 2005 when Greenfire purchased the burnt-out remains of 120122 W Main Street and painted it green in an act of hubris that come to haunt them in 2008, but still, ten years is plenty in a city that has transformed radically in that time.

It’s been home to many things, as detailed in the tweet above, but it was also just a lovely quiet space in the middle of the city where you could find a patch of grass, walk, have a pleasant lunch in the sunshine, or just have a sit-down. Something that I feel every city needs.

To be fair, the wall was rotting, the buildings it enclosed had been burnt out since 2001 and the whole thing could have collapsed in the face of a gale force wind. I’m not the hugest fan of the plans to build a massive 26-storey office/retail/condo block on the parkland and two storefronts deep into the block, but maybe it won’t be as bad as it sounds.

Not everything can be saved. We had ten years of the Green Wall which probably would have been three or four if not for the Great Crash of 2008. Let’s be sad about the event, but be thankful for the memories we made.

It's 3am, Do You Know Where Your Certs Are?

So, things are trundling along on your Rails project. You’re doing things properly and have lots of tests, which get kicked off by a Jenkins CI server every time a developer pushes to the repository. Then, it comes time to replace your SSL certificate on the domain you use for testing. Suddenly, your tests start breaking hard with this error when they try doing fun things with your SSL-enabled API:

OpenSSL::SSL::SSLError
SSL_connect returned=1 errno=0 state=SSLv3 
read server certificate B: certificate verify failed

And yet, when you open a pry or irb session, you can connect to the URL in question fine and dandy. What’s going on?

Welcome to the wonderful world of OpenSSL::SSL::VERIFY_NONE. The Net/HTTP code in Ruby can be passed an SSLContext for setting up SSL connections. Unfortunately, the default verification step in a new SSLContext is OpenSSL::SSL::VERIFY_NONE, which as you can imagine from the name, does no checking whatsoever. So that explains why your pry or irb session is connecting fine.

But why is it failing in the tests? Well, chances are that you’re using something like HTTPClient for your REST calls, aren’t you? And these gems, being somewhat better behaved, set their SSL context to be OpenSSL::SSL::VERIFY_PEER, to at least check the chain of certificates. And that’s what’s failing - Ruby can’t complete the certificate chain.

Which leads to another question - you’ve paid out the money for the SSL cert, it comes from a reputable provider (ours was from RapidSSL), so why isn’t it verifying? Well, it turns out that as well as setting the default check to VERIFY_NONE, Ruby also doesn’t load any of the cert files in the OpenSSL gem; instead it just relies on the OS’s certs. Which is probably fine in most cases, but as it turns out, not if you’re running Ubuntu 12.04 and using a RapidSSL cert.

How do we fix this? Well, you could monkey patch either OpenSSL and/or Net/HTTP to ignore any options and always use OpenSSL::SSL::VERIFY_NONE. Please don’t do this. Ruby using it by default is a bad thing, leaving you open to MITM attack; HTTPClient is doing the right thing. What we really want to is tell Ruby to load its cert store up with with the certs both in the OS and the supplied OpenSSL gem (which includes the latest root GeoTrust cert for RapidSSL). Luckily, this is fairly simple once you know how:


http_client = HTTPClient.new
http_client.ssl_config.clear_cert_store
http_client.ssl_config.cert_store.set_default_paths

Magically, the RapidSSL cert now gets accepted and the Jenkins tests can go off and go their merry thing, hopefully preventing panic as the builds return to passing.

The Chocolate Gods

About a month ago, I spent all weekend making chocolates for sale over at Fallout Durham. As a result of this, I was somewhat burnt out on the whole confectioner thing for a little while, so I took Christmas off (okay, I did make a chocolate tart while I was at home, but come on, it took all of about ten minutes to make the ganache and pour it into the pâte sucrée base I had made the day before). And then when I got back, I got sucked into the Fortnight of Fun at work which sucked up my evenings as well as my days. This weekend was going to be my return to the realm of sweets and cocoa butter emulsions.

But chocolate is a fickle mistress, and seems to resent the fact that I abandoned her for a month. I planned to make something this morning, though I wasn’t really sure what until I lifted the temperer from its heavy moorings and took account of my current chocolate stocks. To ease myself in, what could be easier than a large bar of milk chocolate mixed with orange essential oils and passed through a cream whipper to create aero bubbles?

As it turns out, the answer to that question is: a lot. I’m still at a loss to what happened: maybe the new whipper I used wasn’t full enough, or the chocolate gods decided to get their revenge, but the chocolate came out of the whipper in pretty much the same way it went in, with no trace of the two nitrous oxide cartridges I fed into it. Bah. So I ended up with a small block of orange milk chocolate. Which is not the worst thing in the world, but not exactly what I planned.

Still, I ended up spending $300 on chocolate afterwards, so maybe that sacrifice will appease the chocolate masters? I need some inspiration for next weekend, too…

Apart from that, it’s been another quiet week. Well, that’s a bit of a lie; it’s been a similar week to the previous one, where I’ve only had a few moments at the end of the day to rest before getting up and starting all over again, but hopefully that shouldn’t be the case in this coming week. Which isn’t to say that I’ll be doing anything interesting, but I won’t be working all hours!