It's 3am, Is Your EventMachine Selecting?

(eventmachine, ruby, kqueue, epoll, select)

A tip for those of you using EventMachine in Ruby. As you may know, by default EventMachine uses the select() system call during its run through the event loop to check for new inputs on file descriptors (which it then uses to hand off to callbacks you may have registered). This is useful, as you’ll find select() on pretty much any UNIX-based system you can care to name. However, there are a few drawbacks:

  • select() is often limited to FD_SETSIZE file handles, which is normally 1024.
  • Because of the way _select()_ runs, EventMachine needs to loop through all the file descriptors twice, once for passing into select(), and once again to see if it has marked a descriptor ready for reading or writing.

This is often fine during developing, and perhaps even testing (especially if you’re not load-testing properly), but you may be in for a surprise when you start getting serious traffic in production.

Thankfully, EventMachine comes with a few strategies to get around this issue. Sadly, it’s not quite as simple as select(), but not too taxing. Both Linux and BSD-derived systems have taken different approaches - Linux provides a system called epoll, while BSD systems have kqueue. Both implementations eliminate the idea of having to read through all file descriptors twice on every call, and are easier to scale past the 1024-descriptor limit. All you have to do is call EM.epoll or EM.kqueue before you start the Reactor.

Having said that, you’ll see a lot of code around the net that looks a bit like this:


EM.run do 
	EM.epoll
	...
	...
end

There’s a slight problem here. It should look like this:


EM.epoll
EM.run do 
	...
	...
end

There’s not much difference there, but the ordering of the lines is incredibly important - because EM.epoll in the first example is inside the Reactor, it will do nothing and will instead fall back to select(). And your code will blow up when it hits a big traffic spike. So be careful out there with rogue function snippets, and always set up with kqueue or epoll before you make the fateful call to EventMachine.run.

blog comments powered by Disqus