Enlightenment though eventlet

I did a presentation on Tornado Web to the local Python user group a couple weeks ago. There is one annoying thing about Tornado and one show stopper for most development. Tornado depends on pycurl for doing client request which is pretty strange since the core module can do client and server sockets. For any nonblocking to work with Tornado it has to have an implementation using the core Tornado library. So out goes all legacy clients that use Python sockets (very few use nonblocking sockets).

Blinded By Idealism
My first impression was that monkey patching the standard library was ugly. In reality this makes eventlet usable for legacy projects as well as very simple to write backward libraries that support both blocking and nonblocking. I was caught in the idealism trap where I wanted a clean paradigm to exist though the entire stack. Re-implementing all socket libraries is to large a barrier to change mindshare that nonblocking is a better programming paradigm. Especially if the new implementation is dependent on a non standard library like Tornado, Twisted ect.

Eventlet
The PyCon 2010 presentation is excellent overview. Bottom line eventlet provide a synchronous interface to nonblocking io or can just write nonblocking io using coroutines. Eventlet is mature, stable, fast and documented. This makes it very usable right now using existing libraries and the benefit of the improved paradigm/performance can be taken advantage of without much code change. Also as I have had more time to use Go and it concurrency model I am realizing the power of coroutines in any language.

In Practice
This all came about because I needed to write something that would traverse a site and validate the responses based on content type and regexes. Things like correct encoding to making sure the proper data is being displayed on a specific url. So a spider with validators that can be hooked in based on a regex or content type. Writing the test code I was able to create a eventlet server and have it output data so that I could test that the full stack of the spider code.

Yet again I learn though my mistakes.