Technology « Bill Shupp

7May/110

Getting started with Kestrel from a PHP application

We've been using Twitter's kestrel queue server for a while now at work, but only from our service layer, which is written in python. Now that we have some queueing needs from our application layer, written in PHP, I spent a few days this week adding queue support to our web application. I thought I'd share what I learned, and how I implemented it.

Goals

The kestrel server itself was pretty straightforward to get up and running. The only thing I would point out is that I recommend sticking to release branches, as master was fairly unstable when I tried to use it. Regarding implementing the client, there were a few goals I had in mind when I started:

Since kestrel is built on the memcache protocol, try and leverage an existing memcache client rather than build one from scratch
Utilize our existing batch job infrastructure, which I covered previously here, and make sure our multi-tenant needs are met
Keep the queue interface generic in case we change queue servers later
Utilize existing kestrel management tools, only build out the the functionality we need

With these goals in mind, I ended up with 4 components: a kestrel client, a producer, a consumer, and a very small CLI harness for running the consumer. But before I even coded anything, I set up kestrel web, a web UI for kestrel written by my co-worker Matt Erkkila. Kestrel web allows you to view statistics on kestrel, manage queues, as well as sort and filter queues based on manual inputs. Having this tool up and running from the get go made it easy to watch jobs get added and consumed from my test queue, and also easily flush out the queues as needed.

Filed under: Code, Kestrel, PHP, Technology, Zend Framework Continue reading

21Mar/110

Preparing your application for serving static content efficiently

(Reposted from the Empower Campaigns blog)

Most web applications have static content such as CSS, JS, images, etc. To save on bandwidth costs and improve load times for your users, it's a good idea to tell the client to cache these items for a long period of time. But even though your static content changes infrequently, when it does, the cached content needs to be invalidated. In addition, high traffic sites often want to go a step further and take advantage of a CDN to offload static content delivery. Below I'll share the method we used here at Empower Campaigns to accomplish both of these things.