Monthly Archives: July 2009

First coffeeshop demo screencast

To demonstrate some of the basic coffeeshop features, I put together a screencast, and after getting over the shock of hearing my own voice in the recorded voiceover (last time I chickened out and just typed what I wanted to say), I put it up on YouTube:

http://www.youtube.com/watch?v=1E_1B8TD6Kw

The screencast shows the creation of a channel, the addition of a couple of subscribers to that channel, the publishing of a message to that channel, and the subsequent delivery of that message to the subscribers. I draw attention to the use of the browser-based part of the implementation, and to the asynchronous nature of the message distribution (I had to do this anyway, as on the App Engine SDK development server, tasks are not executed automatically — you have to start them manually in the admin console).

‘Coffeeshop’ – lightweight HTTP-based pubsub

Coffeeshop‘ is a lightweight, REST-orientated HTTP-based publish/subscribe implementation that I’ve been working on for the last few days. It is a culmination of:

  • an early and long-standing interest in pubsub
  • a fascination with using HTTP properly, i.e. as an application protocol, not a transport protocol
  • an excuse to experiment in the area of webhooks
  • a desire to learn more about Google’s App Engine as a cloud platform, and more specifically as an HTTP runtime platform, much like SAP’s Internet Communication Framework (ICF)

There seems to be a growing interest in pubsub and webhooks; one recent article in particular – “The Pushbutton Web: Realtime Becomes Real” conveys a lot of the ideas behind these concepts.

With coffeeshop, entities — Channels, Subscribers and Messages — are resources, with URLs. You interact with entities using the appropriate HTTP methods. The implementation, being HTTP, is both browser-based (human-facing), and agent-based (program-facing). You can navigate the resources with your web-browser. You can interact with the resources with cURL, POST, or your favourite HTTP library.

Here’s a simple example:

> # Create Channel:
> echo "Test Channel" | POST -Se http://giant:8888/channel/
POST http://giant:8888/channel/ --> 201 Created
Location: /channel/5/

> # Add to Channel a new Subscriber with
> # a callback resource of http://atom:8081/subscriber/alpha
> echo "name=alpha&resource=http://giant:8081/subscriber/alpha" 
>  | POST -Se http://giant:8888/channel/1/subscriber/
POST http://giant:8888/channel/1/subscriber/ --> 201 Created
Location: /channel/1/subscriber/2/

> # Publish a Message to the Channel
> echo "hello, world" | POST -Se http://giant:8888/channel/1/
POST http://giant:8888/channel/1/ --> 302 Moved Temporarily
Location: http://giant:8888/channel/1/message/ahFxbWFjcm8tY29mZmVlc[...]RgIDA

As you can see from this example, POSTing to the Channel container resource

/channel/

creates a new Channel, POSTing to the Channel 1 subscriber container resource

/channel/1/subscriber/

creates a new Subscriber, and POSTing to the Channel 1 channel resource

/channel/1/

creates a new Message, which is delivered to the Channel’s Subscribers. The resource returned to a Message POST is that Message’s unique address

/channel/1/message/ahFxbWFjcm8tY29mZmVlc[...]RgIDA

where the details of that Message, including the Delivery status(es), can be seen.

For more information on the resources and methods, have a look at the ResourcePlan page.

I’m using Google App Engine’s Task Queue API to have the Messages delivered to their (webhook-alike) endpoints asynchronously.

The code is early and rough, and available on github. You can download it and try it out for yourself locally or create an app on App Engine cloud domain appspot.com. I’ll probably publish a public-facing instance of this implementation in the next few days. All comments and feedback appreciated.

One last thing: I know of at least a couple of HTTP-based pubsub implementations: pubsubhubbub, and Watercoolr. Both are great, but for me, the former is a little complex (and ATOM-orientated), whereas the latter I thought could be more RESTian in its approach (hence coffeeshop).