Webmachine as an application front-end

Webmachine 1.4 is pushed to bitbucket, and is providing the HTTP face for a few interesting new software systems.

The very cool guys at Collecta have built a different sort of search engine, great for watching the flow of social-network sorts of conversation occurring about whatever topics you are interested in.  The REST API powering that engine is written in Webmachine, and takes advantage of some of Webmachine’s more interesting features while interacting with other components.  I’ll leave more detailed explanation of their technology to their excellent team and just say that they’ve built something very deserving of attention.

We have also used Webmachine to provide the HTTP interface to our decentralized document store: Riak.  Webmachine’s ability to provide the full richness of HTTP’s capabilities while not dictating anything else about the shape of your application makes it a natural fit for the front end of a system like Riak.  The first incarnations of Riak and Webmachine were each built at about the same time: late 2007.  As they grew up together there were some internal interface design decisions that were more obvious to us as a result.  The structure of Riak’s core operations map nicely to the universal interface that includes GET, PUT, POST, and DELETE not just by name but by their essential properties such as idempotency, safety, and defined semantics.

One aspect of the Web that has been essential to its success as the biggest distributed system in the world is the notion of links.  This is also one of the things that differentiates Riak from some other data storage systems.  It’s not exactly a graph database, but it adds some elements of a graph database on top of the great benefits that come with having a decentralized key/value store at the core.  Documents in a Riak cluster can have links to other documents in that cluster.  Riak itself can take great advantage of this internally, as the MapReduce programming model that we use is ideally suited to walking links in order to build up inputs for the next phase of computation.

These links ought to also be useful to clients, and in the context of HTTP this should be possible in a way that does not assume application knowledge on the part of a client.  To that end, the newest release of Riak includes support for the Link header in HTTP responses.  This allows clients to explore the link structure of a set of related documents without having to read or understand the body of those documents.  Based on our past experience building applications atop Webmachine and Riak, we expect this to be an added bonus for rapid development.

Comments

Guys from Collecta it is valid good fellows, have created really convenient system

Posted by Aleks on 06 Sep 2009 at 11:02

Add comment