Stephan Schmidt had already written Facebook BigPipe for Java, but using a synchronous servlet model, not asynchronous. I decided to implement it using Jetty continuations and the Jetty HTTP client, but the code should be easy to adapt to servlet 3.0 AsyncContext.

The code initially constructs the page with a few empty divs that will contain the pagelets, for example:

<div id='pagelet3'></div>

I keep the connection open from the browser to the server while I render pagelets. As pagelets become available, I flush them onto the browser, which in turn inserts them into the right place in the DOM using Javascript. Assuming we get back a response from the remote module protocol:

<span>some useful message for module: pagelet3</span>

We wrap it inside Javascript so that it looks like:

<script>arrived('pagelet3', '<span>some useful message for module: pagelet3</span>');</script>

All is left is for the arrived function to update the pagelet3 element in the DOM.

In terms of concurrency, the initial page construction is done synchronously. As soon as I have the frame in place with the divs that will eventually hold the pagelets, I suspend the execution of the servlet, which in turn releases the thread that was attached to the client connection. At that point, I fire in parallel HTTP client requests to the remote module server.

For resuming the continuation, I could have used a counter, but since each HTTP client execution is a separate thread, unlike in node.js, I did not want to have to acquire a lock and synchronize to be able to update the counter. Instead, I used request attributes, as setting, getting and removing them is thread safe. The code keeps a list of references to the elements that we are offloading to a remote module server, as an ArrayList with the ids to fire parallel requests to http://localhost:8080/module?id={id}; each id is also kept as a request attribute. As each request comes back from the remote module server, I write and flush the response buffer, so that the element appears on the browser immediately, and I remove the corresponding request attribute.

Although the code is not as readable and straight forward as what Subbu got with node.js, I am actually surprised how simple the async Java solution actually turned out to be.

We have an “admin” node process which is started with as input the number of “feedly” node processes it should launch and monitor. When the admin starts, it spawns multiple “feedly” node processes, padding as input the port the feedly node should listen to 9701 for the first one,…9710 for the 10th one.

On the server we have a varnish server which load balances the traffic from m.feedly.com to 127.0.0.1:9701 — 127.0.0.1:9710.

Each “feedly” node process has a 127.0.0.1:970x/heath endpoint where it reports stats about its execution. The admin node monitor each feedly node every 2 minutes and makes sure that the feedly node is healthy and responding. If not, it kills the child and restart a new instance of the feedly node listening to the same port.

All feedly nodes point to a redis instance for shared memory/session management. This allows incoming HTTP requests to be load balances across any feedly nodes transparently. In dev/staging, we have varnish and all the node processes running on the same box.

In production, we can scale out by having varnish on one server and multiple node cell (where a cell is an admin+10 feedly). This allows us to do rolling upgrades, etc. without any interruption to the service.

Functional programming languages have long understood this, and designed mechanisms to avoid the risk of a programmer getting it wrong, such as immutable data structures (take a look at java.lang.String, designed by Lee Boynton, a true Lisp guy), or message passing (erlang).

Surprisingly, another line of programming has ended up in the same place. PHP can be deployed multi-threaded on Apache, but unfortunately most native libraries are not thread safe, so one ends up running processes as workers. Similarly, Python has the GIL, and Guido has no interest in fixing that. And yet, PHP and Python scale really well using processes.

So, here’s the interesting bit about node.js. It’s written on a functional programming language, Javascript, and provides a reactor server (remember POSA2?) on a single thread! I hear you say, how does it scale to multiple cores? It does not.

A single thread is only able to utilize one CPU. Even though most web apps are initially I/O bound, at some point programmers somehow manage to make them CPU bound (such as marshaling and wrangling JSON and XML). This will limit the scalability on the single process. In order to scale beyond 1 core, two possibilities exist. Right now, one would need to launch one node.js process per core, and then setup a load balancer. In the future, launching the processes needs to be done inside node (maybe this is fixed in the future?). The second option, which might also be implemented based on the node.js web site, is to adopt the Web Workers spec.

Now, because of the (single) threading model, response times are fairly unpredictable. One would expect a Java server on an async stack, such as Jetty, to provide more consistent response times, as it uses multiple threads. A test corroborates such hypothesis (look at the curve distribution for node.js vs jetty).

At the same time, because of the (single) threading model, memory usage is very low in node.js, compared with Netty, or Jetty, where as a result of being multi-threaded, memory usage will be higher for the same load and latency. A discussion on ycombinator seems to be along these lines. I am of the opinion that being able to run both on low and large memory profiles is critical to becoming successful, especially as many developers run their code in shared hosts.

If you have followed me this far, putting memory aside as that can be shared on the heap, you will then conclude that a multi-threaded Java-based async I/O server, like Jetty, should be better that node.js. And I would agree, only if it was not Java.

So here’s the deal. Except a counted number of exceptions, Java libraries carry significant legacy of the wrong kind of concurrent programming, either abusing or ignoring locks. Either they do not scale to multiple cores, or they corrupt integrity of state. And this will continue to be a problem, since Java does not prevent bad programming practices in a concurrent setting. But there is a light: using the JVM, ignoring Java and most existing Java libraries.

Building new libraries for the JVM that are truly designed for concurrency requires using the right abstractions, and this is where functional programming comes into place. There are three key functional programming language contenders to the JVM: Javascript (Rhino), Scala, and Clojure.

First, there is Javascript, with Rhino, Mozilla’s Javascript implementation for the JVM. Ringo is a nodejs alternative based on Rhino. Unfortunately, Rhino is slow and quite a memory hog. Additionally calls in Ringo are blocking, which defeats the programming model. And Ringo uses the same Java libraries that were not coded with concurrency in mind. Compared to nodejs, Ringo has no community (a quick scan reveals 9701 messages in the node.js mailing list in 12 months of usage, against only 181 in 2 years of usage).

Next, come Scala and Clojure. Both are very intriguing, perhaps Clojure being more of my taste. For Clojure, we now have Aleph, a thin Netty wrapper. On multi-core, it seems to beat node.js latency and throughput wise, at the cost of memory. And Clojure allows message passing, immutable data structures, ala Lisp, all transparently mapped onto threads. Beautiful I say.

But Scala and Clojure are not Javascript. Javascript attracts many developers from the browser world, it provides a consistent event-driven programming paradigm for both server runtime and client runtime. Eventually, this pushes most UI logic to the browser, and leaves the server as a smart data source. As much as I find Clojure extremely exciting, I don’t think it can realistically compete with Javascript.

So, realistically, only node.js is a viable alternative today. Maybe Clojure truly succeeds, and a Clojure based async I/O server appears. But for now, node.js is more interesting.

But don’t get me wrong: node.js is very young, truly bleeding edge, and needs substantial work. Some of that work will make it easy for applications to scale beyond one core. Scaling beyond one core will improve consistency in response times. But given the vibrant community and excitement (and hype) around node.js, I am fairly confident this will happen.

But I believe we are finally at a turning point. Multi-core architectures imply the end of the free lunch. It is happening. Writing highly concurrent server-side applications that are able to scale linearly as the number of cores increases is something that we should be preparing for.

Once you believe reactor servers are the future, and somewhat the present, comes the question of programming language. When it comes to programming for the web, there are many religions and options: PHP, Ruby, Python, Java, C#, … However, when it comes to writing web applications, there are three languages on which all web developers agree universally on: HTML, CSS and Javascript.

And this is what makes node.js exciting. node.js develops a reactor server on top of Google’s V8 Javascript engine. First, it solves the free lunch problem using a language that is universally accessible to web developers. Second, it opens the door for innovation to a world of extremely high concurrency, for example, MMORPGs, and potentially to a completely different web interaction paradigm.

With motivation in mind, I set myself to learn a bit more about node.js. I chose one possible stack, node.js + express + mongodb, but there are many more to learn. What follows in the rest of this post are my raw notes, which I am posting here just in case it helps fellow developers getting up to speed in all this new cool technology.

Installation

First, we install node.js itself, and kiwi, a packaging system for node.js, using homebrew (I have dropped Macports in favor of homebrew, as it allows me to create my own installation recipes really easily).

$ brew install node
$ brew install kiwi

That installed for me:

kiwi (0.3.1)
node (0.1.98)

Once we have kiwi, we install the express framework:

$ kiwi -v install express

Finally, we install mongodb:

$ brew install mongodb

If this is your first install, automatically load on login with:
    cp /usr/local/Cellar/mongodb/1.4.3-x86_64/org.mongodb.mongod.plist ~/Library/LaunchAgents
    launchctl load -w ~/Library/LaunchAgents/org.mongodb.mongod.plist

If this is an upgrade and you already have the org.mongodb.mongod.plist loaded:
    launchctl unload -w ~/Library/LaunchAgents/org.mongodb.mongod.plist
    cp /usr/local/Cellar/mongodb/1.4.3-x86_64/org.mongodb.mongod.plist ~/Library/LaunchAgents
    launchctl load -w ~/Library/LaunchAgents/org.mongodb.mongod.plist

Or start it manually:
    mongod run --config /usr/local/Cellar/mongodb/1.4.3-x86_64/mongod.conf

And finally, the native adapter to mongodb for node.js:

$ kiwi install mongodb-native

Hello World

Fire text editor and create app.js:

var kiwi = require('kiwi')
kiwi.require('express')

get('/', function(){
  this.contentType('html')
  return '<h1>Welcome To Express</h1>'
})

run()

And back to the terminal:

$ node app.js

Point the browser to:

http://localhost:3000/

Tutorial App

I found a good blog with many node.js tutorials, including one on building a blog app example that included source code using express and mongodb.

To run the demo, first start mongodb, and then node.js (I used two terminal windows):

$ mongod run --config /usr/local/Cellar/mongodb/1.4.3-x86_64/mongod.conf
$ node app.js

And then visit http://localhost:3000/. I could not create a new blog post by visiting http://localhost:3000/blog/new, as I’d get an error:

/Users/brunofr/Projects/express-mongodb-2/app.js:39
        title: article.title,
                      ^
TypeError: Cannot read property 'title' of undefined
    at /Users/brunofr/Projects/express-mongodb-2/app.js:39:23

Looking at the code, it seems like the route blog/* is being executed before blog/new:

get('/blog/*', function(id){
  var self = this;
  articleProvider.findById(id, function(error, article) {
    self.render('blog_show.html.haml', {
      locals: {
        title: article.title,
        article:article
      }
    });
  });
});

get('/blog/new', function(){
  this.render('blog_new.html.haml', {
    locals: {
      title: 'New Post'
    }
  });
});

Reversing the order fixed the problem.

Benchmark

I found one interesting benchmark, on Express vs sinatra, which as always should be taken with a grain of salt. But the benchmark illustrates some of the properties of reactor servers that we have been talking about.

And although it’s a great achievement, I regret the locked-in and centralized nature of Facebook’s Open Graph Protocol. The web is an open environment, where open wins at the end.

Entity Identifiers

Facebook assigns and owns internal identifiers to entities. These identifiers do not look unique, nor something that external parties, disconnected from Facebook could assign, inspect or do anything useful with. By owning the identifiers, Facebook is owning the entity graph.

The assumption of a single party owning the identifiers for entities is fundamentally flawed. Entities appear, disappear, merge and fork. Known entities change constantly, and most importantly what you and I, or anybody else understands and knows as an entity is different. IMDB and Netflix will both describe the movie “The Book of Eli”, however each use different identifiers. To assume that one unique identifier can be used consistently and universally across the web leads to information lock-in.

Luckily, the web is an open environment. And, at the end, open wins on the web (prove by induction to the avid reader).

Identity on the open web is federated, starting at the top-level domain identifiers, and all the web to a fully qualified Universal Resource Identifiers (URIs). In reality, there are many sources of truth. Some of those are canonical, or trusted, yet many other sources exist. Movies data from IMDB or Netflix will usually be considered canonical. Yet that does not stop Wikipedia, or any web publisher, from creating its own entities. All it takes is a URI.

Anybody on the web, be it Facebook or Freebase, can make statements such as: this URI on Netflix is about the same movie as this other URI on IMDB. All it takes is a web page with a couple of links. The difference between Facebook and the “web” way, is that a web page is identified by a URI, and anybody can create one in a federated way. Not just Facebook.

Semantic Web Annotation Techniques

The most interesting value of the web is not about describing resources, or the roles of the relationships between resources, but the relationships themselves. The statement “Madonna is related to Guy Ritchie” is, in relative terms, more important than the more fact-complete “Madonna is Guy Ritchie’s ex-wife”. Although the fact “ex-wife” increases knowledge, it has only value once we asserted that the relationship exists. Establishing relationships can be achieved through links and URIs. That’s what the web is about. Pages and links to other pages.

The problem however with the current web of pages and links is that all URIs are anonymous. Ideally, I would like to bring some structure to those relationships so that I can codify exactly what “ex-wife” means. That’s what semantic annotations address: naming the role in the relationship between resources in a structured format.

Inlined semantic annotations, ala Facebook, is one possible way of linking data sets. The issue however with this approach is that the meta-data about the resource is mingled with the data. This creates a number of problems. First, there is an arbitrary distinction between what constitutes data and what is metadata. Additionally, one needs to be able to parse the resource format in order to find the semantic annotations. Aside the parsing computing cost, adding semantic annotations to videos, images, etc. would require custom extensions to be container format, which is practically unfeasible. Practically speaking inline semantic annotations are only partially useful, and only for text-readable content types.

A better alternative would be to use out-of-line semantic annotations. In this model, we cleanly separate meta-data from the resource data itself. This separation should be not just syntactical, but structural. Semantic annotation constructs should treat the data as an opaque resource. For all we care, we should treat all data as binary resources. If we wanted to look into the data, an specialized parser would read the data, and surface interesting facts that we could then promote to metadata.

The first possible construct that provides out-of-line semantic annotations is the Link HTTP Header. Using link headers we can describe a web resource without having to parse the payload, by simply looking at the HTTP headers. Examples of usage for link headers could include:

  Link: <http://www.cern.ch/TheBook/chapter2>; rel="Previous"
  Link: <mailto:timbl@w3.org>; rev="Made"; title="Tim Berners-Lee"

Although powerful, link headers are not particularly accessible for most publishers, since they require programming access to the web server to generate those link headers. Additionally, if we are only interested in the semantic annotations, fetching a full document only to throw it away is highly inefficient both for consumer and publisher.

A better construct to provide out-of-line semantic annotations is to create a separate resource altogether representing the semantic annotations for the parent resource. This alternative, alike link headers, also differentiates between data and meta-data, but does not require stack changes to HTTP. Furthermore, it does not require the publisher to generate those semantic annotations. Anybody can publish semantic annotation documents for any resource on the web. In that sense, the use of out-of-line semantic documents that describe web resources is truly open and federated.

The Linked-Atom

There are many possible formats for describing resources in an out-of-line fashions. One of the most interesting formats ones is the The Atom Publishing Format. With Atom, the focus is on identity and linking resources. The actual content is hidden away.

But within the context of semantic annotations, Atom has its shortcomings. Rather than abusing Atom, perhaps we need to create a separate, specialized out-of-line resource descriptor. I call such format the “Linked-Atom”. There are a few differences between Atoms and Linked-Atoms. Whereas the atom is a generic format for content publishing, the linked-atom is only using for linking web resources.

Let’s consider a graph whereby:

• Each resource on the web is unique identified by a URI. Let’s make such resource a vertex, and the identity of this vertex be the URI.
• A resource can link to other resources, also identified by URIs. We’ll make each link a directed edge, and the link’s identity the URI of the target resource.
• Edges have a type, which corresponds to the type of the target resource.
• Edges might be named, or remain anonymous.

In this graph, the set of a vertex and its outbound edges constitutes a linked-atom. The linked-atom introduces some additional constraints:

• A linked-atom is immutable. A change in the graph (adding or removing edges, or changing edge types or edge names) creates a new Atom, with the vertex identifier and a new revision number.
• Linked-atoms are identified by a composite key composed of the vertex identifier and the revision identifier.
• The list of all linked-atoms describing all revisions of a vertex constitutes a collection.
• Collections are uniquely identified by the vertex identifier.

A possible JSON representation for a linked-atom would be:

    {
        "id": "http://example.com/foo.html",
        "rev_id": 1,
        "links": [
            {
                "id": "http://example.com/bar.html",
                "rev_id": 2,
                "type": "text/html",
                "name": "bar" 
            },
            {
                "id": "http://example.com/toto.png",
                "rev_id": 1,
                "type": "image/png",
                "name": "toto" 
            }
        ] 
    }

A more compact representation of the atom is perhaps more interesting for extracting information. Instead of grouping all the edges into a single document, we would create describe each vertex-edge relationship as a N-tuple. A possible implementation of the linked-atom could use n-tuples to store the data:

    http://example.com/foo.html 1 http://example.com/bar.html 2 text/html bar
    http://example.com/foo.html 1 http://example.com/toto.png 1 image/png toto

The Graph

Using linked-atoms, we can model the information in the web, not simply as a graph of pages and links, but a graph of named and typed links between vertices. Each linked-atom represents a statement about a web resource, a piece of knowledge. The advantage of a linked-atom graph is that anybody can publish a document making statements via linked-atoms and collections of linked-atoms, and not just the publisher of the web resource.

This is in contrast with Facebook’s Open Graph Protocol, where only the publisher of the web resource can make such statements, and where only one consumer assigns identifiers to those statements. Maybe the Linked-Atom is not the perfect construct, but it provides an alternative to what I see as a centralized lock-in model that threatens the open nature of the web.

Microsoft saw the same threat with Netscape and Java more than 15 years ago. When the application developers were developing for Windows, Microsoft was controlling the ecosystem, but most importantly, reaping the rewards of the complementary value generated the applications created by those developers. The more applications that were out there, the more users wanted to use Microsoft Windows.

But Netspace and Java were promoting a cross-platform environment with the famous write once, run anywhere. In the Internet, data formats and viewers are open, and any developer can create new applications. Being able to run those applications on multiple platforms is a significant advantage, especially in the context of the Internet. Microsoft reacted by exposing the Windows API, what developers were familiar with, through ActiveX and Internet Explorer. Developers could reuse their tools and code and use these to create great applications that would only run on Internet Explorer and Windows. Great apps attracted users, and more users attracted more developers. This is how Microsoft attacked the jugular of Netscape and Sun. Netscape eventually went bankrupt, but that’s probably partially because of its own doing.

To Apple, Flash and Java technologies pose the same cross-platform threat that Netscape and Java were to Microsoft. Apple wants to control the hardware, the operating system, and the content that goes onto the devices through iTunes. Apple is locking developers onto its own APIs and its own distribution system, controlling all the feedback loops that the Apple ecosystem has.

Steve Jobs clearly learned the lessons well from Bill Gates. By allowing the Flash and Java runtimes, Apple would be opening the door to competitor products and applications running on multiple platforms, which could result in users switching platform if a competitor platform would for example lower costs or subsidize users or developers. Today, developers, and users, are locked in to the Apple platform.

In a market where the economics of developing applications for the iPhone are not that attractive, allowing cross platform would result in killing the value of developing for that platform. Most developers cannot afford to write code in parallel for Symbian, Android and iPhone, as the cost of doing so would eat up whatever small margins they are now getting. But because the alternative platforms to the iPhone have even smaller margins, developers are “happy” to be locked into Apple.

However, such lock-in is dangerous as it is a vector of Apple becoming a monopoly in the portable computing device market. Apple might argue about how ensuring a perfect user experience is the main reason why they control the whole ecosystem. However, Apple not only controls the user experience, but the runtimes. Apple could allow Flash and Java runtimes, and only approve those applications complying with the user experience guidelines. However, it is not likely to do so, because it would be opening the door to make a competitor platform, namely Android, stronger.

This situation makes me highly upset. I still believe that the iPhone SDK is one order of magnitude better than what you get with Symbian, and still a few knots better than what Android has to offer. Symbian and Android suffer, although not on equal degrees, of the multiple identity problem. There is no one Symbian, there are many Symbians. There is no one Andriod, there are many Androids. This atomization of the device and the user experience makes the developer’s jobs significantly more complex than it ought to be.

Flash and Java have some of the same problems as Symbian or Android. Since the target platform is unknown, the developer has to work much harder to provide a user experience that works in all environments. It’s a trade off the developer opts in to. As a developer, I want to have the freedom to opt into a harder development environment, if that means getting cross-platform support because my business model so it requires. And until Apple gives me that, I won’t get anywhere close to the iPhone.

Apple needs a strong competitor, and the only one that is positioned to potentially become one is Nokia. Nokia unfortunately has been playing the disintegration harakiri game really well, and completely opened itself for a competitive attack. As a developer, I don’t care whatsoever what their internal device platform strategy, but externally, as a developer, and consequently as a user, I care about consistency of APIs, distribution channels, tools, etc. Now that Android Linux is becoming the standard for Java on the mobile phone, I wonder if all is left for Nokia is to become the standard for Flash and AIR on the mobile. Except, of course, if they really believe that they can compete with their own Linux distribution.

Back in April, Yahoo made me an offer to move to California, with my family, to take on a new role. Since then, we’ve had twins, boy and girl, moved house once, finished our ever-lasting DIY project, a little summer bungalow in the middle of Kent, only this time using contractors, sold our cars, packed everything, cancelled the school, the utilities, the direct debits, the payments, changed addresses, changed insurance policies, and finally put our lives on a plane.

And it is surprising how many things are broken, that you only realize when you get out of the usual cycle. Returns for faulty products that take weeks, P&C carriers that don’t want to take your money. Or the complexity of servicing or selling your car when you only have one car. The utter absurdity of BT, making as pay 9 months of non-fulfilled line and broadband contract. A TV license officer poking around thinking that everybody in the UK must watch TV, actually the BBC, arguing that otherwise they are weirdos. A council planning officer arguing that our carport is closer to the highway than the main elevation of the house and asking us to pay for planning permission, even though every single aerial photograph shows they are incorrect. A school asking us to pay a full term of tuition, and then denying our daughter from attending. Or well, the length of processing the legal paperwork to work in the US with a Finnish-Spanish family.

It is also saddening, because of many things aren’t broken and we have to leave a little behind. Like the goodbye photo-book our eldest got yesterday from her classmates. Or our beloved Greek-German friends that we’ll miss so much. Or my mahogany Bechstein 1980 model M. Or my new Startrek home office. Or the beautiful views at dawn over the grazing land from our conservatory.

Tomorrow, a new exciting period in our lives starts. As much as I am sad about leaving, I am also thrilled about the opportunity to work for Yahoo! Inc in its Sunnyvale headquarters doing one of what I think is one of the most exciting jobs available in Yahoo! and in the industry. And it’s not just me me. We all are excited. The wife is clearly hoping for more days of sunshine, which should not be difficult to meet, easier life with children, and most definitely, not DIY in years to come.