Since Subbu wrote BigPipe using node.js, I had to see how the same thing would look like in a Java async servlet.

Stephan Schmidt had already written Facebook BigPipe for Java, but using a synchronous servlet model, not asynchronous. I decided to implement it using Jetty continuations and the Jetty HTTP client, but the code should be easy to adapt to servlet 3.0 AsyncContext.

The code initially constructs the page with a few empty divs that will contain the pagelets, for example:

<div id='pagelet3'></div>

I keep the connection open from the browser to the server while I render pagelets. As pagelets become available, I flush them onto the browser, which in turn inserts them into the right place in the DOM using Javascript. Assuming we get back a response from the remote module protocol:

<span>some useful message for module: pagelet3</span>

We wrap it inside Javascript so that it looks like:

<script>arrived('pagelet3', '<span>some useful message for module: pagelet3</span>');</script>

All is left is for the arrived function to update the pagelet3 element in the DOM.

In terms of concurrency, the initial page construction is done synchronously. As soon as I have the frame in place with the divs that will eventually hold the pagelets, I suspend the execution of the servlet, which in turn releases the thread that was attached to the client connection. At that point, I fire in parallel HTTP client requests to the remote module server.

For resuming the continuation, I could have used a counter, but since each HTTP client execution is a separate thread, unlike in node.js, I did not want to have to acquire a lock and synchronize to be able to update the counter. Instead, I used request attributes, as setting, getting and removing them is thread safe. The code keeps a list of references to the elements that we are offloading to a remote module server, as an ArrayList with the ids to fire parallel requests to http://localhost:8080/module?id={id}; each id is also kept as a request attribute. As each request comes back from the remote module server, I write and flush the response buffer, so that the element appears on the browser immediately, and I remove the corresponding request attribute.

Although the code is not as readable and straight forward as what Subbu got with node.js, I am actually surprised how simple the async Java solution actually turned out to be.

I have been using emacs and the command line for now almost 20 years. Once in a while, I dip into IDEs, but always go back to the command line. My biggest gripe with IDEs is that it keeps me away from the actual build, and that I normally have to spend time duplicating building configuration in the IDE. The canonical source for build configuration should be the build system (Make, ant, maven, etc.). With an IDE one would have to always be things in sync, leading to errors and all sort of weird stuff.

But I keep trying looking for the holy grail. In the past, I’ve been relatively happy with Eclipse for the odd refactoring job. I would fire it up, fix, and close it, back again to the command line.

This time around, I wanted to see if an IDE would make me stop using emacs. I set myself a few requirements:

• Auto-complete. Not intellisense crap. Basic emacs completion is enough.
• Maven is the source. I don’t want to keep a parallel reality when maven poms have all the information already on it.
• Unit testing. I want to run and debug my unit tests from the IDE.
• Basic re-factoring, especially code usage reports.
• Light on resource usage.

I looked into IntelliJ IDEA 8.1, Netbeans 6.7 and Eclipse 3.5. Here are my findings.

IDEA

I got an evaluation license and only used it a few days. IDEA is a fantastic IDE, and despite high memory consumption, it felt light-weight and kept the system very responsive. In principle, it met all the requirements, and the refactoring support was simply the best out of the lot. However, at some point things started to go wrong:

• Compilation errors. Cannot find symbol, although running maven from the command line would not throw any errors. I did not want to spend figuring out what was wrong here. I am sure some IDEA expert would have told me I was doing something wrong, or that some configuration had to be changed. But IDEA was already failing the requirement: maven is the source. I don’t want to fiddle around with the IDE settings.
• Mercurial integration problems. The plugin kept on throwing up and breaking the environment. I had to uninstall it, but that’s really a minor point.

Eclipse

Then came Eclipse. Overall it looked good, but there was no Maven support out of the box. I tried installing Q4e, but getting JUnit support in debug mode was just too painful. m2eclipse worked fine, and Eclipse met all requirements with it.

Netbeans

Netbeans was last. I’ve never liked Netbeans. But I did now. A lot.

It looked good. It mapped nicely into OS X, which the others did not, in terms of user input and display. It supported Maven out of the box as a source of truth for build configuration. It offered substantial more refactoring features than Eclipse, but less than IDEA. Mercurial integration was spot on. And I could run my unit tests in debug mode.

But most importantly, Netbeans did not get on my way. I did not have to configure anything to get going. It was like using emacs, but with all the goodies.

The winner

I am going for Netbeans. IDEA looks good, but I am not sure I would pay out the license fee for the powerful refactoring features. Others might, but I probably won’t. Eclipse was also improved, but having to fiddle with plugins to get Maven support was really sub-standard.

So, it is Netbeans.

I have been working on AtomPub and coding a prototype over the past few evenings to get Apache Abdera talking to CouchDB. I know there is an experimental adapter in Abdera for CouchDB, but it does not use CouchDB the way I need to.

I decided to write my own provider, workspace manager, adapter, etc. I have been using couchdb4j, the Java library binding used to talk to CouchDB, and json-lib, a JSON library for Java.

Why, it has been really especially extremely very painful!

HTTP and JSON, REST a bit, and to a lesser degree XML, are supposed to be technologies that make your life easier as a programmer and as a server. But with Java, it’s the other way round. What should be easy, becomes complex. I am very disillusioned about the suitability for the Java language in pretty much any domain related to HTTP.

With Java in HTTP, you have to jump API hurdles, pull hundreds of dependencies, and end up with stack traces deep as a black hole. Yeah, it works, mostly because of the amazing development support and tools for Java, such as Maven and Eclipse. But that alone does not make Java a viable technology for programming HTTP.

Java has it uses, and it is a good programming language. I don’t think I’d ever code C++ again if I could use Java. But that’s more on the number crunching, for the Hadoop of this world.

I’ll stay with my PHP, Ruby and Python while I can. Perhaps it’s also time to learn a functional programming language like Erlang.

This is truly amazing. Sun and Oracle are set to buy all letters on the LAMP stack, with now Sun buying MySQL for 1 billion dollars (thanks Pascal for the ping). Maybe now Monty and David will start competing with Larry Ellison’s extravagant yacht department. Seriously, 1 billion dollars is a record price for any open source company. Will Oracle now finally buy RedHat to put pressure on SUN?

I am hoping to see the JAVA ticker symbol company improve MySQL support, and licensing, for Java. Since, honestly, right now it sucks so much that one would say the only viable RDBMS alternative for Linux and Java is Oracle (please don’t get me started with DB2).

The Java EE 5 architecture, with its use of annotations brings the Java world close to being agile. The architecture is absolutely fantastic and powerful, with superb messaging (JMS), persistence (EJB3), transactions (JTA), and integration (JCA) capabilities.

However, on the presentation side, for the web world, Java is still behind both .NET and agile frameworks like Rails or Django. JSF is cumbersome, and still too attached to the systems-programming intensive request-response Model2 that ended up in JSPs and Struts. Wicket and Tapestry are possibly the top component-based development web frameworks for Java. But they are both only used by a relatively small community, and documentation, mostly in the case of Wicket, is deficient, especially when it comes to topics such as integration with persistence frameworks.

In terms of setting a complete development environment, it is painful. Setting up Netbeans/Eclipse, JBoss/Geronimo/Glassfish, Wicket, and EJB3 to play nicely with each other is not easy, and it has lots of rough edges. It takes too long to setup an environment and start coding.

That myriad of frameworks and standards is what makes Java EE so powerful, yet so cumbersome. I was hoping annotations could have solved most of the integration complexity, but I am disappointed. So, sadly, my hopes for a “new Java” are yet once again down. Java EE still gets on your way to being productive.

I am back to Rails for the time being, at least until Java EE 5 matures.

Latest benchmark results on WebSphere 6.0.1 seem to indicate that SOAP/HTTP outperforms RMI/IIOP. Interesting, it makes me wonder if the reason comes down to IBM abandoning CORBA and placing the best developers on the SOA field. Otherwise there is really no other explanation I can think of. I’ll have to instrument some code to understand the why.