Erlang Doesn’t Fit The JVM

Hypothetical Labs - kevin - April 03, 2009

There’s been a lot of talk recently about why Erlang should (must?) be ported to the JVM. I happen to think this is a bad idea for several reasons.

But before we get to my list, let me present my Java and Erlang bona fides.

In the not too distant past I spent almost 10 years developing nearly exclusively in Java. I hopped on the Java train when JDK 1.1.8 was released and rode it thru the early JDK 1.5 (aka Java 5) days. I worked on projects both small and large. I was even responsible for implementing two Java standards: JSR-168 and WSRP.

I’ve done extensive performance testing and tuning for entire application suites (think IBM-scale application suites and you’re not far off the mark). I consider myself very familiar with the major parts of the JVM including the JIT and garbage collectors (in all their many forms).

In 2004 I discovered Erlang and toyed with it a bit. Back then there wasn’t much in the way of docs and none of my friends or colleagues were interested in trying out Erlang so I set it aside. A couple of years later, in 2006, the Prags published Programming Erlang and I was hooked. I dove into Erlang and never looked back.

Since 2006 I’ve produced a popular series of Erlang screencasts and put on a training class designed to help developers quickly climb Erlang’s learning curve. Along the way I’ve scored some nice full-time paid Erlang positions, first with the Rails hosting company Engine Yard and now with my own company, Hypothetical Labs.

And now, The List:

1. The JVM has poor support for distributed programming.

   Erlang’s ability to elegantly extend the actor model across VM boundaries is extremely compelling. With very few caveats you can take code which originally ran on a single VM, break it apart, and run it across many nodes with no code changes.

   There’s nothing even close to this on the JVM. Oh, you can get close using third party libraries but it’s not the same as having the feature baked into the language and runtime. With external libraries there are tradeoffs to be made, configuration to fuddle with, and integration issues to conquer. You could make the argument that these libraries would be subsumed into the Erlang4J runtime and, thus, become transparent to developers but I think you’d wind up with a compromised implementation at best as you deal with the inevitable tradeoffs and integration issues.

2. The JVM has no credible support for distributed data.

   I can take any Erlang term and serialize it to a binary form using the function term_to_binary/1. I can take that binary form and squirt it across a network to another node. On the receiving node I can call binary_to_term/1 on the binary blob and turn it back into Erlang. Well, that’s not quite true. There’s a very small subset of data, such as file descriptors and sockets, which make little sense to serialize. So I can’t serialize any Erlang term but I can serialize most of ‘em.

   The JVM has the Serializable interface, ObjectInputStream, ObjectOutputStream, and friends which try to do the same thing. I think they fail utterly. Why? Because developers have to incorporate serialization into the design of their classes from the beginning to make it work. It’s been my experience the vast majority of Java developers think about serialization rarely, if at all. Which means they’re left with tons of classes which cannot easily be serialized. Hacking serialization onto established classes is definitely Not Good Times.

   Even if Erlang4J could provide transparent serialization for Erlang terms the problem would still crop up at the integration boundaries between Java and Erlang code. Non-serializable objects are the rule rather than the exception.

   I don’t want to single out a particular project but I’ve seen enough people suggest Terracotta as a possible solution I want to take a second and explain why it’s a poor substitute, at least as compared to Erlang.

   One, Terracotta is an external library and will force certain tradeoffs to be made. See here for an example of the work required to get Clojure running on a Terracotta cluster. Erlang’s serialization and distribution is seamless and deeply integrated into the language.

   Two, Terracotta requires configuration in order to work. See here for an example of the configuration required. I guess Terracotta could be baked into a language runtime but I don’t see how without requiring lots of configuration. Erlang requires minimal configuration — a single cookie value and a call to net_adm:ping/1 to set up a reliable and distributed network. Erlang’s message passing semantics also simplifies inter-node communication and dealing with inevitable node failures. I’m not sure the same claims could be made for Terracotta.

   Three, Terracotta’s “network memory” model represents a shared everything approach which is the opposite of Erlang’s shared nothing model. Once again this is a case of a library creating a hurdle to overcome at best or forcing a compromise in the overall design at worst.

   ---

   Observation: Supporting concurrency without also supporting distributed programming diminishes the usefulness of the system overall. In other words, it’s interesting to be able to painlessly deal with lots of threads on a single machine but it’s vastly more interesting to deal with lots and lots of threads and data running across many machines.

   ---

3. The JVM doesn’t support first class closures.

   Period. End of sentence. An implementation of Erlang on the JVM will need to provide closures so the only alternative is to simulate them in the language runtime. This will result in a definite performance hit. See the various JVM Scheme implementations and the hoops they jump thru to implement closures to get an idea of the magnitude of the problem.

4. The JVM lacks tail recursion.

   See previous point and multiply the impact by 1000. Erlang uses tail recursion all the time. Halfway measures, like Clojure’s recur keyword, will not suffice. Erlang4J will have to implement its own call stack to provide proper tail recursion which means seamless Java interop just got loads harder.

At it’s heart the JVM is designed to run OO languages very efficiently. This is why languages like Ruby and Python are (mostly) easily
ported to the platform. It should also be no surprise these languages also see significant performance improvements on the JVM due to the platform’s maturity and overall high quality.

However, the JVM is less suited to running non-OO languages. Languages like Erlang, Haskell, and Scheme provide features, like tail recursion, closures, and continuations, which are not prominent in the mainstream OO world the JVM targets. They depart far enough from the OO model to make the JVM a poor platform choice.

I believe we are at the cusp of a major shift in our industry. FP is in the beginning stages of its ascendancy. It seems only natural to me for new platforms to increase in popularity as new languages come to the fore.

Categories: Blogs  Hypothetical Labs  

Comments

 1 2 >

Add comment

Name:

Email:

URL:

Smileys

Remember my personal information

Notify me of follow-up comments?