Greetings from the Bottom of the Benchmarks

I can guarantee that if you write a benchmark pitting Erlang’s dictionary type against that of any other language, Erlang is going to lose. Horribly. It doesn’t matter if you use choose the dict module or gb_trees; Erlang will still have an embarrassing time of it, and there will be much snickering and posting of stories on the various programming news aggregation sites.




Is the poor showing because dictionaries in Erlang are purely functional, so the benchmark causes much copying of data? Or perhaps because Erlang is dynamically typed?




Neither. It’s because the standard Erlang dictionary modules are written in Erlang.




In that light, the low benchmark numbers are astoundingly impressive. The code for every dictionary insert, look-up, and deletion is run through the same interpreted virtual machine as any other code. And the functions being interpreted aren’t simply managing a big, mutable hash table, but a high-concept, purely functional tree structure. The Erlang code is right there to look at and examine. It isn’t entangled with the dark magic of the runtime system.




There are still some targets of criticism here. Why are there multiple key/value mappings in the standard library, one with the awkward name of gb_trees, in addition to the hackier, clunkier-to-use “Erlang term storage” tables? Why would I choose one over the others? Why is dict singular and gb_trees plural? Let’s face it: The Erlang standard library is not a monument of consistency.




But performance? I’ve used both dictionary types in programs I’ve written, and everything is so instantaneous that I’ve never taken the time to see if a disproportionate amount of time is being spent in those modules. Even if I’m overstating things, over-generalizing based on the particular cases where I’ve used dictionaries, it’s still high-level Erlang code going up against the written-in-C runtime libraries of most languages. And that it comes across as “instantaneous” in any kind of real-world situation is impressive indeed.




(If you liked this, you might enjoy Tales of a Former Disassembly Addict.)