Thinking Erlang

I set out this morning to learn how Joe Armstrong handles responses in his Simple Web Server.

But pressing questions about Erlang have been gathering in my mind like looming thunderclouds. Felt it might be more productive to tease them out into the light of day.

The most nagging question is this:

It’s been far more difficult than I expected to trace the core logic of Joe’s Web Server. And, it’s been damned tedious, as you may have noticed.

Why should this be?

No doubt much of the problem stems from my own inexperience with the language. Certainly my ignorance of anonymous functions is a stumbling block.

But I talked with a young Austrian programmer a few months ago who warned me that Erlang source could be rather opaque. And, I’ve noted passing references to this problem in various Erlang-critical blogs.

With your indulgence, I’d like to get your input on this before we continue on. Rather than criticize, I’d rather analyze, synthesize, and strive toward deeper understanding. I’d even hope that we could strive toward an explicit approach to Erlang development and documentation that would vastly simplify code review.

If such methodology already exists, by all means please point the way.

As I see it now, the problem is four-fold:

1) Processes

Cheap concurrent processes are the heart of Erlang. But as Joe spins them off in his Simple Web Server I find it increasingly difficult to maintain clarity of mind. It’s like trying to keep track of all the magician’s spinning plates. I try to make little bubble diagrams, but still lose track of who’s passing what to whom.

2) Factoring

Factoring is the art of breaking down complexity into a set of simple essential functional components, then explicitly denoting the interfaces between them.

It appears to me that factoring decisions face us at three levels in Erlang:

Functions: Taking into account performance constraints, how do we design functions that are simple and easy to understand, yet maximally powerful in combination with other functions?

Modules: Erlang groups functions into modules, presumably to aid conceptual understanding of the program, e.g. hide the complexities of individual function design; and to provide useful “toolkits” or libraries of functionality that can be reused across programs.

The issue here, again, is how do we most efficiently map module functionality to the conceptual components of our various problem domains? In other words, how do we decide which functions should go where in our module scheme?

Processes: How much functionality should we include within a process? When and how should we decide to spawn off a new process?

In general, I found the functions that comprise Joe Armstrong’s Simple Web Server clean and easy to understand. Moreover, the conceptual factoring of modules into web server, http driver, and tcp driver makes sense on the surface.

But the concurrent processes that comprise Simple Web Server seem to run cross-grain with the modules. To understand a given process, I needed to move across different modules. It was this shift back-and-forth across modules, it seems, that tended to break my thread of thought.

I sent Joe Armstrong an e-mail asking his thoughts about factoring Erlang source code. Haven’t heard back. Perhaps he never caught it.

3) Anonymous Functions

Much said in earlier posts. I’m still not confident that my walk-through of Joe’s Simple Web Server to date is 100 percent correct, largely because I lose track of which anonymous function is being passed where.

4) Documentation

Erlang has excellent documentation standards and tools. Joe neglected to avail himself of them.

So, do my difficulties following the logic of Simple Web Server ring a bell with you? Is this a generic problem with Erlang? Or is it a function of a given Erlang programmer’s style? Or is it my own ignorance?

What’s your experience? Does tracing Erlang source become easier as one accumulates understanding of the language? Is there profit in continuing this discussion? Or would you rather move on with the walk-through? Or maybe move on to another topic altogether?

Please Let me know.