Webmachine — Another painful lesson learned

Just set out to report on the nifty things I’ve learned about Webmachine over this past week only to be bushwhacked by bloody ignorance and oversight.

Justin’s quickstart docs contain this step for bringing up a simple demo web site :

./scripts/new_webmachine.erl mywebdemo /tmp

Works like a charm. But can you see implicit looming disaster?

I didn’t and here’s what happened:

I spent the week playing with different URI dispatch patterns. I wrote functions to report out consequential header variables to follow the action. I played with form input and processing. I learned some stuff and generated a few useful questions. And so, as of yesterday, I’d built up a bunch of newbie tutorial code to share on this blog. All simple-minded stuff, but possibly of value to other newbies.

Thought I’d have a lot to report today.

Not.

Can you see what happened?

Justin’s command builds the Webmachine demo in /tmp. What I’d forgotten is that certain if not all /tmp files are bombed into oblivion when you reboot the system. Guess that’s why they call it /tmp. Had to update and reboot my system today. And… guess what?

Newbie Lesson: If you want to keep your work-in-progress around a little more permanently, modify Justin’s quick start instruction just so:

./scripts/new_webmachine.erl mywebdemo ~/wmplay

Now cd to ~/wmplay/mywebdemo and away you go.

You can substitute any directory name you want for wmplay, but I suggest that you keep it short. You’ll be typing it frequently.

Note: All above assumes that you are working on a LInux platform and you’ve unpacked Webmachine in your home directory. Can’t say if you’ll run into similar issues with Windows or if Webmachine code exists elsewhere.

I do recall one pesky question that I’ll ask now in hopes that a kind wizard will step up with a revelation:

When I created a form in resource module xyz1 as…

<form name="input" action="xyz2" method="get">
<input type="text" name="user" />
<input type="submit" value="Submit" />
</form>

…and submitted it, the input value value showed up just fine.

But…

<form name="input" action="xyz2" method="post">
<input type="text" name="user" />
<input type="submit" value="Submit" />
</form>

…returned a totally blank page, even though xyz2 showed up in the URI, e.g., http://localhost:8000/xyz2.

Anyone know why and how to deal with posted input data in a Webmachine resource?

Anyway, it’s doubtful that’ll have time to get back to Webmachine until mid October. Headed off with my love on Wednesday for an anniversary week Santa Fe. Heaps to do in the meantime.

Hasta más adelante,

LRP

Webmachine — Getting to Know You 1

You know you’re getting old when you walk in on your 50th high school reunion to find wall-to-wall gray hairs.

You really know you’re getting old when it takes three days to recover from the ravages of a red-eye flight from San Francisco to Boston.

Where did all those years go? And where the vim, vigor, and vitality?

Yes, I can hear the young’uns smirking away so smugly. All I can say is, your time is coming.

Well, I’m back now and continuing my exploration of Webmachine.

First thing I noted as I eased back into the saddle was changes to the quick start page.. So, ignore much of what I said in my last post and simply follow the quick start instructions. Works like a charm for me this time around.

As I started delving deeper into the documentation, however, newbie anxieties began rising up out of the pit of my stomach. It’s all there, I’m sure, everything you need to know to build awesome websites on top of Webmachine. But some of the plain English was turning to Greek before my very eyes.

What to do? Maybe hacking out a few examples to challenge and verify my feeble understanding of the Webmachine docs would help. So, in this spirit, I decided to start with URI dispatch configuration and work my way down into more esoteric matters.

All that follows may bore wizards silly. But with all good hopes of helping along a newbie or two, here’s what I did:

At the tail end of the quick start page Justin suggests that you modify the resource /tmp/mywebdemo/src/mywebdemo_resource.erl. Rather than modify mywebdemo_resource.erl, I made six copies — res1.erl … res6.erl.

I then opened each copy and updated the module name: e.g. -module(mywebdemo_resource) to -module(res1), etc.

I also modified the html payload: e.g. {“Hello, new world”, ReqData, State} to {“Hello, res1.”, ReqData, State}, etc.

Now I have seven web pages in my demo site. But before I can access them, I need to update :/tmp/mywebdemo/priv/dispatch.conf.

Here’s what dispatch.conf looked like when I was done:

{[], mywebdemo_resource, []}.
{["res1"], res1, []}.
{["res2"], res2, []}.
{["res3"], res3, []}.
{["res4"], res4, []}.
{["res5"], res5, []}.
{["res6"], res6, []}.

Finally, I killed the Webmachine webserver, executed /tmp/mywebdemo$ make, and restarted the webserver.

So now I have a test bed for exploring Justin’s URI dispatch configuration examples.

Here’s how to access each page:

http://localhost:8000/
http://localhost:8000/res1/
…
http://localhost:8000/res6/

… but you knew that.

I’ll share my next steps in my next post.

See ya’ll.

Webmachine — Toe in the Water

As noted a few posts back, my plan had been to start developing in earnest on Nitrogen. But then, days later, I found that Nitrogen is undergoing major revision. Many moons ago Jaamal suggested that I look at Webmachine. So… what better time than the present.

The first thing you’ll notice when you link into the webmachine wiki is the crisp, clear, and inviting presentation. This doesn’t surprise me. I was fortunate to chat briefly with Justin Sheehy at Erlang Factory/Palo Alto. He definitely impressed me as a competent, no-nonsense, easily approachable professional. Both the website and Justin inspire instant trust and confidence.

I had less than an hour to play when I linked into the Webmachine quickstart page. Looked simple as pie, so I dove in.

And it was simple as pie — well, a few minor chin scratches which I’ll note below to save other newbies the ten minutes or so they cost me:

1. Make sure that you have a working Erlang/OTP release, preferably R12B3 or later.

This, for me, was the hardest step. I’ve recounted my experience in My Day Of Terror — Compiling Erlang on Lenny, August 29, 2009.

2. Get the webmachine code: svn checkout http://webmachine.googlecode.com/svn/trunk/ webmachine-read-only

First, make sure you have a subversion client on your system. I needed to download the binary from the Debian repository. Otherwise, worked like a charm.

3. Build webmachine and mochiweb:
   • cd webmachine-read-only
   • make
   • (cd deps/mochiweb ;make)

   I was momentarily confused by the function of the parens around the last step, but said, what the heck, and just did it. It worked.

• ./scripts/new_webmachine.erl mywebdemo /tmp

  The only hitch here is that the last step left me in deps/mochiweb, so I had to cd back to …/webmachine.

• cd /tmp/mywebdemo
• make
• ./start.sh
4. Take a look! Point a web browser at http://localhost:8000/

And that was it. My browser rewarded me with “hello, new world.”

Now it was time for sheer daring:

1. To make this resource handle URI paths other than /, modify the Dispatch term in /tmp/mywebdemo/src/mywebdemo_sup.erl.

   This step registered a blank until I looked back into the wiki to review the dispatch syntax. Clear as a bell. BUT, when you change source, don’t forget to kill the webmachine server (press cntrl C) then a), run make from /tmp/mywebdemo; and ./start.sh.

2. to make that resource to more interesting things, modify the resource itself at /tmp/mywebdemo/src/mywebdemo_resource.erl.

And — that was it. In less than 37 minutes I had a custom webpage.

Yeah, Justin!

Erlang — One Step Forward; Two Back

A pebble in the road can loom like Mt. Everest to a newbie.

For many months now I’ve been a spectator at the Erlang marathon. Then, several weeks ago I concluded that it was time to lace up my sneakers and set off apace — that is, adopt the Nitrogen web framework and see if I could develop a serious application for myself.

Now keep in mind, I’m not a professional programmer. I can program in several languages, yes. But on my worst day I can barely boot my computer, much less turn out productive code. My best day may well fall short of your worst. So this decision to get serious about Erlang is a major step in and of itself.

But no sooner did I gird myself with resolve than I came smack up against a mountain. No — two mountains; no doubt simple pebbles from the perspective of the wizards and gurus whizzing their way down the course.

First, I found out that developer Rusty Klophaus is in hibernation contemplating a significant revision of Nitrogen. Okay, I could start with what he’s produced to date. Others have and have developed some promising aps. But what’s the profit of spending hours learning a chunk of technology when you know it’s going to be obsolete by the weekend?

The second mountain has to do with setting up an Erlang development system. I did this once on my old workstation, but may have done it wrong and have lost my notes in any event.

The problem is this:

To develop on Nitrogen I need to run Erlang R13B01 on my Debian Lenny system. The current version of Erlang in the Debian Lenny repository is 12B3 — too far back even for Webmachine. Since I can’t find a backport of R13B01, it seems I need to compile it. Shudder!

So far I’ve downloaded and unzipped the source into /usr/local/src. And I’ve read the README. If I cut through the cruft it looks like:

./configure [ options ]
make

But the ./configure options total confuse and intimidate me. So, before I shoot myself in the foot, I’m hoping that some kind soul can guide me through the process. Or, better yet, point me toward a backported Debian package.

All this was two weeks ago. Rather than proceeding apace on the Erlang course I’ve frittered away valuable time on other stuff.

Amazing how such small pebbles can loom up as major obstacles.

Newbie Test: iserve – plunging deeper down the start-up sequence

Our recent (March 30, April 3, and April 6) explorations into Christian Sunesson’s fork of iserve taught us much about OTP/Erlang functionality and conventions, including directory structures, supervision, and gen-server functionality.

In the April 6 post we explored how iserve’s start-up logic created two child processes, a supervisor, iserve_server_sup, and a gen_server, iserve_master.

Now we’re in a position to plunge deeper into the heart of iserve. So let’s poke into iserve_server_sup.

There’s repetition of material from earlier posts here. Think of it as cram review before your midterm.

And, all due cautions here: beware possible Newbie misunderstandings or misinterpretation in what follows. As always, I invite corrections.

First, note the -behavior(supervisor) directive. This tells us that iserve_server_sup is an interface to the supervisor pattern in OTP/Erlang.

iserve_server_sup exports four functions: start_link/0, start_link/1, add_server/2, and init/1.

Recall that during the iserve start_up sequence, iserve_sup created a supervisory process registered as iserve_sup. iserve_sup’s initialization process, iserve_sup:init/1, in turn, called iserve_server_sup:start_link/1, passing in iserve_sup’s PID in the variable Supervisor.

The function iserve_server_sup:start_link/1 creates and registers a new second-level supervisory process called iserve_server_sup. The name is defined in the macro SERVER declared in the -define directive at the top of the module source listing.

When the supervisory process iserve_server_sup is created, supervisor:start_link/3 calls init/1, which initializes the new supervisory process.

iserve_server_sup:init/1 first instantiates the variable AChild with a tuple that defines a call to iserve_server, then fires off the message {ok, {{simple_one_for_one,10,300},[AChild]}}.

So who’s waiting with abated breath for this message?

The OTP/Erlang generic supervisor. The message defines a simple_one_for_one restart strategy for iserve_server_sup, with no more than 10 restarts within 300 seconds. AChild defines which child processes iserve_server_sup should start and monitor.

The value of AChild is:

{iserve_server,{iserve_server,start_link,[iserve_master]},
temporary,2000,worker,[iserve_server]}

So how do we interpret all this? We’ve gone through this before, but let’s it again (nothing like repetition to sharpen the Newbie mind):

iserve_server – a name used to identify the child specification by the supervisor.

iserve_server, start_link,[iserve_master] – Start function in form of M,F,A; e.g. iserve_server:start_link(iserve_master).

temporary – restart function.

2000 – Shutdown. The supervisor will tell the child process to terminate by calling exit(Child,shutdown) and then wait for an exit signal with reason shutdown back from the child process. If no exit signal is received within the specified time, 2000 seconds in our case, the child process is unconditionally terminated using exit(Child,kill).

worker – child type

iserve_server – Module; used by the release handler during code replacement to determine which processes are using a certain module. Module is the callback module, if the child process is a supervisor, gen_server or gen_fsm.

(April 22, 2009 — Mea Culpa!: I think I stepped off the track when I wrote the following. Don’t believe a word of it. I’ll revisit as soon as I’m more confident that I know what I’m talking about. LRP)

So it’s looking like the supervisor iserve_server_sup is going to be looking after the gen_server iserve master.

Let’s check it out by poking into iserve_master.

First off, Christian tells us that iserve_master provides “Bookkeeping of running servers.”

Next, the -behavior(gen_server) tells us that it provides the callback functions for gen_server.

I hesitate to plow through the bloody details of gen_server. Been there. Done that. So I won’t. You can read about it here.

For our purposes, the things to note are the internal functions call_add_server and call_del_server.

Now these look interesting, but by now you’re undoubtedly bored to tears and I’m running out of pep.

So let’s pick it up another day.

Web Templates for Erlang: Continued from previous post

“8) If we fore go fancy GUI content editing and styling tools, then this system can be fairly simple to implement; puts no added burden on the one-man-band web site developer.

9) In my experience with systems that conceptually approach this model I’ve seen substantial acceleration of productivity.

So, I argue, divide and conquer! “

Fact is, this is the kind of system I’m working toward. Would welcome Erlang wizard input.

Web Templates for Erlang: A Different Take

Much discussion of web templates on the Google Groups – Nitrogen forum recently.

I stuck in my two cents and generated some interesting discussion.

This is a topic I care a bunch about based on experience and hard knocks.

Today I posted the rationale behind my thinking, thought I’d share it.

Note, I define a “Part” as an encapsulated single-function black box of code that can be configured to display CSS stylized web content. You can think component, applet, mini-application, or what have you.

So here it is:

“I’d venture that the truest thing one can say about templates is, ‘different horses for different courses.’

Page-level, monolithic templates are fine for relatively simple pages.

But I argue that there is a conceptual mismatch between the way many web developers, particularly those on the programming side, see the web and the way many many sites are presented today, indeed, the way many page content developers and designers conceptualize a web page.

On the one hand, the conceptual unit is the ‘page.’ Makes sense, since the page is the basic unit of HTML.

But for well-trained designers the page is simply a frame for a ‘page grid.’ Squint your eyes and look at any magazine, newspaper, or even well-designed brochure, and you’ll see a mathematical “grid” structure, where each panel in the grid is a container for content. True, many designers spread content over two or more panels, but the basic structure holds.

The grid system is a fundamental concept in many graphic design text books. Why? Because it provides a coherent way of organizing disparate elements of content.

Squint at many, if not most, well-designed, high-traffic websites and you’ll see the grid system at work organizing the content. The grid system is like a well-organized closet with shelves and cubby holes — everything has a place and everything is in its place, whereas a page-oriented design is more like one of my wife’s kitchen drawers, everything tossed in a jumble with everything else (I should talk; my closets and drawers are even more disorganized).

Now take a look at amazon.com. I have no idea of how Amazon constructs its pages. Maybe as someone suggested it’s a big hairy content management system. Doesn’t matter to my basic argument.

Call up page source for the Amazon home page. Now squint your eyes and substitute variables for every substantive element; e.g. in your mind, turn the source into a template. Now imagine a programmer giving you a list of variable names for different dynamic content elements and working through the template to match content item with place-holder in your big, long, hairy template file.

I’m sure no one does it quite this way, but you get the idea.

Now take another look at Amazon’s home page. Squint your eyes and you’ll see a simple three-column grid, or “make-up” as a designer would say; e.g. left and right side bars surrounding a main content panel.

Look more closely at the top of the grid and you’ll see three content elements (personalization line, personal menu, and search bar) that span the content column and right side bar. They’re still consistent with the basic grid, but they add visual diversity and interest while keeping content well organized.

Further note that the basic columns contain simple lists of content elements stacked one atop another.

Now, look at the variety of content elements that fill the panels in the grid and think about the code, HTML formating, and CSS styling tags needed to create each element. You’ll note that some elements are straight text, some are text and graphics, some made up of hyper links, some form elements, some buttons, and some report data drawn up from a database.

When I talk about ‘Parts,’ this is essentially what I’m talking about. Content elements can be abstracted into a set of ‘Parts,’ distinguished by underlying logic.

Now imagine a page construction process that goes like this:

1) Programmers provide an inventory of Parts, single-function black boxes of code that can be configured with content and styling to generate content elements.

2) Select the Parts you need to convey the information you want. Enter content for each.

3) Design (or select from an inventory) a page grid, essentially a list of DIVs (see my earlier post for example).

4) Develop (or select from inventory) CSS file(s) for your page

5) Plug parts into the page grid

6) Tweak styles in each part if necessary to maintain stylistic consistency with page.

A system like this:

1) Is conceptually consistent with the way content developers and trained designers think

2) Does not necessarily require the page designer to adopt any additional tools beyond HTML and CSS.

3) Breaks down a potentially confusing task into a set of relatively simple tasks.

4) Content and styling can be configured with a simple text editor on the one hand, or sophisticated GUI tools on the other. Indeed, GUI tools could be an additional layer on top of the simpler system.

If the base system were open-source, then specialized parts and GUI systems could add value to meet specialized requirements or support a variety of business models.
E.g., Parts could be open-source or proprietary; offering the best of both worlds.

5) Encourages substantial code reuse.

6) Pushes code complexity down into the Parts. With careful factoring, code complexity may never even be an issue but for the rarest and most algorithmically challenging Parts.

7) Offers great flexibility. Change the page grid and/or the high-level CSS file and you change the basic look of the page. Reuse content elements across pages. Change a content element with little to no interaction with other content elements.”

(Getting too long here… continued next post)

Newbie Test: iserve – Oh oh… Beware!

Just spotted this on Christian Suneson’s blog…

“I probably broke iserve build-simplicity tonight.”

Don’t know if Christian has fixed it yet. But if you have trouble bringing up iserve, it’s a good guess that Christian’s on the case.

Newbie Test: iserve – start-up logic

In our last two posts, March 30 and April 3, we poked into Erlang/OTP conventions, supervisor processes, and application resource files.

Yes, I know, this is supposed to be a Newbie Test. Well, it is. But the opportunity to study how iserve is set up as an Erlang/OTP application is just too good to pass up.

Newbies need to nibble down nifty knowledge at every pass.

iserve’s application resource file, iserve.app, tells us that we must poke into iserve_sup.erl to see how iserve is fired up.

As noted in our March 30 post, you can recognize iserve_sup.erl as a supervisor callback module by its name, *_sup.erl, and the -behviour(supevisor) directive near the top of the iserve_sup.erl module.

iserve_sup.erl exports two functions: start_link/0 and init/1.

Here’s start_link/0.

start_link() -/>
supervisor:start_link({local, ?MODULE}, ?MODULE, []).

supervisor:start_link/3 creates a supervisor process as part of a supervision tree and ensures that the supervisor is linked to the calling process (its supervisor).

Note that the first parameter of supervisor:start_link/3 is of the form {local,Name}.

This means that the supervisor is registered locally as Name using register/2.

register/2 expects a name, Name, and a process identity, P. It associates Name with the port or process identity P.

The second parameter is the name of the callback module, in this case, iserve_sup.erl.

The third parameter is an argument passed to init/1.

So, we fire off iserve:start_link/0. This kicks off a locally registered supervisor process with the name iserve_sup. Our new supervisor now calls init/1 to find out about restart strategy, maximum restart frequency and child processes.

In our case, init/1 starts two new child processes, a supervisors, iserve_server_sup, and a gen_server, iserve_master.

We looked at how supervisor processes are configured in our March 30 post.

Christian kindly provides source comments to help us understand the functions of these two processes:

%% The servers supervisor is for keeping individual servers,
%% adding and removing them dynamically

%% The master is for book keeping all running servers and
%% to hold information about them.

So what do we find when we poke into iserve_server_sup?

Well, my eyes are too blurry to find out. But I’ll try to get back to this when the dust settles. Promise.

Back anon.

Note to wizards: corrections/comments welcomed.

Newbie Test: iserve – application resource file

March 30 Poking Around Erlang touched on iserve’s application resource file, iserve.app.

If you’re an Erlang newbie, it’s worth studying this file closely. Much can be gleaned about the application you’re setting out to understand.

At risk of boring wizards beyond tears, here is iserve.ap in all it’s glory:

{application, iserve,
[{description, "Web Server"},
{vsn, "%ISERVE_VSN%"},
{modules, [ iserve_sup,
iserve_app,
iserve_server,
iserve_socket
]},

{registered, [ iserve_sup]},
{applications, [kernel, stdlib, sasl]},
{mod, {iserve_app, []}},
{env, [{port, 8080}, {callback, iserve_test}]}]}.

Why is this file important and what does it tell us?

The Erlang app man page is a good place to go for answers.

Erlang/OTP conventions call for one application resource file for each application in the system. The file should be named Application.app, where Application is the name of the application. It should be located in the application’s ebin directory.

The application resource file is read by the application controller when an application is loaded/started. It is also used by functions in systools when generating start scripts.

Note that the structure of the iserve’s resource specification is a series of comma-delimited tuples, Each tuple is prefaced with an atom, italicized below.

I’d hoped to have decoder rings made up, but the following will have to do in the interim:

application (name of the application): iserve

description (one-line description of the application): Web Server

version (version of the application): “%ISERVE_VSN%” NOTE: instantiated elsewhere

modules (all modules introduced by the application): iserve_sup, iserve_app,
iserve_server, iserve_socket

registered (all names of registered processes started in this application): iserve_sup

applications (all applications which must be started before this application is allowed to be started): kernel, stdlib, sasl

mod (application callback module and start argument): iserve_app, []

env (configuration parameters used by the application): port, 8080; callback, iserve_test

So, now we know that among all the files in the iserve directories, we must pay closest attention to iserve_server and iserve_socket to understand functionality. We must start up kernel, stdlib, and sasl before we start iserve. To start iserve, we need to call iserve_sup.

Useful overview for one short file. And the good news is that you’ll find a similar file in the ebin directory of every Erlang/OTP application.