Powered by

This site is maintained by François Pinard. It is currently served by Express. Genuine sources for pages are often reST files or Org notes. Page generation is driven by site.mk files, relying mainly on GNU Make, Docutils, XSL, and Python scripts.

The following sections quickly state the steps which yielded me to the present solution.

Along the years, the site pages have been served successively by Apache, my own Webert server (in Python), CherryPy (in Python), Yaws (in Erlang), antinode (in JavaScript) and finally Express (in JavaScript). The last two are based on Node, of course. Web servers are always a nice way to explore new programming languages or avenues!

I usually avoid the direct writing of HTML, the documents are kept in more efficient editing formats. Allout has long been my favorite, to the point I later re-implemented the tool for Vim when I got away from Emacs.

David Goodger's excellent Docutils project seduced me enough that I replaced many Allout files with reST files, and tried to prefer it for my sources, whenever possible.

At some later time, the Tomboy note taker happily replaced Yaws wiki pages for me. It was so efficient at editing time that I learned to keep reST only for bigger or more formal documents. I converted the remaining Allout files.

Then, I used the marvelous WorkFlowy Web folding editor to host a great deal of my notes on the cloud. It allowed a fruitful consolidation of all this information. Yet, the notes were mostly private, I had difficulty really integrating it into my publication habits.

While Tomboy and WorkFlowy are really excellent and very polished tools, which I much liked, the impressive Org tool (an Emacs mode) displaced both in my personal work habits. Among many other virtues, Org's format is much simpler and easier to manage than Tomboy's XML, and it gives me the fine grain control I need over the publishable and private parts of my notes.

I swung between static generation of Web pages, with bits of dynamism here and there, and full dynamic on the fly generation by the server.

The technique used for generating static HTML pages, as well as the overall aspect of the site, has been initially stolen and adapted from René Seindal's old GNU m4 site. It was rather heavy on Makefiles however. The marvelous Python language later reduced and simplified it all. The Browse and Folders buttons, once found in the menu section of most projects, were fully served by Python scripts.

I then went a bit overboard with dynamic generation at Webert and CherryPy times, as both used the Traiter tool, a powerful and efficient templating system of mine. reST documents, which are heavier to turn into HTML, later brought me back to reason, and to Makefiles!

Adopting Tomboy, and its XML, I grew a rather complex tool (named tboy, and almost lost within the Git history of my FP etc. project) which was able, among other things, to statically convert a public subset of my notes to HTML. Moreover, once swimming in XML, the dynamic Traiter processing has been mostly replaced by static XSL style sheets. The few remaining cases of dynamism may be addressed by JavaScript code on the server.

On the Org file side, Org's HTML converter and Web site publisher well replace leveraging tools I wrote around Tomboy and WorkFlowy.

Wikis are collaborative devices in which pages are easily modified. Blogs receive chronologically sorted documents on possibly random subjects, relieving the writer from formally inserting information into a sounded, structured tree. I wanted to make a few tries in that direction.

For a while, I linked PyBlosXom to CherryPy. Later going to Yaws (From CherryPy to Yaws), I activated Yaws' Wiki. The above Wiki has not been so productive in practice, most contributions were spam or defacing. So, I abandoned it all, and returned to read-only mode, not allowing comments. For my little blog needs, a tiny Python script replaced the PyBlosXom machine.

Blogger solved many of these problems; it also opened the door to completely different styling. Yet, I discovered I'm not so tempted to use blogs unless I can integrate them intimately with the remainder of my overall documentation setup, and handling Blogger internal formats through automatic processes proved to be too much of a hurdle.

So, I moved the information contents of my blogs from cloud applications, back to my machines (see Icule, Arts and Ordi), and now intend to drive them from Org, would it mean read-only mode. Who knows, I may feel like progressively improving on presentation and adding features while experimenting around…

[2006-07-26 mer] About maintaining documentation and templates as reStructuredText files, I'm using the same set of techniques for almost all of my Web sites (about two dozens of them, yet some are private or local).

Most original pages are reST, and are organised in a directory hierarchy identical to the URL hierarchy for the equivalent rendered HTML pages. Then, I have a Python script which walks down hierarchies, and a bit like a Makefile would do, compares modification times between the reST hierarchy and the HTML hierarchy, and uses docutils for processing anything which recently changed. In most cases, the source and destination hierarchy coincide (visitors may retrieve the reST source by replacing .html by .txt in the URLs). So I can edit reST files directly in the DocumentRoot hierarchy, and because I managed so these hierarchies live within my own account home, I do not even have to install files. But I ought to run my updating script, however.

Save for my blog, I did not want to use docutils on the fly, because some files are long and complex, they would delay page delivery enough for being annoying. Moreover, the generation time would have to be spoiled at each visit. For my blog, which is likely to change often, I decided the other way in favour of my own ease; as blog entries are all rather small, processing time should never be inordinate. And because I run it under a CherryPy server, no extra time is needed for starting Python or for importing docutils on each request.

I also have a third setup much in use. Nearly each local project, at one place I work, has a doc/ hierarchy holding its documentation, most of it being in reST format. We use our own setup module, which is a wrapper around the distutils.core one: this wrapper takes care of automatically and transparently installing the whole documentation hierarchy into our internal Web server, and docutils gets automatically called at project install time, through timestamps examination, for any documents needing to be recomputed. So, later accesses are always fast. All in all, it's fairly convenient for us.

I use my own templating engine (not published either). Keywords are French, as it is meant for French designers or programmers here. Directives are kept within recognisable HTML comments, so HTML pages are valid HTML even before processing. The engine itself is kept within a single Python script, an not a big one. Given a page to process, it does two passes: the first compiles it to some intermediate code, using some moderate optimization, while the second executes that intermediate code and produces the resulting HTML. The added overhead seems quite negligible: we do not feel it at all, given of course that the page does not require serious or hard Python computation on its own. This templating language is well adapted to our needs, and because Python is fully available within it, it is also quite powerful.

Most pages do not use templating at all, except that the scripts or tools transforming reST into HTML automatically insert some boilerplate (two or three lines of directive at the beginning, one or two lines of directive at the end) if they happen to see that an HTML template exists for the site. The boilerplate ensures that the HTML template drives the final rendition, made on the fly: most of the times in this context, the HTML code resulting from docutils processing is mere data. The HTML template uses templating directives for dynamically creating the overall site aspect, and for generating menus, the contextual navigation menu, and special buttons like, for example, the site-wide search facilities.

It means that, once the mechanics in place, the crumbs of complexity are mainly concentrated into HTML templates, usually one per site. Being lazy, I often reuse the same template for many sites. Not something I should be especially proud of, as it shows a bit too much! :-).