Putting Lore to reST

Back in 1999, when I started the Open Book Project, I went looking for tools that would enable educators to collaborate on the creation of free educational materials. My criteria at the time were:

1. Easy enough to use that high school teachers (such as myself) could focus on pedagogy instead of technology.
2. Enable nice looking output for the web.
3. Enable nice looking output for the printer.
4. Support the auto generation of a "back of the book" index with links from selected terms to the locations in the document where they occur.

The last of these requirements was the most difficult to find (in fact, I didn't find any adequate solution), but I needed it for marking up How to Think Like a Computer Scientist: Learning with Python.

Allen Downey's original version of the book was marked up in LaTeX, which supported requirement 4 just fine, but only for a printed version of the text. I was much more interested in the web version. LaTeX did not support requirement 4 for the web, and did a pretty bad job with requirement 2 as well. It also failed to meet requirement 1.

Someone at O'Reilly recommended that I try DocBook, but the only book available at the time for learning it was over 500 pages long and "Hello World" didn't come until somewhere in the middle. So DocBook failed to meet requirement 1, which was a show stopper. I wrote back asking if someone could mark up one chapter of my book in DocBook, and I could than just copy what they did with the other chapters, but no one took me up on that request.

Unsolicited, Moshe Zadka, from the twisted project, sent me chapter 1 of the book marked up using lore, the documentation system that twisted had created for their internal use. Lore used a subset of XHTML for its mark up, making it by far the best tool at meeting requirement 1. It met requirements 2 and 3 very well also, since the twisted folks had created it with solving problems similar to mine in mind. That left only requirement 4 unmet, and after pleading with my friend George Paci, he hacked on lore enough to get it to the point where I could use it for the book.

I've been using lore for years, and I am very likely one of the only people left in the world still doing that. It seems that the twisted project itself has abandoned it, so I realized that it was a technology without a future (Moshe pretty much confirmed that when I talked to him at PyCon this year).

Enter the Sphinx

At PyCon 2008, David Goodger suggested that I look at reStructuredText. Since reST did not have a solution to requirement 4 at the time, I looked at it but decided not to make any commitment. The situation is totally changed now thanks to Sphinx. Sphinx meets all four requirements very well. It uses reST as it's markup language, making it easy enough to learn. it produces beautiful web output, and uses conversion to LaTeX to then produce PDF output for printing.

With the help of html2rst.py, I was able to quickly convert all my lore files into reST. Using Sphinx, it was easy to create a great looking web version of the Gasp Python Course from the newly generated reST sources. Since teaching duties call me, I'll have to provide the details of how to do this in a later post...