Putting Lore to reST

Back in 1999, when I started the Open Book Project, I went looking for tools that would enable educators to collaborate on the creation of free educational materials. My criteria at the time were:

1. Easy enough to use that high school teachers (such as myself) could focus on pedagogy instead of technology.
2. Enable nice looking output for the web.
3. Enable nice looking output for the printer.
4. Support the auto generation of a "back of the book" index with links from selected terms to the locations in the document where they occur.

The last of these requirements was the most difficult to find (in fact, I didn't find any adequate solution), but I needed it for marking up How to Think Like a Computer Scientist: Learning with Python.

Allen Downey's original version of the book was marked up in LaTeX, which supported requirement 4 just fine, but only for a printed version of the text. I was much more interested in the web version. LaTeX did not support requirement 4 for the web, and did a pretty bad job with requirement 2 as well. It also failed to meet requirement 1.

Someone at O'Reilly recommended that I try DocBook, but the only book available at the time for learning it was over 500 pages long and "Hello World" didn't come until somewhere in the middle. So DocBook failed to meet requirement 1, which was a show stopper. I wrote back asking if someone could mark up one chapter of my book in DocBook, and I could than just copy what they did with the other chapters, but no one took me up on that request.

Unsolicited, Moshe Zadka, from the twisted project, sent me chapter 1 of the book marked up using lore, the documentation system that twisted had created for their internal use. Lore used a subset of XHTML for its mark up, making it by far the best tool at meeting requirement 1. It met requirements 2 and 3 very well also, since the twisted folks had created it with solving problems similar to mine in mind. That left only requirement 4 unmet, and after pleading with my friend George Paci, he hacked on lore enough to get it to the point where I could use it for the book.

I've been using lore for years, and I am very likely one of the only people left in the world still doing that. It seems that the twisted project itself has abandoned it, so I realized that it was a technology without a future (Moshe pretty much confirmed that when I talked to him at PyCon this year).

Enter the Sphinx

At PyCon 2008, David Goodger suggested that I look at reStructuredText. Since reST did not have a solution to requirement 4 at the time, I looked at it but decided not to make any commitment. The situation is totally changed now thanks to Sphinx. Sphinx meets all four requirements very well. It uses reST as it's markup language, making it easy enough to learn. it produces beautiful web output, and uses conversion to LaTeX to then produce PDF output for printing.

With the help of html2rst.py, I was able to quickly convert all my lore files into reST. Using Sphinx, it was easy to create a great looking web version of the Gasp Python Course from the newly generated reST sources. Since teaching duties call me, I'll have to provide the details of how to do this in a later post...

Sugar Labs DC is Born

I think Matt now understands what needs to be done regarding Sugar Labs DC. He sent me the following two links to help guide us on setting up the wiki page:

• http://wiki.sugarlabs.org/go/Local_labs
• http://wiki.sugarlabs.org/go/Sugar_Labs_Ghana

Using these as guides, I setup our new wiki page:

• http://wiki.sugarlabs.org/go/Sugar_Labs_DC

I'll meet with Jamie and Matt later today to talk more about how to use this site to organize our activity.

The Customs Saga Continues...

On other news, Douglas, Raul, Paco, and I met on irc yesterday to deal yet again with the still unresolved problem of getting our XOs out of customs. After a long series of back and forth communications, the evening ended with Adam Holt calling Douglas. I hope between the two of them they are able to find a way out of this problem.

When in Rome...

The XOs sent to El Salvador are still stuck in customs, and Douglas is at wits end as to what to do about it. Paco, Douglas and I have been meeting daily to find our way out of the mess. My only contribution has been to suggest a "Plan B": have UPS return the laptops and then have them shipped to Arlington, Virginia. They are small and light enough that we could then send them to our project in Chalate little by little with friends and family members as they travel back and forth to El Salvador. I'll check in with Douglas and Paco today and see if I should contact Adam Holt to propose this to him.

On a more encouraging note, I'm moving ahead with plans to setup the web infrastructure for Sugar Labs, DC. Our three biggest needs are wiki space, a mailing list, and a code repository. The last item should have feature and bug tracking facilities along with it. I asked Luke what he thought we should do, and he suggested the following:

• "http://wiki.sugarlabs.org/go/dc/YourPageHere" for wiki
• SugarLabs mailing list hosting using mailman and forum intergration using nabble (free Software-as-a-Service)
• http://git.sugarlabs.org for code
• http://dev.sugarlabs.org/ for bug tracking

This is very helpful information. It is a no brainer to follow the old adage, "when in Rome, do as the Romans do." Since the Sugar Labs community already has a set of practices for doing what we need, we should simply follow them. It is great that Luke is active enough in the community to be able to send me this list. I'll get together with Jamie and Matt today and talk about moving this forward. I may have to setup the wiki myself, since neither Jamie nor Matt have a penchant towards writing. It is extremely important for them to learn to write about technical matters, however, so I'll struggle with them first before giving in and doing it myself.

What's Next?

I was able to make two SD cards boot Xubuntu on the XO yesterday. My good friend Paul Flint always says that there are two answers to all Unix questions:

1. It's a dash, dash something or other, look it up in the man page.
2. It's a permissions problem, I'll fix it later.
   

This turned out to be to be the second one. In copying everything from the SD card to /opt, files that needed to be owned by the olpc user became owned by root. I could have resolved this by using the -p switch with cp, but instead I used tar. Here is what I did from a root prompt:

• cd /media/OLPCRoot
• tar czvf /opt/XO_Xubuntu_OLPCRoot.tgz *
  

Now when I want to create a new Xubuntu SD card, after partitioning and formating the card, I:

• cd /media/OLPCRoot
• tar xzvf /opt/XO_Xubuntu_OLPCRoot.tgz

A Few Questions Before Moving On...

I am more than ready to get back into developing Python curriculum materials and learning reStructuredText. The following questions remain to be answered at a later time:

1. Is 0.82.1 the Sugar version we want to run on the XOs?
2. Given how immature Sugar still is, can new Activities easily be written that will run on 0.82.1 and the latest version of Sugar?

These questions reflect the problem I've always had resolving the contradiction between two opposing goals: direct contribution to the development of the project (which usually means running the latest version of the software), and enough stability to be able to deliver a usable platform to our student users (which usually doesn't work well at all with the latest version of the software).

I'll have to come back to this in a later post, as teaching duties now call me away...

Xubuntu Intrepid on SD Card

I've had an SD card since last December with Xubuntu Intrepid on it customized for the XO. I created it with these instructions. I've been happily using it on my own XO since that time and have made several small modifications while at the same time keeping all the packages up to date.

Now I want to see if I can produce 10 to 20 of these cards to use with our new XOs. Today I plan to test out a process for creating these cards using:

• ativa SDHC 4GB cards
• my Asus eeepc 1000

I started by putting the working SD card into the eeepc. The card reader on that laptop is awesome. I have yet to find a card it wouldn't read, and I put several cards into it that were not able to be read by other laptop readers.

It recognized the card as /dev/sdc1 and mounted it on /media/OLPCRoot. At a root prompt on the eepc, I then did the following:

• mkdir /opt/OLPCRoot
• cp -R /media/OLPCRoot/* /opt/OLPCRoot

It took around 5 minutes to copy all the files over. Since teapot's instructions assume you are making the new card from an XO, I had to adapt them for the eeepc. Here is what I did next (from a root prompt):

• inserted the ativa SD card into the eeepc card reader (it recognized it again as /dev/sdc1)
• umount /dev/sdc1
• dd if=/dev/zero of=/dev/sdc bs=4096 count=1 (to erase the partition table)
• echo -e ',,L,*\n\n\n' | sfdisk /dev/sdc (to create a new partition table)
• mke2fs -jLOLPCRoot /dev/sdc1 (to create an ext2 file system on the new partition)
• removed the card and reinserted it so it would automount
• cp -R /opt/OLPCRoot/* /media/OLPCRoot/

It took about 9 minutes to copy the files to the new SD card. Actually, when I went to unmount the drive using the Gnome media browser, it popped up a window saying it could not unmount the drive, and than another window titled "Single Flash Reader" which said "Writing data to device". The activity bar in this window remained active for about 3 minutes and then closed.

It Works (sort of)!

I took the new SD card out of the eeepc's card reader and put it in my XO. I turned it on and... it worked! Actually, it booted as far as the custom Xubuntu splash screen and then seemed to freeze. After what seemed like 2 minutes, I was about to turn off the machine using the power button when the progress bar started advancing. It finally proceeded to the gdm login screen, and I tried logging in using the olpc username and password I have configured on the source card. I got an error message that the $HOME/.dmrc file should be owned by the user with permissions of 644.

It looks like this process is close, but will require some tweeking. I'll talk to Matt about it tomorrow. In the mean time, I have confirmed that the ativa SD cards will work, so I can put in an order for a bunch of them tomorrow while the sale is on at Office Depot.

Another Rockin' OLPC Learning Club Meeting!

The OLPC Learning Club DC (http://olpclearningclub.org) continues to be the most exciting user's group in which I have ever had the good fortune to participate. It brings together an eclectic mix of scientists, educators, children with parents in tow, hackers, and small computer techies that has made for a dynamic intercultural exchange.

I was first exposed to Scratch and eToys at these meetings. Scratch has since become a regular teaching tool I use in the classroom. I met biologist Dr. Frank Linton at these meetings, which led to the bee project and set the groundwork for aquiring the skills we will need to start SugarLabs, DC.

Today's meeting featured a presentation on cloud computing by Kim Toufectis. The timing could not have been better, since I was planning on giving a presentation to one of my classes on cloud computing on Monday. Now I can just give Kim's presentation, saving me the prep time I would have needed to prepare one on my own. Ten years ago, when I started the Open Book Project, I had hope that the Internet would one day provide a platform for educators to share in the development of high quality educational materials for the benefit of all. That day now seems to be approaching.

9 Out of 10 Aint' Bad

Of the 10 XOs we received, 9 of them worked perfectly and updated without problem. One of them reported a write error when I attempted to update Sugar on it. I've given that one to Matt to see if the can confirm the problem.

Documenting the Laptops

Paco and I met at his house the meeting to prepare for our meeting on irc tonight with the folks from Chalate. The first thing we did was to create a google doc spreadsheet listing each of the XOs we will be using locally with the project. This will help us keep track of our resources and record how we are using them.

¿Habla Español? - Part II

The Wikipedia activities were indeed language specific (that makes perfect sense). In the four XOs we are naming "azucar1" through "azucar4" and setting up to use in Spanish, we needed to remove these activites. The Activies page on the OLPC wiki says to do the following :

• cd Activities
• rm -f WikipediaEN.activity

That worked without a hitch, and after rebooting the laptop the Wikipedia activites no longer appear in the favorites ring.

Streamlining the Process

It seems that my agonizingly slow wireless access was the cause of the hours long wait for the update. I upgraded another XO at school this morning using a usb ethernet adapter, and it the whole process took less than an hour.

Less than 10 minutes of the time requires active involvement of the person doing the install, so I will be able to update all 11 of our XOs myself by the early part of next week. With 30 machines to do in El Salvador, Douglas will probably want to train some folks in Chalate to do the process themselves.

¿Habla Español?

As soon as I finished updating the new machine in school, I tried switching the language to Spanish. Douglas and I talked about the project on skype yesterday, and he asked about how difficult it would be to switch languages. While I haven't done any serious testing yet, at first glance it appears to be as simple as switching the language setting in the control panel. It requires a reboot, and then everything seems to work in the new language. I'll setup the next few machines for Paco and I to use in Spanish.

Removing Packages

Switching to the language brings up another issue. There is a Wikipedia activity that loads 96 megabytes onto the laptop, and it appears to be in English only (the name of the package implied that, anyway). I asked Matt about removing packages, since we would want this removed from the laptops in El Salvador. If there is a Wikipedia Spanish package, we could install that instead.

Stuck in Customs

The 30 XOs which arrived in El Salvador two weeks ago are still stuck in customs. Raul and Douglas are both trying to resolve the problem. In the mean time, Douglas has installed Sugar on a Stick and is beginning to explore Sugar.

I sent Luke the LAPTOPS.DAT file I created on April 3rd. The same day he sent me back a develop.sig file with the developer keys for each laptop. Now I need to go to each one and install it.

I brought one home to try out the process. Here is how it went:

• opened the terminal activity
• su - root gave me a root prompt (#)
• put in the usb stick containing develop.sig
• used mount command to locate where the usb stick automounted
• copied develop.sig from the usb stick to /security on the XO
• rebooted, and at the ok message, I hit the escape key (x with a circle around it)
  
• typed disable-security and pressed enter (repeat this twice)

This all seemed to work without a hitch. Matt was right about there being no problem having the developer keys for all 11 XOs in the same develop.sig file. It appears the correct key was found and copied to firmware, where it will now happily allow this particular XO to boot off of SD cards and for unsigned versions of Sugar to be installed on it.

The next step was updating the version of Sugar that came on the laptop. For that I connected the XO to my wireless network and then followed the simple instructions here:

http://wiki.laptop.org/go/Olpc-update

I'll be looking into which version of Sugar makes sense for our project, but for now I just installed the latest stable build using the command:

• olpc-update 767

from a root prompt. The upgrade process took more than an hour, and I observed a few error messages along the way:

• rsync: failed to open "/versions/updates/60ab7dd13168405e4f9207c48a452a5a/usr/sbin/udevmonitor", continuing: No such file or directory (2)
• rsync error: some files could not be transferred (code 23) at main.c(1385) [generator=2.6.9]
• Update failed: Command '['/usr/bin/rsync', '-aOyz', '--no-r', '--dirs', '-l', '-c', '--delete', '--numeric-ids', '--no-whole-file', 'rsync://updates.laptop.org/build-767/root/usr/sbin/', '/versions/updates/60ab7dd13168405e4f9207c48a452a5a/usr/sbin']' returned non-zero exit status 23

It then gave me a new message to let me know it wasn't giving up:

• Trying irsync_dirty update from rsync://updates.laptop.org/build-767
• - Creating contents for existing tree.
• - Fetching contents for update tree.
• - Performing incremental rsync.

The install was started at 9:30 pm. At 12:30 am, it was still on the -Performing incremental rsync message. I went to sleep fearing it wouldn't work, but in the morning I was greated with the following when I touched the mouse pad on the laptop:

Verifying update.
Installing update in /versions/{pristine,run}/60ab7dd13168405e4f9207c48a452a5a
-bash-3.2#

After typing reboot at the root prompt, the XO restarted and completed the software update process and I was looking at a newer version of Sugar.

The XOs Have Arrived

The UPS shipment arrived at ACC (Arlington Career Center -- http://www.careercenter.arlington.k12.va.us) yesterday, bringing it's payload of 10 XO computers.

We are going to have a few immediate challenges:

• Getting developer keys on each of the 40 machines.
• Getting an language appropriate version of Sugar installed (English in Arlington, Spanish in Chalatenango).

I jumped on #olpc and found the channel very helpful. It looks like the best process for obtaining developer keys is described here.

Following the instructions, I:

• created a /boot directory on an empty USB drive.
• copied Actos.zip and Runos.zip into the /boot directory.
• rebooted each XO with the USB disk in the machine.
• watched the message indicating that the "Laptop data recorded successfully", after which it shut itself off.

OK, I've repeated this process with all 11 laptops. When I put the USB disk in another computer, I found a LAPTOPS.DAT file with one line for each laptop. I was wondering how I was going to remember which line went with which XO, but each line begins with the serial number, so it will be easy to figure out.

As a last step for today, I've attached the LAPTOPS.DAT file to an email to help@laptop.org requesting a developer key for each one.