Migrating from VirtualBox to KVM/Qemu

I've been using VirtualBox for years on my Ubuntu systems to test and use other operating systems on the host machine.  Since I am working with five of my students this year to help them prepare for the RHCSA exam, I've been learning to use KVM for virtualization instead. KVM with virt-manager has proven to be easier to use than VirtualBox, since I don't have to install any add-on video drivers to get screen resolutions changed, and it seems to run faster and with less resource overhead than VirtualBox.

First thing I did was install:

$ sudo apt install qemu-kvm libvirt-bin bridge-utils
$ sudo ap install virt-manager

Then I added myself to the libvirtd group, so that I would have access:

$ sudo adduser [user] libvirtd

I converted my VirtualBox hard drive images to KVM images with:

$ qemu-img convert -f vdi oldImage.vdi -O qcow2 newImage.qcow

Here is a screenshot of Debian Jessie running on Ubuntu Yakety:

Resources

• How to Install KVM and Create Virtual Machines on Ubuntu
• Converting a virtual machine from VirtualBox to KVM

Welcome to PIXEL!

A little over a week ago an old friend from the OLPC Learning Club DC, Kim Toufectis, dropped by our Thursday night Code for NOVA meetup and we took to chatting about what's going on in the world of the free software education community. A group of us at the Code for NOVA meetup have been exploring the Raspberry Pi. While admittedly a bit far removed from Code for America's civic hacking mission, the Raspberry Pi and the growing community around it are providing wonderful educational resources for learning to program in Python, and learning to use and manage GNU/Linux computer systems, both of which will be extremely useful for folks coming to our meetup wanting to acquire the skills they need to contribute to civic hacking projects.

From Raspbian to Debian + PIXEL

What Kim told me about that excited me the most was the recently released i386 port of the desktop used on the Raspberry Pi. I've been following the Raspberry Pi since I came home from Pycon 2013 with 4 of them. What makes this new port a potential game changer is that it would allow me to run the same desktop on both the computers in my classroom and on the Raspberry Pi's. This means that the Pi's amazing collection of educational resources could be used without change on my lab computers.

At present, i386 Debian + PIXEL is only available as a live image that boots from either DVD or USB.  I want to be able to install it on the hard drive of lab machines, and indeed I have an old Dell laptop that won't boot from USB which has only a CDROM drive, so neither of the available options would work on this machine. I also want to be able to run it in virtual machines using KVM on my Ubuntu hosts. With the help of two resourceful students, I developed the following first draft of a process by which to do this.

Installing the Debian + PIXEL Desktop on a KVM Virtual Machine

1. Start with a base Debian 8 (Jessie) i386 install

I used a netinstall iso image, which is both really small and which provides maximum flexibility, as long as you have an available network connection on the machine on which you are installing.

Arlington Career Center

Since I don't know yet which of the general option offered by the installer provide which specific software, I decided to deselect everything and use the minimal installation offered.  I allocated 1 Gig of RAM, 1 processor, and 20 Gigs of virtual hard drive space to this machine.

2. Login to the completed virtual machine as root and run the following commands:

# apt install ssh sudo vim
# add user [username] sudo
# ifconfig

I got the IP address of the virtual machine from ifconfig, I used ssh to connect to it from a GNOME Terminal on the host machine, so that I could easily copy and paste the rest of the commands I wanted to run.  Here is a screenshot showing the virtual machine running in the Virtual Machine Manager and the GNOME Terminal session preparing to ssh into it:

3. Get the gpg key and setup the apt repositories. Run:

Arlington Career Center

$ wget http://archive.raspberrypi.org/debian/raspberrypi.gpg.key
$ sudo apt-key add raspberrypi.gpg.key

 Then set the contents of /etc/apt/sources.list to:

deb http://httpredir.debian.org/debian/ jessie main contrib non-free
deb-src http://httpredir.debian.org/debian/ jessie main contrib non-free
deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free
deb http://httpredir.debian.org/debian/ jessie-updates main contrib non-free
deb-src http://httpredir.debian.org/debian/ jessie-updates main contrib non-free

and set the contents of /etc/apt/sources.list.d/raspi.list to:

deb http://archive.raspberrypi.org/debian/ jessie main ui staging
# Uncomment line below then 'apt-get update' to enable 'apt-get source'
#deb-src http://archive.raspberrypi.org/debian/ jessie main ui

4. Install the PIXEL desktop. Run:

$ sudo apt update
$ sudo apt dist-upgrade
$ sudo apt install desktop-base pix-plym-splash
$ sudo apt install lxde dhcpcd-gtk
$ sudo apt install pi-greeter pi-package pimixer pipanel
$ sudo apt install raspberrypi-artwork raspberrypi-net-mods
$ sudo apt install raspberrypi-sys-mods raspberrypi-ui-mods
$ sudo apt install chromium

 After this, I exited from the ssh session and rebooted the virtual machine, again accessing it through the Virtual Machine Manager.  Here is what greeted me:

Logging in revealed:

After running LXRandR and tweeking the LXTerminal to my liking, I arrived at:

This is only a very rough process at this stage. The menus will require configuration to match what students see on both the Raspberry Pi and the Live image of Debian + PIXEL.

The next step for us at the Arlington Career Center will be to reach out to the Raspberry Pi community and see if we can begin contributing directly to the project. The OLPC Learning Club DC was the most exciting user group in which I've ever had the pleasure to participate.  It brought together an eclectic mix free software geeks, scientists, hobbyists, and parents with their young geniuses to be into the kind of gather that just leads to great things.  I'm hoping that the local Raspberry Pi community we are beginning to create can at least approach the magic of that experience.

AP CS Principles - The College Board Gets It Right

Beginning this Fall, students in high schools across the United States were able to will be able take a new course - AP CS Principles.  I can't say I've always been a big fan of the College Board and the impact this very powerful institution has on education in my country. I have to say, however, that I think they got it right this time. From my experience thus far, I am a huge fan of the new CS Principles course.

Working with the NSF, the College Board has approved three on-line curricula for the launch this Fall:

• The Beauty and Joy of Computing
• Mobile CSP
• Thriving in Our Digital World

Last Spring I used the Mobile CSP curriculum with students who were taking dual-enrolled CSC 200: Introduction to Computer Science with me through Northern Virginia Community College.  While I haven't researched the details of its history, it is obvious that CSC 200 was created by the Virginia Community College System (VCCS) to receive credits from the new AP CS Principles course.

As a free software activist of many years, I found little gems inside the Mobile CSP curriculum that let me know kindred spirits where involved in its creation:

This Fall I am again teaching CSC 200 using Mobile CSP. The more I use App Inventor the more I love it.  While I've been too busy to keep up with this blog of late, it is time to get back to more regular posting so I can document our experience this year with this wonderful new curricula.

My Annual Love Letter to SchoolTool

It is the end of August, and time once again for me to setup my SchoolTool instance for the coming school year. Each year for the last several years, I have sent an email to the SchoolTool mailing list expressing both my joy and my deep felt gratitude at the pleasure I experienced setting up my very own student information system (SIS).  Sadly, we may have come to the end of an era, and SchoolTool's days may be numbered, so I felt it more appropriate this year to express my gratitude here in my blog.

The reason I feel compelled to write a "love letter" to the SchoolTool developers each year is that they have made the very complex process of setting up an SIS smooth and painless, both by the design of the software itself, and by the wonderful SchoolTool Book written by English teacher and former project manager, Tom Hoffman.  It provides a model of what technical documentation should be, and the influence that the documentation process and the user interface design process had on each other is apparent. With nothing more than the book to guide me, I can create a school year, populate it with courses, sections, terms, time tables, instructors, students, and skills. In just a few minutes I am ready to start the new school year. The ease with which everything works is a total thrill!

I teach dual-enrolled high school / community college classes in a career and technical school in Arlington, Virginia. The SIS provided by Arlington Public Schools is not setup to properly handle the odd configurations of overlapping high school and college classes that I need to make my CS / IT program work.  SchoolTool provides me with a customizable SIS that meets my needs, while providing a host of added benefits to my students.  The CanDo Skills Tracking system lets students see their progress on the explicit skills they are expected to acquire in my classes. The SchoolTool Quiz component enables me to create custom tests and quizzes automatically linked to the skills tracking system.

SchoolTool will forever epitomize what free software means to me and why I've dedicated much of my energy over the past 20+ years fighting for software freedom. It was created in an open process with input throughout by the real users of the system.  I became one of those users back in the Summer of 2005, when I met Tom Hoffman at Pycon and began the collaboration that continues to this day. Both the CanDo and Quiz components were added at the initiative of a small group of us in Virginia, and students of mine contributed directly to the development of both components. There is still no other software, either free or proprietary, that does what SchoolTool does.

The development of SchoolTool was driven by the desire to provide use value and to create a tool to help change the world, specifically by positively impacting education in Africa and the developing world. Thanks to the sponsorship of Mark Shuttleworth, it was freed from the imperative of commodification for an extended period of time. It never could have developed the innovations it has otherwise.

Some big mistakes where certainly made along the way to SchoolTool's development, the biggest of which is probably building SchoolTool on a dead end web application framework, Zope 3, which has now isolated SchoolTool from the larger free software development community. That mistake may cause SchoolTool to follow its framework into abandonment. You never can tell with free software projects, however, since they can be taken up by anyone at any time who finds them useful.  Perhaps SchoolTool will find new life in some unexpected way, or perhaps some of its innovations will find their way into another free software SIS.

I am just happy that it will be available for at least the next several years on Ubuntu servers.  I plan to keep using it for as long as I can, since it is a tremendously effective resource to help my students monitor their learning, and since I get to experience the joy and excitement each year of setting it up!

Setting Up a RHCSA Practice Laptop - Part 1

In order to practice for the RHCSA at home, I took an old Dell Latitude E6500 with 4GiB of RAM and purchased a 500GiB hard drive on which I could install CentOS 7 with a server GUI as the base operating system and them multiple virtual machines using KVM with which to experiment and learn.

My package selection for the install was a Server GUI installation. I partitioned the hard drive with 1 GiB of swap, 500 MiB on a standard partition for the /boot, and 30 GiB for the root partition.

I also created a 60 GiB partition for /home, and then allocated everything that remained (375.27 GiB) to /var.  The reason for giving some much space to /var is that the default KVM / qemu setup on CentOS 7 places virtual hard drive images in /var/lib/libvirt/images, so I wanted plenty of space available for multiple images.

After the install finished I wanted to see what the partitions looked like, so I ran

$ sudo yum install system-storage-manager

and then

$ sudo ssm list

which revealed the following:

-----------------------------------------------------------------
Device        Free       Used      Total  Pool        Mount point
-----------------------------------------------------------------
/dev/sda                       465.76 GB              PARTITIONED
/dev/sda1                      500.00 MB              /boot
/dev/sda2  0.00 KB  465.27 GB  465.27 GB  centos
-----------------------------------------------------------------
--------------------------------------------------------
Pool    Type  Devices     Free       Used      Total
----------------------------------------------------
centos  lvm   1        0.00 KB  465.27 GB  465.27 GB
----------------------------------------------------
--------------------------------------------------------------------------------
Volume           Pool   Vol size  FS   FS size    Free      Type   Mount point
--------------------------------------------------------------------------------
/dev/centos/swap centos 1.00 GB                             linear
/dev/centos/root centos 30.00 GB  xfs  29.99  GB  26.73 GB  linear /
/dev/centos/home centos 60.00 GB  ext4 60.00  GB  55.88 GB  linear /home
/dev/centos/var  centos 374.27 GB ext4 374.27 GB 349.02 GB  linear /var
/dev/sda1               500.00 MB xfs  493.73 MB  293.59 MB part   /boot
--------------------------------------------------------------------------------                                                                          

To be continued in Fall of 2016..

Resources

• How to manage LVM volumes on CentOS / RHEL 7 with System Storage Manager
• KVM Virtualization in RHEL 7 Made Easy

Moving an ArcGIS File Geodatabase to QGIS

I am taking GGS 553: Geographic Information System this semester at part of my graduate studies at George Mason University.  In a previous post I described how I ended up in this Geographic Information Science graduate certificate program, which I have now been pursuing for almost 2 years.  GGS 553 is a required course, and the first one in the program that has required me to use proprietary software, since much of the course is focused on learning to use ArcGIS.

I am both philosophically and ethically opposed to proprietary software, since it runs dead against the expansion of our shared cultural space, which I believe is vital to the survival of our species. This is a required course, however, and in the large scheme of things I am willing to compromise when I need to. I like to think of it as dancing with the devil, learning the devil's moves in order to be able to freely out dance him in the future. In this case that will mean applying what I learn in GGS 553 to mastering QGIS, the free software alternative to ArcGIS. I had intended to try to do each of our assigned labs this semester in both ArcGIS and QGIS, but when I found it difficult enough just to complete them on time in ArcGIS, I gave up on that idea after the first week.

This week we have a sort of half size assignment, so I thought I would use the extra time available to see if I could do it in QGIS.  The first challenge will be to load the project data into QGIS.  We were given the data in ArcGIS's file geodatabase format. QGIS can not yet read and write to this format directly, but there are tools available to convert it into PostGIS, with which QGIS can work well.

Last Summer I wrote a blog post documenting how I setup a PostGIS server on Ubuntu 14.04.  Since this year I am also needing to learn RHEL, I'll use this guide to setup the server on the little Centos 7 server I have at home for just such purposes, and then connect to it from QGIS running on my Ubuntu desktop.

Installing a PostGIS Server on Centos 7

$ sudo yum install postgis postgresql-server postgresql-contrib
$ sudo postgresql-setup initdb
$ sudo -i -u postgres
$ psql
postgres=# \password postgres
Enter new password: 
Enter it again: 
postgres=# \q
$ exit
$ sudo vi /var/lib/pgsql/data/pg_hba.conf

Change this line (near the bottom):

host    all             all             127.0.0.1/32            ident

to this:

host    all             all             0.0.0.0/0               md5

Next allow database connections from outside:

$ sudo vi /var/lib/pgsql/data/postgresql.conf

Change:

#listen_addresses = 'localhost'

to this:

listen_addresses = '*'

Create a new database user with superuser privileges:

$ sudo su - postgres
$ createuser --superuser [user]
$ psql -c "ALTER ROLE [user] PASSWORD '[password]'"
$ exit

Then as that user create the database and add gis extensions:

$ createdb webster
$ psql -d webster -c 'CREATE EXTENSION postgis'

Then after copying over the Webster.db directory containing the file geodatabase, I ran:

$ ogr2ogr -f "PostgreSQL" PG:"dbname=webster user=[user] password=[password]" Webster.gdb

After which I connected my desktop QGIS to the PostgreSQL server running on my little household server and loaded the three layers I found there:

Resources

• Working with File Geodatabases (.gdb) using QGIS and GDAL
• How To Install and Use PostgreSQL on CentOS 7
• How To Set Up a Firewall Using FirewallD on CentOS 7
• How to migrate GDB data into PostGIS without ESRI apps?

Software Management with YUM

YUM (Yellowdog Updater, Modified) is the package management tool used on Red Hat Enterprise Linux and its derived versions, CentOS and Scientific Linux. It acts as a front end to the RPM Package Manager (RPM), and is used to install, remove, and update software on Red Hat based systems.

I first encountered YUM when installing Yellow Dog Linux on PowerPC based Macintosh computers back at the dawn of the 21st century.  When I switched over to Debian based GNU/Linux systems with the release of Ubuntu in 2004, I completely lost touch with the RPM world until my Spring semester Linux System Administration course's pursuit of RHCSA certification brought me back into the fold.

I am writing this post to use as a handy list of the most common things I need to do when managing software:

1. Update the software on the system
   

   $ yum check-update
   $ sudo yum update package_name
   $ sudo yum update [to update all packages]
   $ sudo yum group update group_name
   

2. List all the currently installed software
   

   $ yum list installed
   $ yum list installed "global expression"
   

3. Search for available packages
   

   $ yum list available "global expression"
   $ yum search term...
   

4. Display information about a package
   

   $ yum info package_name
   

5. Install a new package
   

   $ sudo yum install package_name
   

6. Remove an existing package
   

   $ sudo yum remove package_name
   

7. List the current repos
   

   $ yum repolist
   $ yum repolist -v
   

That covers the basics. I also need to learn how to clean up the cruft that accumulates over time as a system is run, in Debian land the kind of thing that would be done with $ sudo apt-get autoremove. It seems that in RPM space that is accomplished with the package-cleanup utility, so I'll look into that.

Resources

• YUM Chapter from Red Hat System Administrator's Guide
• How to clean installed rpms using package-cleanup