Saturday, May 9, 2015

A Brief History of 22 Years as an OER Activist

Charles Severance (aka Dr. Chuck) just sent Allen Downey and me this video he shot of his first ever meeting with Allen at Pycon last month.  I had met Allen face to face for the first time only two years earlier at Pycon 2013, so I understood his thrill at the moment.

Both of us had created "remixed" textbooks based on Allen's original 1999 How to think like a computer scientist.  1999 was at the very beginning of what has become the open educational resource (OER) effort, and Allen was a true pioneer.  In translating his Java textbook into Python, I became his first follower, and helped him, in the manor described so well by Derek Sivers' How to start a movement, broaden his impact as an OER leader.

The book as since been mixed and remixed several times in several different languages, both programming languages and natural languages. There is even an on-line interactive edition.

Seeing Dr. Chuck's video made me reflect on my 22 year journey as an open educational resource activist, and I thought now might be a good opportunity to write a bit of that history down, so that as old age approaches, I won't forget it all. It has been a joyous journey, and I feel fortunate to have been a part in turn of the free software movement, the Python community, and the open educational resources project.

 

1994

I enrolled in a masters degree program at Bowie State University.  The computing environment there consisted of a VAX machine loaded with GCC.  GCC introduced me to the GNU project and the free software movement, which appealed immediately to my left-wing politics and knee-jerk egalitarianism.

Wanting to run GCC on my home PC, I found GNU/Linux. My first distribution was Yggdrasil followed by Slackware.

 

1995

I participated in a Summer workshop organized by Prince George's County Public School's visionary supervisor of Mathematics, Dr. Martha Brown, in which she brought together a group of math teachers from the school system to create our own educational resources.  The experience was both liberating and invigorating, but we failed to develop an effective means of distributing the materials we made once we were finished making them. It occurred to me that the World Wide Web, which was just coming into common use then, could provide such a mechanism for the collaborative development and distribution of educational materials. 

 

1999

Motivated by the College Board's switch from Pascal to C++, I went looking for a suitable programming language for our first year programming course and found Python.  Python was wonderful and the community was warm and welcoming, but at the time there were no textbooks using the language for introductory programming (and I don't think another high school anywhere in the world using Python).  Looking on the web, I found Allen Downy's Java textbook, How to think like a computer scientist.  HTTLCS was released under the GNU GPL, and I was thus free to make changes to it.  While I didn't feel competent to write a textbook from scratch, I did feel able to translate Allen's Java textbook into the Python textbook I needed, so that's what I did.  Soon after starting the translation process I setup the Open Book Project on ibiblio to host the work.

 

2007

The Knowledge Trust honored me with their Education Award.  I'm not usually one for pomp and circumstance, but I have always treasured this award, since I know it was given in the same spirit in which it was earned.  While I tend to be much more intrinsically than extrinsically motivated, it does feel good to be recognized for my efforts by folks for whom I have the deepest respect.

As I embark this year on new OER projects for Firefox OS, GIS, and MicroPython, it is great to have the opportunity to take a brief look back.

Wednesday, May 6, 2015

Getting Started with LIDAR Data

LIDAR, as I learned in my Geospatial Science Fundamentals course last Fall, is an active remote sensing technology that produces three dimensional representations of the earth's surface by gathering reflected laser light, usually from equipment mounted in an airplane passing over the modeled surface.  My Summer independent study at GMU will involve learning to process LIDAR data from the Northern Virginia region, with the aim of using the results to provide useful information to building owners in the region on the viability of putting solar panels on their roofs.

I will of course want to see if I can accomplish this task using free software, since citizen science is ill served by fees or licenses which restrict open collaboration on the widest possible scale.

I've been casually gathering information for the past few months, and here is what I've done so far:
  • read about the US National Lidar Dataset project and found out Virginia's dataset status is listed as "partial".
  • created an account on the USGS's EarthExplorer website.
  • searched for Loudoun County Virginia and filtered on Lidar, which returned a single data set with LiDAR Entity ID: VA_LOUDOUNCO_2012_000298.
  • downloaded the 76.7 MB "LAS Product" of this data.
  • handed over the data to one of my students, Sam Phillips, who is helping me with this project.
Sam's blog is here. We plan to use our blogs to both document our work and have a digital dialog of our efforts. The impetus for this project comes from the GIS manager of the Northern Virginia Regional Commission, Shafi Khan, who came to our Code for NOVA meet up to propose the idea. I took notes from our first on the project, which included the following:
Finally, to provide structure to the theoretical study related to the project, I plan to make my way through Penn State University's GEOG 481: Topographic Mapping with Lidar, which wonderfully includes OpenCourseWare for the course.  I've ordered the required text book and will begin working on the lessons this week.

My next post on this topic will describe my first experiments using Python to process LIDAR data...

Tuesday, May 5, 2015

Learning and Teaching Firefox OS

With only about five weeks left in the school year, and with my Spring semester grad school class completed, I'll now shift focus to three things which are all part of student projects this year, Summer internships, and technologies I hope to integrate into the IT program of Arlington Tech next year:
  1. Firefox OS app development
  2. Mapping LIDAR data with Python
  3. Robotics with MicroPython and the pyboard
I will document each of these efforts in separate blog posts.



First on the list is Firefox OS app development.  I've written about my early enthusiasm for FXOS in a previous post.  The challenge now will be to gather and organize a collection of accessible and high quality resources to support incorporation of FXOS into a high school / community college curriculum.

The principal obstacle in meeting this challenge is that Firefox OS is a rapidly developing technology, so in attempting to write educational materials for it you are aiming at a moving target.  I've already found this to be a problem with existing materials.  One of the best written tutorials I've found thus far for FXOS is on the blog iRomin.  Written as a series of 10 episodes, Episode 1 : Setting up your Development Environment, is already woefully out of date.  It describes setup procedures that have already been replaced by better and easier to use technologies.

It seems important to me to get folks who want to help nurture the fledgling OS to release their books, blogs, and tutorials as open educational resources. I certainly will, but my life as a free software activist would be made so much easier if I could build on the work of others as I set about creating a curriculum for FXOS.  I'll be contacting the authors of the usable materials I find and asking them if I can update and adapt their work.

Here is a list of resources I plan to start with:
One of the first things I want to do is create a Firefox OS app "zoo", along the lines of the jQuery Zoo. The idea is to populate the site (and accompanying github repo) with a collection of small, easy to understand apps that are designed primarily to meet pedagogical goals.  I should start with a "hello world" app and build from there.

I've got a student coming in tomorrow morning for help getting started with FXO, so I best get working on the app zoo now...


Saturday, May 2, 2015

Geospatial Representation of Roads

It turns out that the polygons in the Roads shapefiles do not represent roads the way I thought they would.  To explore the shapefile, I wrote a script using pyshp: explore_shapefile.py.  The pyshp module is definitely more rewarding to use than GDAL, since it is pure Python and it is thus possible to use Python's type introspection to explore.  I've found that all of the scripts I've written in Python 3 will run also in Python 2, which is convenient.  I also came across one of the big improvements Python 3 has over Python 2 in terms of type introspection.

The following line of code prints out the type of the first element of a list from a shapefile: 

print('type(sr[0]): {0}'.format(type(sr[0])))

Running the script in Python 2 results in: 

type(sr[0].shape): <type 'instance'>

which is not very informative.  Python 3 does much better: 

type(sr[0]): <class 'shapefile._ShapeRecord'>

Anyway, since my final project is due tomorrow, I had to cut the investigation into using pyshp short, but I'll be sure to continue looking into it soon.

My goal was to represent Columbia Pike in bright yellow or orange on my map. Columbia Pike is the road on which the proposed streetcar that had such a profound impact on the Vihstadt / Howze election ran.  I thought adding it to the map would provide visual interest.  It also gave me a chance to explore adding more than one layer to a map.

I wrote several scripts to make maps as part of investigating this.
draw_roads.py
Renders all 883 shapes from the Roads shapefile in a map.
a_road_shape.py
Renders a single shape from the Roads shapefile as a new layer on the precincts map.
roads_near_columbia_pike.py
I used trial and error (and a bit of divide and conquer) to find shapes among the 883 that represented roads on and near Columbia Pike.
election_map_with_roads_near_cp.py
Finally, I added this roads layer to the election results map I made yesterday.
The first script, draw_roads.py, generates a map of the roads in Arlington County:
I have yet to begin studying QGIS, but I googled how to load a shapefile as a vector layer to see how it would compare:
Since I generally feel more comfortable expressing my thoughts in code rather than using tools, I think I'm going to generally prefer writing python scripts to using qgis to make maps, but I realize I'll have to do both.

I assumed that each of the 883 shapes in the Roads shapefile represented a single road.  I was disappointed that the metadata didn't include names associated with each of the shapes, but I figured I could use a kind of binary search to print out half of them, render the map, and see which half Columbia Pike was in, cut that in half, and so forth, until I found Columbia Pike.  Soon I'll be able to read geospatial coordinates from the shapes and write a script that selects the shapes I want, but for now I figured this approach would be the quickest way to get to my goal.

It turns out that the reason there are no road names associated with the shapes is that each shape represents a cluster of connected road segments rather then a single road.  a_road_shape.py illustrates this:
The orange line segments are from a single shape from the Roads shapefile.  Once I realized that, I spent some time gathering shapes that had a piece of the Pike on them, and made roads_near_columbia_pike.py to render them:
Finally, I merged this with the color coded election results map I made yesterday in election_map_with_roads_near_cp.py to produce:
The color contrast is bad here, but I'm running out of time to complete the project, and since this fails to render just Columbia Pike the way I wanted anyway, I decided not to spend any more time on it.

Where I'll Go From Here

OpenStreetMap is an incredible resource for open source mapping projects, and yesterday I did a quick search to see if it might provide me with a solution to my rendering Columbia Pike project.  I found out that OSM stores streets as a collections of ordered nodes called ways. This way holds a piece of Columbia Pike near where I live:

https://www.openstreetmap.org/way/334620323#map=18/38.85606/-77.10834

Here are the things I think I should investigate next:
My immediate project goal will be to render Columbia Pike, this time from OSM, onto the election map as I had originally intended.  I anticipate that the pyproj module will be the tool to use to convert among different geospatial data formats.

Friday, May 1, 2015

A Color Coded Election Results Precinct Map

Learning to use Python to make maps is becoming more fun the more time I spend with it.  This morning I was able to successfully generate a color coded precinct map showing which precincts were won by Alan Howze and John Vihstadt in last November's Arlington County Board election.

Here is what I did:
  1. Found a data set containing the election data.  The Virginia Department of Elections website has a CSV file with the data I needed here.
  2. Learned to use Python to extract the data I needed from this source.  This turned out to be surprisingly easy (maybe not so surprising, that's why I love Python!). I just needed the cvs module which comes with the standard library.  I then wrote a small script, precinct_winners.py, which produced a list containing the winner for each precinct.
  3. Wrote a script, vihstadt_howze_election_map.py, using mapnik and my precinct_winners.py script to generate a color coded map showing precincts which Howze won in green and those Vihstadt won in red.
Here is the resulting map:


I plan to spend the rest of my available time today trying to overlay Columbia Pike, the road along which the proposed streetcar that caused such a stir in this election had been planned to run.

I found a shapefile with all the roads in Arlington County here.  I'm planning to explore this file with the pyshp rather than the GDAL module, since pyshp is pure python instead of a wrapper around a C++ library and thus will be easier to work with.

My next post will report on my results.

Thursday, April 23, 2015

Drawing My First Map with Python

I was able to use python to read data from a shapefile and generate a map of voter precincts in Arlington County, Virginia.

Arlington County, Virginia makes numerous maps available on-line here.  The specific shapefile I started with holds boundaries for voter precincts in the county, and is available here.

I used the Voter_Precinct shapefile and adapted what I learned about gdal and mapnik from Python Geospatial Development. Specifically, the following chapters contained the examples I needed for this first map:
  • Chapter 3 - Python Libraries for Geospatial Development
  • Chapter 5 - Working with Geospatial Data in Python
  • Chapter 8 - Using Python and Mapnik to Produce Maps
I also read through the ESRI Shapefile Technical Description.

The Scripts

I wrote the following two scripts, which are now located in my gis_front_and_back repo on github:
The README.md file, also in the github repo, contains a link to where to get the shapefile.

Since I use Python 3 whenever I can, the first script uses Python 3, while the 2nd, which requires mapnik, is written in Python 2.  Running the first script and redirecting the output to a file produces this.

Running the 2nd script generates this map:



Making this map by adapting the example scripts from the text was fairly trivial.  I've been unable to get filters to work thus far, however, which will enable me to vary the colors of the individual precincts based on rules.

I've been wanting to make a map ever since last November's election which will color code the results by precinct in the race between John Vihstadt and Alan Howze (see John Vihstadt beats Democrat Alan Howze in race for Arlington County Board seat).  The key issue in this race was the proposed Columbia Pike Streetcar, which was canceled soon after the election (see Arlington Cancels Streetcar Program).

I grabbed precinct election results from the elections.virginia.gov website and saved them in a PDF file which I still have here. I was struck by how the support for the streetcar was clearly visible near the communities most effected by it (along the Columbia Pike corridor), while opposition (assuming a direct correlation between opposition to the streetcar and support for Vihstadt) was strongest in North Arlington, furthest away from where the streetcar would have run. This kind of data cries out for a good map, so I've been wanting to make one ever since.

I just downloaded the complete election result data in CVS format from here.  My most immediate problem now is to figure out how to color individual precincts based on filters.  I'll also look further into how to manipulate shapefiles, perhaps using the pyshp library, which can be installed on Ubuntu through the python-pyshp and python3-pyshp debian packages for versions 2 and 3 of Python respectively.  Then I'm thinking I'll either figure out how to read the election result data from the CVS file, or use the PDF to read the election results for each precinct and to color code them on the map accordingly.

Wednesday, April 22, 2015

The pyboards Have Arrived!

Arlington Tech, the name we have informally given to the new school being planed for the current Arlington Career Center, will have a focus on STEM ("green STEM" in particular).  Our carpentry instructor has both experience with and a big interest in high school robotics, and since robotics is an inherently interdisciplinary endeavor, I'll be working with him next year to help students learn to program the robots.

When I heard about the programmable microprocessor that runs Python 3 called the pyboard at Pycon a last week, I was naturally interested in investigating it as a possible tool for our robotics effort.

I ordered two pyboards, which arrived today.  Here is a picture of one of them:



Here is a screen shot of one of the example scripts from this tutorial, which prints out value from the accelerometer on the board:


I have to stay focused on GIS programming and Firefox OS, but I have a friend interested in microprocessors who plans to explore this further.