Drawing My First Map with Python

I was able to use python to read data from a shapefile and generate a map of voter precincts in Arlington County, Virginia.

Arlington County, Virginia makes numerous maps available on-line here.  The specific shapefile I started with holds boundaries for voter precincts in the county, and is available here.

I used the Voter_Precinct shapefile and adapted what I learned about gdal and mapnik from Python Geospatial Development. Specifically, the following chapters contained the examples I needed for this first map:

• Chapter 3 - Python Libraries for Geospatial Development
• Chapter 5 - Working with Geospatial Data in Python
• Chapter 8 - Using Python and Mapnik to Produce Maps

I also read through the ESRI Shapefile Technical Description.

The Scripts

I wrote the following two scripts, which are now located in my gis_front_and_back repo on github:

• analyze_voter_precinct_shapefile.py
• make_precinct_map.py

The README.md file, also in the github repo, contains a link to where to get the shapefile.

Since I use Python 3 whenever I can, the first script uses Python 3, while the 2nd, which requires mapnik, is written in Python 2.  Running the first script and redirecting the output to a file produces this.

Running the 2nd script generates this map:

Making this map by adapting the example scripts from the text was fairly trivial.  I've been unable to get filters to work thus far, however, which will enable me to vary the colors of the individual precincts based on rules.

I've been wanting to make a map ever since last November's election which will color code the results by precinct in the race between John Vihstadt and Alan Howze (see John Vihstadt beats Democrat Alan Howze in race for Arlington County Board seat).  The key issue in this race was the proposed Columbia Pike Streetcar, which was canceled soon after the election (see Arlington Cancels Streetcar Program).

I grabbed precinct election results from the elections.virginia.gov website and saved them in a PDF file which I still have here. I was struck by how the support for the streetcar was clearly visible near the communities most effected by it (along the Columbia Pike corridor), while opposition (assuming a direct correlation between opposition to the streetcar and support for Vihstadt) was strongest in North Arlington, furthest away from where the streetcar would have run. This kind of data cries out for a good map, so I've been wanting to make one ever since.

I just downloaded the complete election result data in CVS format from here.  My most immediate problem now is to figure out how to color individual precincts based on filters.  I'll also look further into how to manipulate shapefiles, perhaps using the pyshp library, which can be installed on Ubuntu through the python-pyshp and python3-pyshp debian packages for versions 2 and 3 of Python respectively.  Then I'm thinking I'll either figure out how to read the election result data from the CVS file, or use the PDF to read the election results for each precinct and to color code them on the map accordingly.

The pyboards Have Arrived!

Arlington Tech, the name we have informally given to the new school being planed for the current Arlington Career Center, will have a focus on STEM ("green STEM" in particular).  Our carpentry instructor has both experience with and a big interest in high school robotics, and since robotics is an inherently interdisciplinary endeavor, I'll be working with him next year to help students learn to program the robots.

When I heard about the programmable microprocessor that runs Python 3 called the pyboard at Pycon a last week, I was naturally interested in investigating it as a possible tool for our robotics effort.

I ordered two pyboards, which arrived today.  Here is a picture of one of them:

I searched up a quick tutorial on using it, and found Micro Python and pyboard - a Python running on a microcontroller scripts your electronics.

Here is a screen shot of one of the example scripts from this tutorial, which prints out value from the accelerometer on the board:

I have to stay focused on GIS programming and Firefox OS, but I have a friend interested in microprocessors who plans to explore this further.

Installing Python GIS Libraries on Ubuntu 14.04

I'm working my way through Python Geospatial Development, which as good fortune would have it, is the book we are using in my Introduction to GIS Algorithms and Programming course at George Mason University.  At the beginning of the course we were given a VirtualBox image of Ubuntu 12.04 with all the software and data files we need for the course preloaded.  Since I'm already running Ubuntu 14.04 as my regular desktop OS, I preferred to just install the packages (all of which are in the standard debian repositories) onto my desktop machines.

Here are the libraries we are using:

• GDAL - The Geospatial Data Abstraction Library (see Python GDAL/OGR Cookbook for Python specific "recipes") for reading and writing geospatial data (GDAL for raster, OGR for vector).
• pyproj - Performs cartographic transformations and geodetic computations.
• Shapely - Manipulation and analysis of geometric objects in the Cartesian plane.
• Mapnik - Toolkit for building mapping applications.

To install them for Python 2 on Ubuntu 14.04, run the following in a unix shell:

• $ sudo apt-get install python-gdal
• $ sudo apt-get install python-pyproj
• $ sudo apt-get install python-shapely
• $ sudo apt-get install python-mapnik2

Even on Ubuntu 15.04 Mapnik is only available for Python 2.  The other three libraries (gdal, pyproj, and shapely) are all available for Python 3.  This means that I will unfortunately have to use Python 2 for my mapping project.

I've now read the first eight chapters of Python Geospatial Development, which contain all the information I'll need for my project:

• Chapter 1 - Geospatial Development Using Python
• Chapter 2 - GIS
• Chapter 3 - Python Libraries for Geospatial Development
• Chapter 4 - Sources of Geospatial Data
• Chapter 5 - Working with Geospatial Data in Python
• Chapter 6 - GIS in the Database
• Chapter 7 - Working with Spatial Data
• Chapter 8 - Using Python and Mapnik to Produce Maps

It is often quite difficult to find good books about rapidly developing technologies like this, but this book does a good job.  The material is relevant, the presentation clear, and the examples both engaging and illuminating.

With background reading done and infrastructure setup, it is time to dig into the project.

Firefox OS and Lowering the Barrier to Entry into ICT

I'm recently returned from Pycon 2015, and still feeling the energizing, rejuvenating effects of this annual pilgrimage to connect with a community I love.  The YouTube channel for the conference talks is here.

Two of the keynotes were especially memorable.  The keynote by Catherine Bracy, Director of Community Organizing for Code for America, left me feeling fortunate indeed to be involved with our local Code for NOVA brigade.  Catherine laid out in clear terms Code for America's vision of transforming the way citizens relate to their government (and the increasing alienation many feel from government) by bringing the community together to work collaboratively to address local needs.

The keynote by Jacob Kaplan-Moss discussed how sexism and the myth of the rock star / ninja programmer where keeping people from entering the programming field and driving women out of it after they do enter.  By describing the ninja programmer myth as an inverted normal distribution, he presented both a memorable image and a clear exposure of the absurdity of the myth.

Lowering barriers to entry into programming is my raison d'être as an ICT/CS teacher in a school serving lower income, immigrant students.  If I can't find a way to lower the barrier to entry, there is really no good reason for me to come to work each day.  It is with that in mind that I have been continuously thrilled by the wonderful work the folks at Mozilla are doing with their development tools for Firefox OS.

The following three files are from the first app introduced in chapter 2 of Firefox OS in Action:

• note.html
• note.css
• note.js

Here is what this code looks like running in a browser (with the widow suitably shrunk to a small size):

 adding one more file turns this into a Firefox OS app:

• manifest.webapp

Here is the app loaded in the WebIDE and running in the simulator:

The deployment story here is simply amazing.  I've never seen anything like it.  The WebIDE comes with Firefox, so there is nothing to install.  Simply press <Cntrl>+<F8> to launch it.  A few clicks of the mouse and a simulator with your choice of version of Firefox OS is running, and deploying your app to the simulator is a matter of loading the app source into the WebIDE and clicking a single button!

It can't get any better than that, right?  Wrong!  The best is yet to come.  What if you want to deploy your app to a real phone?

To show you how amazingly simple this is, let me first introduce a more interesting app, the LibriFox audio book player that two of my students are working on:

• LibriFox Git Repo

To deploy this app to my Firefox OS phone, I simply plug in the phone with the USB cable, and it now shows up in the list of available targets:

Firefox OS(Flame) is my phone

Clicking the same triangle button I used to deploy to the simulator loads the app onto my phone.  Wow!

I've never seen a development environment that even approaches the simplicity and elegance of this one.  The Mozilla Foundation is delivering on its goal of "making the Web better and more accessible for everyone everywhere".  I'm an ICT teacher just getting started in mobile app development, but I'm hooked on Firefox OS already.  By lowering the barrier to entry into this important technology, Mozilla will surely attract new developers far and wide, promising great things for the future of this free software phone operating system.

Thank you, Mozilla!

Serendipitously Approaching GIS from Both Ends

"Serendipity" is the word I think most fitting to describe how I ended in a Geographic Information Sciences Graduate Certificate program at George Mason University.  In brief, after getting involved with Code for NOVA with the aim of finding real world learning opportunities for my students and hearing the folks there talking enthusiastically about publicly available data sets, I decided to enroll in GGS 550: Geospatial Science Fundamentals during the Fall 2014 semester.  This course got me totally hooked on GIS!  It provides me with a way to apply my mathematics and computer science degrees to real problems.

It seems like my timing has been wonderfully serendipitous as well.  A few years ago it appears I would have been limited to proprietary software and military industrial applications. Now it's free software and citizen science.  I mentioned last post how my current GIS course is a python programming course.  I'm beginning work on my final project for that course:

Mapping Housing Affordability in Arlington Virginia

Generate maps of Arlington Virginia which show the location of affordable housing units, both committed affordable (CAFs) and market rate affordable (MARKs).  If time permits, I'd like to be able to produce a map that shows changes in availablility over time (availability is rapidly declining), but that may be beyond what I can do in our available time.

My goal is to do this using Python with the needed libraries.  Our Python Geospatial Development book addresses everything I would need for this project, but it will require finishing all of the book, since working with shape files isn't discussed until chapters 11 and 12.

Since the project is due May 3rd, I will only be able to get started with the full map by that time, but if I begin to generate custom maps of Arlington and to place geospatial data on them (housing locations) together with relevant meta-data, I'll be confident from there that with time and can map whatever maps I want.

We are using Python Geospatial Development as our text, and I'm working my way through all the examples, converting them from Python 2 to Python 3. I'll be documenting what I'm learning in posts over the next several weeks.

With python libraries and things like GeoDjango to take care of the back end, I'm looking at Firefox OS for front end mobile application development.  With built-in GPS and other sensors, mobile devices are great tools for citizen scientists, and Firefox OS's mission of expanding access to information is just what I was looking for.  I've got several students working on Firefox OS projects this Spring.  I've started reading Firefox OS in Action with an eye toward both learning to program on the platform myself and to using this book as a resource with students.  I'll be documenting that as over the next several weeks as well.

Python GIS and Firefox OS

I haven't made a post since January, but all that is about to change.  A lot has happened in the past 3 months:

1. After several false starts looking into app development for Ubuntu phones, I came across Firefox OS, with its supportive, community friendly staff and real focus on community building and education. Three of my current students are now working on Firefox OS related projects, and being mentored by Mozilla employees in the process.
2. My Spring graduate GIS class at George Mason University, titled "Introduction to GIS Algorithms and Programming" turned out to basically be a Python programming class with a focus on GIS (how cool is that!? ;-) As a result, my love affair with GIS has only deepened.  I will be doing an independent research project over the Summer that will involve applying and extending what I am learning in this class to a project involving using LIDAR data to inform Northern Virginia residents about the viability of putting solar panels on their homes.  I will be spending a two month "retreat" with my 100 year old grandmother in New Jersey immersing myself in both this project and in developing curriculum materials for teaching mobile app development with Firefox OS.
3. Our Code for NOVA brigade continues to grow and is reaching the size needed to provide multiple opportunities for real synergies between developers (both experienced and inexperienced), and local, regional, and national government agencies looking for help with IT tools to serve the public and conduct citizen science projects.  Our National Science Foundation host continues to make incredible learning opportunities available to Arlington Public Schools students.

I'll begin posting to this blog much more regularly now, as I document progress on each of the above.  Stay tuned...