Sunday, April 19, 2015

Installing Python GIS Libraries on Ubuntu 14.04

I'm working my way through Python Geospatial Development, which as good fortune would have it, is the book we are using in my Introduction to GIS Algorithms and Programming course at George Mason University.  At the beginning of the course we were given a VirtualBox image of Ubuntu 12.04 with all the software and data files we need for the course preloaded.  Since I'm already running Ubuntu 14.04 as my regular desktop OS, I preferred to just install the packages (all of which are in the standard debian repositories) onto my desktop machines.

Here are the libraries we are using:
  • GDAL - The Geospatial Data Abstraction Library (see Python GDAL/OGR Cookbook for Python specific "recipes") for reading and writing geospatial data (GDAL for raster, OGR for vector).
  • pyproj - Performs cartographic transformations and geodetic computations.
  • Shapely - Manipulation and analysis of geometric objects in the Cartesian plane.
  • Mapnik - Toolkit for building mapping applications.
To install them for Python 2 on Ubuntu 14.04, run the following in a unix shell:
  • $ sudo apt-get install python-gdal
  • $ sudo apt-get install python-pyproj
  • $ sudo apt-get install python-shapely
  • $ sudo apt-get install python-mapnik2
Even on Ubuntu 15.04 Mapnik is only available for Python 2.  The other three libraries (gdal, pyproj, and shapely) are all available for Python 3.  This means that I will unfortunately have to use Python 2 for my mapping project.

I've now read the first eight chapters of Python Geospatial Development, which contain all the information I'll need for my project:
  • Chapter 1 - Geospatial Development Using Python
  • Chapter 2 - GIS
  • Chapter 3 - Python Libraries for Geospatial Development
  • Chapter 4 - Sources of Geospatial Data
  • Chapter 5 - Working with Geospatial Data in Python
  • Chapter 6 - GIS in the Database
  • Chapter 7 - Working with Spatial Data
  • Chapter 8 - Using Python and Mapnik to Produce Maps
It is often quite difficult to find good books about rapidly developing technologies like this, but this book does a good job.  The material is relevant, the presentation clear, and the examples both engaging and illuminating.

With background reading done and infrastructure setup, it is time to dig into the project.

Thursday, April 16, 2015

Firefox OS and Lowering the Barrier to Entry into ICT

I'm recently returned from Pycon 2015, and still feeling the energizing, rejuvenating effects of this annual pilgrimage to connect with a community I love.  The YouTube channel for the conference talks is here.

Two of the keynotes were especially memorable.  The keynote by Catherine Bracy, Director of Community Organizing for Code for America, left me feeling fortunate indeed to be involved with our local Code for NOVA brigade.  Catherine laid out in clear terms Code for America's vision of transforming the way citizens relate to their government (and the increasing alienation many feel from government) by bringing the community together to work collaboratively to address local needs.

The keynote by Jacob Kaplan-Moss discussed how sexism and the myth of the rock star / ninja programmer where keeping people from entering the programming field and driving women out of it after they do enter.  By describing the ninja programmer myth as an inverted normal distribution, he presented both a memorable image and a clear exposure of the absurdity of the myth.

Lowering barriers to entry into programming is my raison d'ĂȘtre as an ICT/CS teacher in a school serving lower income, immigrant students.  If I can't find a way to lower the barrier to entry, there is really no good reason for me to come to work each day.  It is with that in mind that I have been continuously thrilled by the wonderful work the folks at Mozilla are doing with their development tools for Firefox OS.

The following three files are from the first app introduced in chapter 2 of Firefox OS in Action:
Here is what this code looks like running in a browser (with the widow suitably shrunk to a small size):

 adding one more file turns this into a Firefox OS app:
Here is the app loaded in the WebIDE and running in the simulator:

The deployment story here is simply amazing.  I've never seen anything like it.  The WebIDE comes with Firefox, so there is nothing to install.  Simply press <Cntrl>+<F8> to launch it.  A few clicks of the mouse and a simulator with your choice of version of Firefox OS is running, and deploying your app to the simulator is a matter of loading the app source into the WebIDE and clicking a single button!
It can't get any better than that, right?  Wrong!  The best is yet to come.  What if you want to deploy your app to a real phone?
To show you how amazingly simple this is, let me first introduce a more interesting app, the LibriFox audio book player that two of my students are working on:
To deploy this app to my Firefox OS phone, I simply plug in the phone with the USB cable, and it now shows up in the list of available targets:
Firefox OS(Flame) is my phone

Clicking the same triangle button I used to deploy to the simulator loads the app onto my phone.  Wow!

I've never seen a development environment that even approaches the simplicity and elegance of this one.  The Mozilla Foundation is delivering on its goal of "making the Web better and more accessible for everyone everywhere".  I'm an ICT teacher just getting started in mobile app development, but I'm hooked on Firefox OS already.  By lowering the barrier to entry into this important technology, Mozilla will surely attract new developers far and wide, promising great things for the future of this free software phone operating system.
Thank you, Mozilla!

Monday, April 6, 2015

Serendipitously Approaching GIS from Both Ends

"Serendipity" is the word I think most fitting to describe how I ended in a Geographic Information Sciences Graduate Certificate program at George Mason University.  In brief, after getting involved with Code for NOVA with the aim of finding real world learning opportunities for my students and hearing the folks there talking enthusiastically about publicly available data sets, I decided to enroll in GGS 550: Geospatial Science Fundamentals during the Fall 2014 semester.  This course got me totally hooked on GIS!  It provides me with a way to apply my mathematics and computer science degrees to real problems.

It seems like my timing has been wonderfully serendipitous as well.  A few years ago it appears I would have been limited to proprietary software and military industrial applications. Now it's free software and citizen science.  I mentioned last post how my current GIS course is a python programming course.  I'm beginning work on my final project for that course:

Mapping Housing Affordability in Arlington Virginia

Generate maps of Arlington Virginia which show the location of affordable housing units, both committed affordable (CAFs) and market rate affordable (MARKs).  If time permits, I'd like to be able to produce a map that shows changes in availablility over time (availability is rapidly declining), but that may be beyond what I can do in our available time.
My goal is to do this using Python with the needed libraries.  Our Python Geospatial Development book addresses everything I would need for this project, but it will require finishing all of the book, since working with shape files isn't discussed until chapters 11 and 12.
Since the project is due May 3rd, I will only be able to get started with the full map by that time, but if I begin to generate custom maps of Arlington and to place geospatial data on them (housing locations) together with relevant meta-data, I'll be confident from there that with time and can map whatever maps I want.

We are using Python Geospatial Development as our text, and I'm working my way through all the examples, converting them from Python 2 to Python 3. I'll be documenting what I'm learning in posts over the next several weeks.

With python libraries and things like GeoDjango to take care of the back end, I'm looking at Firefox OS for front end mobile application development.  With built-in GPS and other sensors, mobile devices are great tools for citizen scientists, and Firefox OS's mission of expanding access to information is just what I was looking for.  I've got several students working on Firefox OS projects this Spring.  I've started reading Firefox OS in Action with an eye toward both learning to program on the platform myself and to using this book as a resource with students.  I'll be documenting that as over the next several weeks as well.

Saturday, April 4, 2015

Python GIS and Firefox OS

I haven't made a post since January, but all that is about to change.  A lot has happened in the past 3 months:
  1. After several false starts looking into app development for Ubuntu phones, I came across Firefox OS, with its supportive, community friendly staff and real focus on community building and education. Three of my current students are now working on Firefox OS related projects, and being mentored by Mozilla employees in the process.
  2. My Spring graduate GIS class at George Mason University, titled "Introduction to GIS Algorithms and Programming" turned out to basically be a Python programming class with a focus on GIS (how cool is that!? ;-) As a result, my love affair with GIS has only deepened.  I will be doing an independent research project over the Summer that will involve applying and extending what I am learning in this class to a project involving using LIDAR data to inform Northern Virginia residents about the viability of putting solar panels on their homes.  I will be spending a two month "retreat" with my 100 year old grandmother in New Jersey immersing myself in both this project and in developing curriculum materials for teaching mobile app development with Firefox OS.
  3. Our Code for NOVA brigade continues to grow and is reaching the size needed to provide multiple opportunities for real synergies between developers (both experienced and inexperienced), and local, regional, and national government agencies looking for help with IT tools to serve the public and conduct citizen science projects.  Our National Science Foundation host continues to make incredible learning opportunities available to Arlington Public Schools students.
I'll begin posting to this blog much more regularly now, as I document progress on each of the above.  Stay tuned...

Monday, January 12, 2015

Cambell's Law

I've had a gut feeling that the data driven decision making that has become the mantra guiding our educational practice in US schools these days is actually harmful to students and education.  I can sense that the problem is related in some way to the erroneous assumption that you can separate the observer from the observed - that the educational system can both gather data on itself and use this data to effectively improve its own practice.

In the case of a political, bureaucratic system like a public school system, relying on data points to determine the effectiveness of our efforts, and then doling out rewards and punishments based on these data points guarantees that it is the data points that will soon take on primary value, regardless of what they actually mean in terms of student growth and development.

It was with great delight that I came across this entry on Wikipedia on Cambell's law.  I take comfort in finding a statement by a well known and respected personage of what I was trying to say stated much better than I could say it:
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
Donald T. Cambell

Wednesday, January 7, 2015

Apache Cordova Development Environment Setup on Ubuntu 14.04

This is my "Happy New Year" post.  I had a goal over Winter break to find a process my students could use to setup an Apache Cordova development environment on their Ubuntu 14.04 laptops (I'm actually using Peppermint OS 5, which is a great light weight version of Ubuntu 14.04).

After several days of false starts and dead ends, I finally came up with something that works. I never succeeded in getting an Ubuntu environment working, and decided to give up on that for now until things settle down and the Ubuntu development team makes the process easier for beginners.  Firefox OS was wonderfully simple, and will definitely be my preferred choice of target system when introducing this to my students.  Android was only a bit more troublesome, since it involved the dreaded Oracle Java, but thankfully Cordova will shield us from having to deal with that directly once we have it setup.

Setup for Building Firefox OS Apps

 

Install Apache Cordova CLI And Create Hello App

$ sudo apt-add-repository -y ppa:cordova-ubuntu/ppa
$ sudo apt-get update
$ sudo apt-get install cordova-cli
Everything else we need to develop for Firefox OS is now built into the browser (how cool is that!? ;-) To make and run the Hello App, choose a good location for the project code, and:
$ cordova create hello
$ cd hello
$ cordova platform add firefoxos
$ cordova build
That's all there is to building the app for Firefox OS.  We now have a ./platforms/firefoxos/www directory that contains our app.  What we need now is an emulator to run it on.  The latest Firefox makes this a snap:
  • Open WebIDE in Firefox by pressing Shift+F8.
  •  Click Select Runtime and select Install Simulator, then choose a simulator (I choose the latest one marked "stable", 2.0 at the time I'm writing this) and click install.
  • Click Select Runtime again and click on your simulator. An emulator is launched running Firefox OS (if only life could always be this easy! ;-)
  • Back in the WebIDE window, click on Open App, then select Open Packaged App ... and navigate to the./platforms/firefoxos/www directory inside the hello directory and click Open.
  • The previous step will load the source directory into WebIDE.  Click the run button in the top center of the WebIDE window (it is a triangle next to a square), and we are rewarded with a running app.

 
Granted, it doesn't do much yet, but this was by far the most pain free process I experienced in my several days of trying to get cordova apps running on emulators.

Setup for Building Android Apps


To build for Android will require dealing with the evil Oracle Java environment and setting up the Android SDK, but the fine folks at the webupd8team and ubuntu-desktop teams have made this easy to do, and cordova will then permit us to develop with HTML, CSS and JavaScript and take care of the rest for us:

Install the Oracle Java 7 JDK 

$ sudo add-apt-repository -y ppa:webupd8team/java 
$ sudo apt-get update 
$ sudo apt-get install oracle-java7-installer oracle-java7-set-default

Install Ubuntu Make 

$ sudo add-apt-repository -y ppa:ubuntu-desktop/ubuntu-make 
$ sudo apt-get update 
$ sudo apt-get install ubuntu-make ant

Install Android Studio and SDK

  • $ umake android
  • Choose the installation path (I choose /home/<username>/.local/tools/android/android-studio to keep things from cluttering my home directory).
  • [I Accept (a)/I don't accept (N)] a
  • Wait while Android Studio downloads and installs… Installation done
  • Start Android Studio from menu (Programming -> Android Studio on Peppermint or use the Dash on regular Ubuntu).
  • [I do not have previous version…] OK [Setup Wizard - Welcome] Next [Custom] Next
  • Android SDK Location: /home/<username>/.local/android/sdk (again, I put things in .local to avoid clutter).
  • Next [Accept] Finish … Long wait while everything installs … Finish
  • Create a .bashrc file (or add to the one you already have) with the following: 
ANDROID_HOME=$HOME/.local/android/sdk 
PATH=$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$PATH


export PATH ANDROID_HOME
  • Start the Android SDK Manager with $ android
  • Select Android 4.4.2 (API 19) and click Install 16 packages… [Accept License] Install
We now have the Android development environment setup.  Cordova CLI integrates nicely with this.  All we need to do to add Android as a target platform for our Hello App is to run the following from inside our hello directory:
$ cordova platform add android
$ cordova build
$ cordova run android

Developing for mobile platforms is becoming a compelling thing for me to do as an IT/CS teacher to maintain student interest in learning.  Cordova makes it possible to use the tools we have already been learning, HTML, CSS and JavaScript, to start working on mobile platforms.

Next: Find good tutorials for learning cordova.

Sunday, October 5, 2014

Server Side Includes

I have a fantastic group of students in my afternoon Advanced Topics in Information Technology class this year.  Its been a long time since I've had a group with this level of skill, self-motivation, and interest.  It is going to be a fun year!

Two students are studying web page design.  They have already completed both the GDW HTML and GDW CSS tutorials, and now they are working on projects to apply what they have learned to their own website.  One of the two students, Jack, has been working on a menu bar that highlights the current page using CSS.  This led us to a discussion of the need to be able to dynamically include standard parts your website (headers, footers, etc.) across multiple pages, so that changes to these cross page components can be made in only one place. This was a great time to introduce the DRY principal in software development.

Another student in the class, Sam, who is studying The C Programming Language this semester, is also interested in system administration, and on overhearing Jack and me discussing the problem, pointed out Server Side Includes to us. I'm surprised that I've never heard of this technology before, since it provides just the kind of solution I'm looking for in terms of motivating a deeper understanding of web technologies step-by-step.  I don't want to jump into a Python web framework (even a micro framework like bottle) at this point.  I'm not a fan of PHP, though I realize that it is not a bad fit for what I'm looking for. SSI looks to be better.

Sam volunteered to explore setting up and documenting SSI.  I told him to setup an Ubuntu server with VirtualBox.  He will be delayed in doing that because he only has 32 bit Ubuntu on his laptop, so while we wait for Sam to reinstall with 64 bit Ubuntu this weekend, I decided to go ahead and document as much of this process as I could.

Setting Up Ubuntu Server in VirtualBox

Here is a screen shot showing the first stage of the Ubuntu server setup using VirtualBox:


I used 1 Gig of RAM and a 20 Gig virtual hard drive (the first option in the menu of virtual hard drive choices, VDI). I then changed the network setting to use a "Bridged Adapter", which gives the virtual machine its own network address on the host network, instead of putting it behind NAT on a 10.x.x.x. network. This makes it easy for me to ssh into the virtual server from my desktop, and for others (students, say, in a classroom setting), to ssh into it from their machines.

During the installation of Ubuntu Server, the only software package option I picked was "Open SSH Server".  I plan to install everything else I want piece by piece.

The next task is to install apache on the server:
$ sudo aptitude install apache2
As soon as that completes, I can point the browser from my desktop at the IP address of the server running inside a virtual machine that it is hosting (I don't care how many times I've done this, I always think it is cool! ;-)


 
My host machine (desktop) in this case has IP address 192.168.1.4, and the server running inside the virtualbox has IP address 192.168.1.11, so my the browser running on my desktop is connecting to the web server running on the virtual machine.

Enabling SSI

Now to get server side includes working, I followed Sam's instructions:
# a2enmod userdir
# a2enmod include
# vi /etc/apache2/mods-availiable/userdir.conf
   change "IncludesNoExec" to "Includes"
# vi /etc/apache2/mods-available/dir.conf
   add "index.shtml" (immediately after "DirectoryIndex"
     and before "index.html")
# service apache2 restart
We now have user local directories and SSI enabled (we hope ;-).  Let's try it out. I created a public_html subdirectory in my home directory, and put two files in it, index.shtml:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>SSI Test Page</title>
</head>
<body>
<h1>SSI Test Page</h1>

<!--#include file="success_message.inc" -->

</body>
</html>
and success_message.inc:
<p>
If you are reading this message, SSI is working... Yeah!
</p>
Now I point my web browser at: 192.168.1.11/~jelkner and:

One last thing to note is that the SSI_PSPserver.vdi virtual hard drive file can easily be copied from machine to machine, bringing all the setup with it.

I talked to Sam about what my educational motivation was for wanting to use SSI.  He quoted Nietzsche on his blog post for that day to let me know he totally understood what I was getting at:
"He who would learn to fly one day must first learn to stand and walk and run and climb and dance; one cannot fly into flying."
-Friedrich Nietzsche
I'll say it again, this is going to be a fun year!