Tuesday, July 19, 2016

Setting Up a RHCSA Practice Laptop - Part 1

In order to practice for the RHCSA at home, I took an old Dell Latitude E6500 with 4GiB of RAM and purchased a 500GiB hard drive on which I could install CentOS 7 with a server GUI as the base operating system and them multiple virtual machines using KVM with which to experiment and learn.

My package selection for the install was a Server GUI installation. I partitioned the hard drive with 1 GiB of swap, 500 MiB on a standard partition for the /boot, and 30 GiB for the root partition.

I also created a 60 GiB partition for /home, and then allocated everything that remained (375.27 GiB) to /var.  The reason for giving some much space to /var is that the default KVM / qemu setup on CentOS 7 places virtual hard drive images in /var/lib/libvirt/images, so I wanted plenty of space available for multiple images.

After the install finished I wanted to see what the partitions looked like, so I ran
$ sudo yum install system-storage-manager
and then
$ sudo ssm list
which revealed the following:

Device        Free       Used      Total  Pool        Mount point
/dev/sda                       465.76 GB              PARTITIONED
/dev/sda1                      500.00 MB              /boot
/dev/sda2  0.00 KB  465.27 GB  465.27 GB  centos
Pool    Type  Devices     Free       Used      Total
centos  lvm   1        0.00 KB  465.27 GB  465.27 GB
Volume           Pool   Vol size  FS   FS size    Free      Type   Mount point
/dev/centos/swap centos 1.00 GB                             linear
/dev/centos/root centos 30.00 GB  xfs  29.99  GB  26.73 GB  linear /
/dev/centos/home centos 60.00 GB  ext4 60.00  GB  55.88 GB  linear /home
/dev/centos/var  centos 374.27 GB ext4 374.27 GB 349.02 GB  linear /var
/dev/sda1               500.00 MB xfs  493.73 MB  293.59 MB part   /boot

To be continued in Fall of 2016..


Sunday, April 17, 2016

Moving an ArcGIS File Geodatabase to QGIS

I am taking GGS 553: Geographic Information System this semester at part of my graduate studies at George Mason University.  In a previous post I described how I ended up in this Geographic Information Science graduate certificate program, which I have now been pursuing for almost 2 years.  GGS 553 is a required course, and the first one in the program that has required me to use proprietary software, since much of the course is focused on learning to use ArcGIS.

I am both philosophically and ethically opposed to proprietary software, since it runs dead against the expansion of our shared cultural space, which I believe is vital to the survival of our species. This is a required course, however, and in the large scheme of things I am willing to compromise when I need to. I like to think of it as dancing with the devil, learning the devil's moves in order to be able to freely out dance him in the future. In this case that will mean applying what I learn in GGS 553 to mastering QGIS, the free software alternative to ArcGIS. I had intended to try to do each of our assigned labs this semester in both ArcGIS and QGIS, but when I found it difficult enough just to complete them on time in ArcGIS, I gave up on that idea after the first week.

This week we have a sort of half size assignment, so I thought I would use the extra time available to see if I could do it in QGIS.  The first challenge will be to load the project data into QGIS.  We were given the data in ArcGIS's file geodatabase format. QGIS can not yet read and write to this format directly, but there are tools available to convert it into PostGIS, with which QGIS can work well.

Last Summer I wrote a blog post documenting how I setup a PostGIS server on Ubuntu 14.04.  Since this year I am also needing to learn RHEL, I'll use this guide to setup the server on the little Centos 7 server I have at home for just such purposes, and then connect to it from QGIS running on my Ubuntu desktop.

Installing a PostGIS Server on Centos 7

$ sudo yum install postgis postgresql-server postgresql-contrib
$ sudo postgresql-setup initdb
$ sudo -i -u postgres
$ psql
postgres=# \password postgres
Enter new password: 
Enter it again: 
postgres=# \q
$ exit
$ sudo vi /var/lib/pgsql/data/pg_hba.conf

Change this line (near the bottom):

host    all             all               ident

to this:

host    all             all                  md5

Next allow database connections from outside:

$ sudo vi /var/lib/pgsql/data/postgresql.conf


#listen_addresses = 'localhost'

to this:

listen_addresses = '*'

Create a new database user with superuser privileges:

$ sudo su - postgres
$ createuser --superuser [user]
$ psql -c "ALTER ROLE [user] PASSWORD '[password]'"
$ exit

Then as that user create the database and add gis extensions:

$ createdb webster
$ psql -d webster -c 'CREATE EXTENSION postgis'

Then after copying over the Webster.db directory containing the file geodatabase, I ran:

$ ogr2ogr -f "PostgreSQL" PG:"dbname=webster user=[user] password=[password]" Webster.gdb

After which I connected my desktop QGIS to the PostgreSQL server running on my little household server and loaded the three layers I found there:


Thursday, March 31, 2016

Software Management with YUM

YUM (Yellowdog Updater, Modified) is the package management tool used on Red Hat Enterprise Linux and its derived versions, CentOS and Scientific Linux. It acts as a front end to the RPM Package Manager (RPM), and is used to install, remove, and update software on Red Hat based systems.

I first encountered YUM when installing Yellow Dog Linux on PowerPC based Macintosh computers back at the dawn of the 21st century.  When I switched over to Debian based GNU/Linux systems with the release of Ubuntu in 2004, I completely lost touch with the RPM world until my Spring semester Linux System Administration course's pursuit of RHCSA certification brought me back into the fold.

I am writing this post to use as a handy list of the most common things I need to do when managing software:
  1. Update the software on the system
    $ yum check-update
    $ sudo yum update package_name
    $ sudo yum update [to update all packages]
    $ sudo yum group update group_name
  2. List all the currently installed software
    $ yum list installed
    $ yum list installed "global expression"
  3. Search for available packages
    $ yum list available "global expression"
    $ yum search term...
  4. Display information about a package
    $ yum info package_name
  5. Install a new package
    $ sudo yum install package_name
  6. Remove an existing package
    $ sudo yum remove package_name
  7. List the current repos
    $ yum repolist
    $ yum repolist -v
That covers the basics. I also need to learn how to clean up the cruft that accumulates over time as a system is run, in Debian land the kind of thing that would be done with $ sudo apt-get autoremove. It seems that in RPM space that is accomplished with the package-cleanup utility, so I'll look into that.


Saturday, March 19, 2016

Centos Command-line Tricks and Tips - Getting Rid of the Terminal Beep

Getting Rid of the Terminal Beep:

My terminal was making an annoying beeping (more like a swoosh beep, actually) every time it couldn't match a tab completion.  I like to listen to music while I work, so this was really driving me crazy.  All I needed to do to stop it was to run:
$ echo 'set bell-style none' >> ~/.inputrc
which appends 'set bell-style none' to the .inputrc file in my home directory.  .inputrc didn't exist in my home directory (I checked before running the command), so running this command created it.
After exiting the terminal and starting another, the terminal maintained the silence I wanted it to ;-)

Friday, February 19, 2016

Setting Up a Centos Router - Part 1

In order to run the kind of experiments we will need to run to really learn proper GNU/Linux system administration, we need our own "safe space" in which to play.  In previous years when I had students with the level of skills our ITN 170 group is quickly acquiring, I always used one of our machines as a NAT Router so that we could isolate our own network traffic and setup custom services within our private network space.

The basic idea is captured in the following illustration.
What is required is a machine with two NICs (represented here by Tux) - one which connects to the outside network and the other which connects to the local network.

Setup Process

Here is what I did to setup a basic router using an old desktop PC:

  • Did a minimal install of CentOS 7 on a machine with two NICs, connecting one of the NICs to the outside network and activating this connection using DHCP on the host network during the installation process.
  • Ran yum update after installation to make sure I had the current software.
  • Ran yum install yum-utils vim to get vim and the package-cleanup utility. I then ran package-cleanup --oldkernels --count=1 to remove all but the current kernel package.
  • I ran ip addr and got back information on three network interfaces:
    1. lo - the loopback interface or localhost, with its network address.
    2. enp0s25 - the NIC on the motherboard which I had activated with DHCP during installation.
    3. enp3s0 - the addon NIC that was not configured during installation. It had the following information:
      enp3s0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
          link/ether 00:15:17:20:b6:e6 brd ff:ff:ff:ff:ff:ff
  • I edited /etc/sysconfig/network-scripts/ifcfg-enp3s0 adding the following:
    GATEWAY="x.x.x.x"  (place your gateway adress here)
I used the resources linked below to try to enable IP routing and NAT, but I was not successful in getting it to route.  I have a laptop running Centos 7 connected to the router machine.  Before attempting this setup I had installed ClearOS on the router and got it to route for the laptop with a setup process using ClearOS's web interface.  An experienced friend of mine shamed me into removing this, however, by telling me he would never hire a sysadmin who only new how to set this up using a web interface.

So for now I have assigned two of my students to continue looking into it, and I'll get together with that friend who shamed me into this to get his assistance on Tuesday if we haven't figured it out by then.

To be continued...


Monday, February 8, 2016

Text Processing and Unix History

Preparing for the RHCSA certification is turning out to be a heap of fun! Despite more than 20 years as a free software activist and personal user of GNU/Linux systems for all my personal computing, and despite being a computer science teacher during that same time, there are a wide range of basic Unix CLI skills that I only scratched the surface of in all that time (shame on me!).

Preparing for the RHCSA is providing the opportunity to address that deficit at long last.  Chapter 4 of the book we are using in class to study for the certification is titled "Working with Text Files". The most enjoyable thing about this investigation into Unix text file processing is the view it provides into Unix history.

In the beginning there was eded begat ex, and ex begat vi... Along the way we got cousins grep and sed too.  Since grep, sed, and vi are part of the Unix admin's toolset, I want to learn to use them at least well enough to be able to help prepare students (and myself) for the RHCSA certification and to be able to present them well to future students in my ITN 170: Linux System Administration class.

Since in the beginning there was ed, let me start with that.  I found a very nice blog post, Actually using ed, which I found to be a wonderful introduction to this tool.  I set myself the task of using ed to create a list of fruits in a file named fruits.txt.  The first thing I found out was that trying:
$ ed fruits.txt
did not create the file for me, instead returning a "No such file or directory" error.  So I did the following, which worked:
$ touch fruits.txt
$ ed fruits.txt
After that, I ran $ cat fruits.txt, and saw that everything was as I wanted it:
Now if I want an alphabetical listing of the fruits in my list, I can run:
$ grep berries fruits.txt | sort
and see this:
RegexOne is a nice, interactive tutorial for learning basic regular expressions.  I wanted to do all the exercises using grep on the command-line as well, and in the process setup a new github repo for resources related to our RHCSA study, here.

Next I wanted to learn sed.  Sed - An Introduction and Tutorial by Bruce Barnett is a wonderful tutorial.  With so much awful document out there, it is great to find something written by someone with a grasp of how people actually learn.

Using the fruits.txt file I created with ed, I ran $ sed s/berries/cherries/ fruits.txt and got:
Since sed uses the same substitution syntax that vim uses, learning it will be a big help in becoming a more effective vim user as well.

Saturday, February 6, 2016

QGIS Delivers Functionality and Freedom

I am taking a graduate course this semester, GGS 553 - Geographic Information System, which is required for the Graduate Certificate in Geographic Information Sciences program that I am hoping to complete.  I like the text book we are using for class, and greatly enjoyed the first lecture.  What I am not happy about is that the labs which will make up a large part of the course assignments require the use of proprietary software, specifically ArcGIS, and then by extension, the Windows operating system on which it runs.

I have been a free software activist for more than 20 years. Software for GIS makes it especially easy to state why I believe so strongly in software freedom. To put it simply, I believe software should be part of humanity's shared cultural heritage, and that all efforts to turn it instead into a commodity are immoral.

Installing ArcGIS made this painfully clear to me.  In the first place, using it required that I use a non-free operating system, so I am running Windows just so that I can use ArcGIS.  Going through the gymnastics (registering an on-line account, figuring out where to enter the product code after missing it the first time through the installation, etc.) required to establish that I was "authorized" to use the commidified resource was most unpleasant. It rubs me deeply the wrong way to see human creativity misspent making the world a worse place rather than a better one.

No matter.  I have to do it to complete this required course, so I am determined to make the best of it.  What that means to me is keeping in mind the well known quote from Sun Tzu,
"Know your enemies and know yourself, you will not be imperiled in a hundred battles."
So I'll count learning ArcGIS as knowing my enemy, and time permitting, I will do each lab assignment in QGIS in parallel.

The first thing I wanted to do was to install the latest QGIS on my Ubuntu 14.04 desktop.  To do this, using this web page as a guide, I added the following to the end of my /etc/apt/sources.list file:

# For QGIS 2.12
deb http://qgis.org/ubuntugis trusty main
deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu trusty main

Then I ran:
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 3FF5FFCAD71472C4
$ sudo aptitude update
$ sudo aptitude install qgis
This is a much easier process than installing ArcGIS. QGIS also runs much faster than ArcGIS, and on the operating system I choose, not the one chosen for me.

It also seems that the wonderful folks who have developed QGIS have modeled its UI after the non-free standard, so the lab notes describing ArcGIS helped me understand QGIS as well. QGIS's Browser is the equivalent of ArcGIS's ArcCatalog. Here is the QGIS Browser showing the shape files from my first lab:
The QGIS Desktop functions like ArcGIS's ArcMap.  Here is QGIS Desktop with my Lab 1 shapefiles in a map:
So far, so good.  I was able to answer all the lab questions using QGIS with the given data, and I learned new things about QGIS through doing the ArcGIS lab exercises.