Saturday, July 4, 2015

Updating the Firmware on my Pyboard


Before I begin the post proper, an aside...

Larry Cuban just made a wonderful blog post titled Fixing Schools Again, and Again in which he says,
"If amnesia were like aphrodisiac pills, policymakers have been popping capsules for years. Memory loss about past school reforms permits policymakers to forge ahead with a new brace of reforms and feel good."
To an educator of 24 years these words ring so true!  Time and time again I've watched my own school system move from one initiative to another, never stopping for a moment to look back at where we've been so that we might learn something from our journey.



In this intensive Summer of learning, I've been trying to juggle several learning goals, each associated with either NOVA Web Development or the future IT curriculum at Arlington Tech (or hopefully, both):
  1. GIS, specially web GIS mapping of LIDAR data
  2. Firefox OS App development
  3. Django and django CMS
  4. MicroPython programming on the pyboard
This post will document the process of updating the firmware on the pyboard.

I'll be using the Pyboard Firmware Update guide my dear friend and co-conspirator Kevin Cole mentioned in a blog post, along with the instructions Kevin documented in the same post.

The first step in the process is to connect my pyboard physically to my to my desktop workstation by USB cable and verify that I can interact with it. When I connected the cable, a filesystem named PYBFLASH auto-mounted, showing these contents:
Kevin says to run screen, when I checked, I didn't have screen installed, so I ran:
$ sudo aptitude install screen dfu-util
    When I tried to connect to the pyboard using screen, it failed:
    $ screen /dev/ttyACM0
    [screen is terminating]
    $
    
    So I checked to see if the device was there:.
    $ ls -l /dev/ttyACM*
    crw-rw---- 1 root dialout 166, 0 Jul 4 15:39 /dev/ttyACM0
    $
    Ahh, I bet I'm not in the dialout group.
    $ groups [username]
    [username] : [username] adm cdrom sudo dip plugdev lpadmin sambashare
    $ sudo adduser [username] dialout
    [sudo] password for [username]:
    Adding user `[username]' to group `dialout' ...
    Adding user [username] to group dialout
    Done.
    $ groups [username]
    [username] : [username] adm dialout cdrom sudo dip plugdev lpadmin sambashare
    That's better.  I still needed to restart my machine before I could connect, but after that screen /dev/ttyACM0 gave me a blank screen, and after I hit <Enter> I saw:
    Traceback (most recent call last):
      File "", line 1
    SyntaxError: invalid syntax
    >>> 
    
    I'm in! Now let me move on upgrading the firmware.  Typing ^D (that's Ctrl+D) at the python prompt gave me:
    PYB: sync filesystems
    PYB: soft reboot
    Micro Python v1.3.10 on 2015-02-13; PYBv1.0 with STM32F405RG
    Type "help()" for more information.
    >>>
    Kevin's next instruction says, "Next, disconnect the USB cable, jumper the BOOT0 and 3V3 pins, and reconnect the USB cable".  There is a picture of this on the Pyboard Firmware Update guide:
    The problem is that I'm at my grandmother's house in a rural part of South Jersey without a car. Just how am I going to "jumper"?  Kevin and I asked our friend Paul Flint in Vermont, who has a knack for solving these sorts of problems.  He suggested a paper clip.  Why didn't I think of that?
    My pyboard with paper clip jumper
    I downloaded the latest micropython image (pybv10-2015-07-04-v1.4.4-65-gb19d273.dfu at the time of this writing) and reconnected by pyboard by USB.
    $ lsusb
    Bus 003 Device 002: ID 8087:8001 Intel Corp.
    Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    Bus 001 Device 003: ID 8087:0a2a Intel Corp.
    Bus 001 Device 009: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader
    Bus 001 Device 008: ID 0424:2602 Standard Microsystems Corp. USB 2.0 Hub
    Bus 001 Device 007: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
    Bus 001 Device 006: ID 413c:3016 Dell Computer Corp. Optical 5-Button Wheel Mouse
    Bus 001 Device 005: ID 046d:0825 Logitech, Inc. Webcam C270
    Bus 001 Device 004: ID 413c:2003 Dell Computer Corp. Keyboard
    Bus 001 Device 002: ID 050d:0234 Belkin Components F5U234 USB 2.0 4-Port Hub
    Bus 001 Device 012: ID 0483:df11 STMicroelectronics STM Device in DFU Mode
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    I've highlighted in green the line we are interested in.
    $ sudo dfu-util -l
    [sudo] password for [username]:
    dfu-util 0.8

    Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
    Copyright 2010-2014 Tormod Volden and Stefan Schmidt
    This program is Free Software and has ABSOLUTELY NO WARRANTY
    Please report bugs to dfu-util@lists.gnumonks.org

    Found DFU: [0483:df11] ver=2200, devnum=12, cfg=1, intf=0, alt=3, name="@Device Feature/0xFFFF0000/01*004 e", serial="346135523333"
    Found DFU: [0483:df11] ver=2200, devnum=12, cfg=1, intf=0, alt=2, name="@OTP Memory /0x1FFF7800/01*512 e,01*016 e", serial="346135523333"
    Found DFU: [0483:df11] ver=2200, devnum=12, cfg=1, intf=0, alt=1, name="@Option Bytes  /0x1FFFC000/01*016 e", serial="346135523333"
    Found DFU: [0483:df11] ver=2200, devnum=12, cfg=1, intf=0, alt=0, name="@Internal Flash  /0x08000000/04*016Kg,01*064Kg,07*128Kg", serial="346135523333"
    Again, I highlighted in green the line we are looking for, the internal flash.
    $ sudo dfu-util --alt 0 -D pybv10-2015-07-04-v1.4.4-65-gb19d273.dfu 
    dfu-util 0.8
    
    Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
    Copyright 2010-2014 Tormod Volden and Stefan Schmidt
    This program is Free Software and has ABSOLUTELY NO WARRANTY
    Please report bugs to dfu-util@lists.gnumonks.org
    
    Match vendor ID from file: 0483
    Match product ID from file: df11
    Opening DFU capable USB device...
    ID 0483:df11
    Run-time device DFU version 011a
    Claiming USB DFU Interface...
    Setting Alternate Setting #0 ...
    Determining device status: state = dfuERROR, status = 10
    dfuERROR, clearing status
    Determining device status: state = dfuIDLE, status = 0
    dfuIDLE, continuing
    DFU mode device DFU version 011a
    Device returned transfer size 2048
    DfuSe interface name: "Internal Flash  "
    file contains 1 DFU images
    parsing DFU image 1
    image for alternate setting 0, (2 elements, total size = 273128)
    parsing element 1, address = 0x08000000, size = 13580
    Download [=========================] 100%        13580 bytes
    Download done.
    parsing element 2, address = 0x08020000, size = 259532
    Download [=========================] 100%       259532 bytes
    Download done.
    done parsing DfuSe file
    $ 
    
    After disconnecting the pyboard, removing the jumper (paper clip), and reconnecting the pyboard and starting a screen session to it, and typing ^D, I was rewarded for my efforts when I saw:
    PYB: sync filesystems
    PYB: soft reboot
    Micro Python v1.4.4-65-gb19d273 on 2015-07-04; PYBv1.0 with STM32F405RG
    Type "help()" for more information.
    >>> 
    
    Now let's see what Kevin and the MicroPython interns can do with our newly flashed pyboards...



    Installing Django CMS on Ubuntu 14.04

    NOVA Web Development is a web design and development coop I started with my friend Kevin Cole about three years ago.  I was motivated by a desire to make the IT program at my school relevant to my mostly working class, immigrant students.  As a career and technical education teacher, I'm suppose to be helping students find career paths, but despite the large number of IT jobs available, my students are caught in the catch-22 that they need experience (most want adds list three years) to get a job, but they need a job to get experience.  NOVA Web Development was created to find a way out of that bind.

    Three years into the project, I would say we have been successful in getting off the ground, given we started with only funding I could provide and very limited experience.  We now have two former students (both of whom are in college) who are emerging as young web professionals, and who will soon be able to act as mentors to future students.

    Our business focus is offering web design and web application development using mostly the Python web application framework Django. One of my goals for this Summer is to develop skills needed to help out with NOVA Web Development projects. To do that I'm going to need to learn how to setup and manage django CMS.



    Since I want to be able to experiment without fear, the first thing I did was install VirtualBox on my Ubuntu 15.04 host machine using:

     $ sudo apt-get install virtualbox

    Then I created a virtual machine (VM) through the GUI interface and installed Ubuntu 14.04 server on it.  I only selected SSH Server from the package installation menu during the install process to keep the machine minimal to start.  After the installation I logged into the VM and ran all the package updates, and then made a copy of the virtual hard drive (the file with extension .vdi in the VM's directory).  This will save me a lot of time in case I mess things up on the VM and want to start over (that's the "experiment without fear" part).


    I always setup my virtualbox VMs with bridged network adapters:
    Which gives the VM an addressable on the same network as the host machine.

    With my VM server ready to go, I went looking on-line for some good reference material and found How to Install the Django Web Framework on Ubuntu 14.04. This guide discusses four alternative methods for installation. Of these I choose Install through pip in a Virtualenv (giving me a virtual environment on a virtual machine ;-).

    Here's what I did:
    • $ sudo aptitude install python3-all-dev python3-pip
    • $ sudo aptitude install libtiff5-dev libjpeg8-dev
    • $ sudo aptitude install libfreetype6-dev liblcms2-dev
    • $ sudo aptitude install libwebp-dev tcl8.6-dev tk8.6-dev
    • $ sudo aptitude install python3-tk python-tk
    • $ sudo pip3 install virtualenv
    • $ mkdir cms1
    • $ cd cms1
    • $ virtualenv env
    • $ source env/bin/activate
    • (env)$ pip install django
    • (env)$ django-admin --version
     1.8.2
    • (env)$ deactivate
    With Django 1.8.2 installed, I left the virtualenv to return later after a break to install django-cms.
    (note: This is an updated version of this post with modified installation steps.  When I ran the following process the first day, I encountered show stopping errors with lack of support in Pillow for jpeg and png -- described by others here. Today I copied over my virtual hard disk image to start with a fresh VM and began the process anew with the modified installation instructions above.  It worked. That's "experiment without fear" in action!)

    Later... 

    To continue the installation process, here is what I did next:
    • $ cd cms1
    • $ source env/bin/activate
    The Django installation guide continues with instructions on how to setup a project, but I want django-cms, so I switched to Installing django CMS for the next steps:
    • (env)$ pip install djangocms-installer
    • (env)$ mkdir testproj
    • (env)$ cd testproj
    • (env)$ $ djangocms -p . testsite
    • Now give the following to the installer's questions:
      • Database configuration (in URL format): sqlite://localhost/project.db
      • django CMS version: stable
      • Django version: stable
      • Activate Django I18N / L10N setting: yes
      • Install and configure reversion support: yes
      • Languages to enable. Option can be provided multiple times, or as a comma separated list: en, de
      • Optional default time zone: America/New_York
      • Activate Django timezone support: yes
      • Activate CMS permission management: yes
      • Use Twitter Bootstrap Theme: yes
      • Use custom template set: no
      • Load a starting page with examples after installation: yes
    • Create admin user when prompted
    • (env)$ python manage.py runserver 0.0.0.0:8000

    I then opened a web browser in the host machine and pointed it at the virtual machine. Here is a screenshot of part of my desktop showing this:
    In the screenshot of my desktop above, my host machine had address 10.0.0.5 and the VM has address 10.0.0.8.  I used ssh from my host machine to login to the VM, since the terminal program on the host is nicer to use than the one you get logging in directly from virtualbox.

    You have to add 0.0.0.0:[port number] to the end of the runserver command to allow web connections to the VM from other machines.  I then point the browser on my host machine to 10.0.0.8:[port number] (8000 in this case, but you can use any port number above 1024 you like) and am rewarded by the Welcome to django CMS home page!

    Tomorrow I'll start learning how to use django CMS.

    Thursday, July 2, 2015

    Getting Started with JOSM

    Today I completed the next lesson in the LearnOSM tutorial, which describes the installation process for JOSM, the Java 7 OpenStreetMap editor.  Here is what I did to install JOSM:
    1. Checked the JOSM package page for Ubuntu 15.04 and found it has version 7995.
    2. Visited the JOSM home page and saw that the recommended version is 8491.  Visited the Ubuntu package repository and read that it contains the JOSM package only, so it won't mess much with my system.
    3. After deciding the JOSM Ubuntu package repo was the way to go, I did the installation dance:
      • $ sudo vi /etc/apt/sources.list
      • Add these lines to the end of the file:
        # JOSM Ubuntu package repository
        # (see http://josm.openstreetmap.de/wiki/Download#Ubuntu)
        deb https://josm.openstreetmap.de/apt vivid universe
        
      • $ wget -q https://josm.openstreetmap.de/josm-apt.key -O- | sudo apt-key add -
      • $ sudo apt-get update
      • $ sudo apt-get install josm
    4. Launched JOSM
    A short while later in the tutorial and I had downloaded a sample file and opened it in JOSM.

    Some OSM Terminology

    I've learned previously that vector data in GIS systems is represented by three types of objects: points, polylines, and polygons.  These three types are described well in QGIS Introduction section on Vector Data.

    OSM uses its own terminology for these same types of objects:

    Nodes - points
    Ways - polylines
    Closed Ways - polygons

    Despite the different names, these objects are interchangeable with their QGIS (and ArcGIS) equivalent, and I plan to learn to use python tools like GDAL, pyproj, Shapely, and Mapnik to work with these files and convert them among the different formats.

    Without using the terms, the tutorial also introduces the concept of meta-data, using the word "tags" that describe information about the nodes, ways, and closed ways.

    It ends with an exercise to create a new map from scratch.  Just to test what is going on, I made the simplest one I could think of with one node, way, and closed way.  I saved it as three_places.osm.  Then I opened it in a text editor to see what it looked like (I used gedit for the benefit of this screenshot):
    Great, a simple XML file!  Before looking at the tutorial any further, tomorrow I'll explore what I can do with this file using Python.

    Wednesday, July 1, 2015

    A Fun First Experience with Learning OpenStreetMaps

    OpenStreetMap is the Wikipedia of geospatial information. It is a vast map of the world editable by anyone with an account, and like other free software and open data projects, it is fundamentally about creating use value for human beings. It thus stands opposed to the commodification of everything pushed by the dominant neoliberal economic regime which has been running our civilization rapidly toward self-destruction since the late 1970s.  Through projects like Map Kibera, it offers the possibility of providing voice to the voiceless and tools of change to the dis-empowered. It is part of the broad movement of democratization of information and citizen science that I hope will enable our civilization to mature and endure for our children, grandchildren, and their children and grandchildren.

    I decided to start my Summer GIS study by going through the LearnOSM tutorial.  I made it through The iD Editor lesson today, and had a great deal of fun in the process! After a bit of background, I'll describe the little project I completed.


    What I'm Doing in New Jersey this Summer

    June 6 Larry James Legacy Fund Bike Ride

    My grandmother (who I call "Mom-mom"), Helene Young of Cologne, NJ, turned 100 last February 18th. While she is in pretty darn good shape, she won't live forever, and I've been vowing to spend the Summer with her for years.  This Summer I am finally doing it.

    My first night here we went to the Egg Harbor City Historical Society meeting.  Mom-mom goes every month, but while I've been a member for years, I've never been to a meeting before. Our family's history has deep ties with Egg Harbor City.  Mom-mom's great grandfather, Philipp Mathias Wolsieffer, was a founder and the first mayor of the city in 1858.

    Choosing a Meaningful Project


    At the end of the second lesson of the tutorial, Getting started on OpenStreetMap.org, learners are suggested to "Move the map to a place that you know very well, such as your town or neighborhood. A good idea is to ensure your home (or your neighbour’s home) and workplace are drawn and given the correct address." Mom-mom was born in a house that her father built at 352 Baltimore Avenue, as was her brother Alfred and her famous sister, Peace Pilgrim.  In the spirit of my visiting with her, I decided that house would be an ideal one to check.

    Darn, It's Not Right! (I Mean, Cool, It's Not Right, So I Get to Fix It ;-)


    Typing in the address: 352 Baltimore Avenue, Egg Harbor City, New Jersey, OSM showed my this map:
    Click on the image to see a large version
    The house, which has an address on the paper road, Baltimore Avenue, is really located on Buerger Street, not one block over on Beethoven Street as in the map.  In case anyone has any doubts about that, I rode my bike into town this afternoon and took a picture of the house:

    So following instructions, I selected "Edit with iD", which gave me a satellite view of the area, and I could see the house. I used the "Draw shape (polygon)" tool to create a polygon near the outline of the house.
    Then I filled in data about the polygon, including giving it a title of "Birthplace of Peace Pilgrim". When I clicked "Save", I saw this:
    A few minutes later I did another search for 352 Baltimore Avenue, Egg Harbor City, New Jersey, only this time OSM returned:
    How cool it that? Obviously I'm feeling empowered and I'm greatly encouraged to keep learning about OpenStreetMap!


    Hmm... What About Mapping History?


    As soon as I labeled the house "Birthplace of Peace Pilgrim", I started thinking about other locations important in Peace Pilgrim's life I could label on the map.  Then I tried searching in OSM for "Peace Pilgrim", and wouldn't you know it, I got this result:
    Building Birth place of Peace Pilgrim, 352, Baltimore Avenue, Egg Harbor City, Atlantic County, New Jersey, 08215, United States of America
    Wow! Then I did a search to see if I could find other folks already using OSM to record history, and I came across Mapping History (Starting with Manchester) by Frankie Roberto.  Frankie says a few slides into his presentation, "I'd like to be able to add a time-slider to maps, so that you can see how the places themselves have changed over time." It is simply mind boggling to think about all the new ways of seeing things that are becoming possible with tools like OpenStreetMap.

    Thursday, June 25, 2015

    "Measure Twice and Cut Once" - An Open Request for Help Planning LibriFox Development

    I'm making this post as an open invitation to anyone who sees it who either can offer advice themselves or who knows someone who could offer good advice to please send it our way!

    As I described in a previous post, I have commissioned a former student to create a Firefox OS app for me that will download and play books from the LibriVox audio books website.

    My motivation for funding this project is twofold:
    1. To get a usable app for playing LibriVox books on my Firefox OS phone.
    2. To explore free software development best practices that we can then use in future projects.
    I'm acting as the on-site customer on the project, but I have to admit to being in a bit over my head.  Alex sent me this email today:
    Hey Jeff,
    Dominic [one of the Firefox OS Music player app developers] hasn't responded to me yet, but I've been thinking more about how the chapter files' metadata should be stored.  I'm actually thinking that maybe json stored in each chapter folder isn't the best solution.  I think sticking with localStorage is the better option, because it is reliable (we'll never have to worry about not having localStorage, whereas json files can be deleted) and only LibriFox can modify its own localStorage, whereas json files could be modified by other apps. 

    The downside to using localStorage, of course, is that the metadata doesn't follow the data, but is instead tied to the device.  But how often are you really going to be swapping sdcards between devices? Worst case scenario, you might have to just redownload the chapters through the app on the new device rather than having them pop up automatically.
    Depending on how much you care about the portability use case, I can also attempt to have it do what you were suggesting and pull data from the server based on the book id, assuming the files were originally downloaded through the app.  Otherwise, it could fall back on ID3 tags if they are present.  This gets pretty complicated, though.

    Also, I know you were concerned about keeping the metadata in sync with the data, so I'm working on metadata validation.  The latest commits to the repo have an object that scans the metadata and verifies all the paths when the app is started.  It looks like there are filesystem events that I can hook into as well.
    Let me know what your thoughts are!
    I don't know how to respond to Alex.  I don't want to go down a path that is "expensive" both in time now and maintenance costs in the future, but I think that switching SD cards may happen more than Alex thinks, or at least it is not at all unlikely that an SD card will be removed, and when that happens, how should LibriFox behave?

    Alex is a terrific young programmer.  He has great instincts and very high standards, which is why I was so happy about him taking on this "Summer of Code" project to begin with.  But he did just graduate high school, and he lacks the wealth of experience that a veteran programmer would bring to a project like this.

    I want to make the best contribution to Firefox OS that I can.  As a teacher by trade, I also want this to be the best learning experience for Alex that it can be.  For both of those goals to be realized, however, input from programmers who do have the experience Alex lacks will be required.  I've been trying everything I know how to do reach out to the Mozilla and broader free software communities to ask for the help we need.

    This blog post is part of that reaching out.  If you think you can provide meaningful feedback to the questions Alex raises in his email, please send it our way.  Thanks!

    Monday, June 22, 2015

    Providing Developer Debugging Feedback on Firefox OS

    I reported a bug on Alex's LibriFox app which is preventing book files from playing.  Alex has been unable to reproduce the bug, since the downloaded book plays on his phone, but it won't play on mine.  He sent me an email this morning detailing the steps I need to take to provide him with useful information from my phone about the bug.

    Since this process will be generally useful, I'll describe in detail what he told me to do and what happened (Alex's instructions are in green).

    1. First, go into developer tools and make sure Console enabled is checked.  For whatever reason, it won't log errors to console without that checked (even though it seems to log console.log statements just fine).  We want errors to be logged, so enable that.


    After clicking on the "Settings" icon on the home screen, and then the "Developer" link, I arrived at the screen above and confirmed that "Console enabled" was already checked.

    2. Next, open the app via the IDE (with your flame connected and set as the USB device) and click the wrench to open developer tools.  Go to the console tab.

    Don't do any app navigation until you have opened the console!
    Once the console is opened, tap your downloaded book, then the downloaded chapter to take you to the (not working) player page.



    It is so cool how well this works!  I connected my device via USB, then selected "Firefox OS (Flame)" from the "Select Runtime" in the Web IDE.  After giving permission on the phone to connect, I was able to select the installed LibriFox from App menu in the upper left of the Web IDE. I didn't even have to click on the wrench, it was already selected, and I saw what is in the screen shot above.

    3. If there is an error being thrown somewhere, your console might look something like this.  I've added a throw statement in my code to try and simulate your issue.
    Inline image 1

    If the file can't be found, your console will look something like this:
    Inline image 2

    Both of these cases will result in a non-functional player and no chapter title in the header, so without seeing console I can't tell which is which.

    Clicking my way through the app as instructed, I saw this:

    Now I'll wait and see what Alex can do with this information.

    What I keep liking about this OS is how friendly it is to developers.  That really isn't surprising, given Firefox's mission of "making the Web better and more accessible for everyone everywhere", but it is great to see this mission being carried out so well!

    Sunday, June 21, 2015

    Problems with Media Files in LibriFox and the Need to Find Good Support

    As I mentioned in my last post, Alex Hirschberg is working on a Firefox OS app to play audio books from LibriVox. While his "Summer of Code" is just beginning, he has already made significant progress on the LibriFox app.  Because of this, we are now confronted with a problem we've encountered for the whole time we've been working with the platform -- it is very difficult to navigate mozilla's labyrinthine network of forums and irc channels to actually find someone in the know who can answer questions.

    I'm hoping we are able to at least partially resolve this problem sooner rather than later, or I fear we may not be able to make the most of Alex's contribution this Summer.

    LibriFox is designed to play audio books, which on the LibriVox website are stored as mp3 files. The problem is that music files are stored in the same format, and the Music app finds these files and automatically loads them into the menu:

    AlĂ­ Primera and Johann Sebas[tian Bach] are music albums, Victor Hugo and Unknow artist are not. As Alex continues working on LibriFox, I would like it to support at least two ways to install books:

    1. Direct download from the LibriVox website using the app.
    2. Copying the audio files from a desktop computer to a USB connected phone.
    To make this work well, we will need to resolve the problem of having the audio books appear in the Music app, and figure out a good place to put the files that get downloaded so that both methods of adding books can be supported.

    I have a 32 Gigabyte SD card in my phone, and I plan to put a lot of books on it as LibriFox develops.  For a first go at this, I created a directory named AudioBooks and copied the audio files for Les Miserables into a subdirectory:

    That's where Victor Hugo is coming from.

    It would save us a lot of time and expense if we could get some guidance from someone at Mozilla on how best to approach this problem.  Our first experiment using a recommended irc channel did not work for us at all.  We posted the same question on each of two successive days, waited 6 hours for someone to answer, and did not get an answer on either day.

    Since I'm hoping these blog posts I'm making will be of use to other folks interested in contributing apps to Firefox OS, I promise to post recommended help resources once I find some that work well.