Author Archive

Displaying maps as background in gpsman

This is a map with background mage data from vogis and a track i recorded on a hike we did this weekend i created with gpsman.

I already used gpsman for downloading data from my garmin and for transforming and exporting tracks and waypoints and described it in Data transfer from GPSmap 60CSx to a linux PC.

Gpsman also supports loading background images as maps and display items on the map. This will briefly describe how to do that.

The first search on google took me to the gpsman documentation on Map background images.

First i tried to create geo-referenced background images using the command line as it’s suggested in the first paragraph. Therefore i tried something like this:

gpsman georef ebnit-vogis.png "affine conformal" N47.37917 E9.74427 N47.34709 E9.74470 "WGS 84" \0 0 943 682

This did not work out this way (probably because of the missing projection argument) so i read on in the documentation.

Soon i found out that you can very easily do this using gpsman gui.

More or less everything you need to do is

  • find map material on some online service (eg google maps, vogis or tiris) and note down the coordinates of some siginificant points (eg the top-left and bottom-right corner) of the differnt images.

  • create waypoints for significant points you noted down for every image

  • load the first image

    • map / background / load
    • choose the correct transform (i chose “aff conformal” for the vogis maps)
    • choose 2 or 3 (depending which transfrom you chose) waypoints and point them out on the map
  • you can load additional images using map / background / change

    (use the right list to add images not excatly located like the ones in the left column)

  • to store your geo-refernce data for the images use map / background / save geo-ref info

You can also download the resources (gpx file and map backgrounds) if you’re interested in trying this out on your own.

Data transfer from GPSmap 60CSx to a linux PC

Garmin 60Csx connected to my laptop

Last month Lovely Systems and Telesis sponsored me a brand new Garmin GPSmap 60CSx -
thanks a lot!

During evaluations for the Loccata project I previously could play around with a
Garmin eTrex Summit.

This is gonna show you a way to transfer data such as waypoints or tracks from your garmin device
to your PC and the other way round.

There might be nicer applications or easier solutions for this, please let me know about them.

Connecting the device with your PC

According to the GPSMan manual linux kernels > 2.6.11 ship with a
driver for garmin devices (http://sourceforge.net/projects/garmin-gps/)

To see if you’re having the drivers installed and can connect to your GPS device
plug in the cables, whatch your syslog tail -f /var/log/messages and power on
your device. You should see something like:

usb 3-2: new full speed USB device using uhci_hcd and address 3
usb 3-2: new device found, idVendor=091e, idProduct=0003
usb 3-2: new device strings: Mfr=0, Product=0, SerialNumber=0
usb 3-2: configuration #1 chosen from 1 choice
usbcore: registered new driver usbserial
drivers/usb/serial/usb-serial.c: USB Serial support registered for generic
usbcore: registered new driver usbserial_generic
drivers/usb/serial/usb-serial.c: USB Serial Driver core
drivers/usb/serial/usb-serial.c: USB Serial support registered for Garmin GPS usb/tty
garmin_gps 3-2:1.0: Garmin GPS usb/tty converter detected
usb 3-2: Garmin GPS usb/tty converter now attached to ttyUSB0
usbcore: registered new driver garmin_gps
drivers/usb/serial/garmin_gps.c: garmin gps driver v0.23

Now you also know that your device has been attached to /dev/ttyUSB0:

usb 3-2: Garmin GPS usb/tty converter now attached to ttyUSB0

The CD included in the package of the 60CSx only comes with USB drivers
for windows.

If you don’t have the driver you can download it on the sourceforge
project page http://sourceforge.net/projects/garmin-gps/

Luckily i already used a Garmin eTrex Summit before and had it’s
driver installed previously.
These drivers also worked for connecting the 60CSx.

Permission settings

By default users must not read or write from/to the device:

frisi:/ # l /dev/ttyUSB*
crw-rw---- 1 root uucp 188, 0 Dec 29 10:00 /dev/ttyUSB0

We change the permissions so ever user can read and write to the device:

frisi:/ # chmod o+rw /dev/ttyUSB0

Other maybe more elegant solutions can be found here: http://www.gpsbabel.org/os/Linux_Hotplug.html

Applications

Recherche

I’ve been doing quite some research on applications to load data to
and from my garmin gps device.

Some of the links i found are listed on http://del.icio.us/frisi/gis.

The apps i’ll describe in this section are GPSMan and GPSbabel.

I’d love to have a kde application for this, so i considered using GRASS GIS
(which i did not yet try out).

KGIS started in 2002, had no development the last years, so i don’t think this is
very promising.

GPSMan

You can download GPS Manager from it’s official website http://www.ncc.up.pt/gpsman/.
GPSMan is very well documented, a PDF describing it’s features, installation and usage and giving
information on the data types waypoint, track, route and group can be found on the
website too.

Start GPS Manager by providing the device your garmin is located at:

frisi@frisi:~> gpsman usb=/dev/ttyUSB0 &

You will see the main window with buttons named Data, GPS receiver, Map,
Definitions and Options.

Configuration

First configure GPSMan using the Options menu.

  • set the GPS Model to Garmin
  • in Receiver parameters
    • set the Default protocol to Garmin NOT Garmin USB
    • check accept all characters to allow spaces and other chars too
      (eTrex Summit could not handle these, the 60CSx can do)
    • you can leave the other settings untouched
  • in Serial port and files set the serial port to /dev/ttyUSB0 or
    whatever device you got in Connecting the device with your PC

These are the must-have settings to connect your device.
You might want to have a look at the documentation for
more information on the other settings.

Transfer data

To download Data you can either invoke the Receiver window under the
GPS receiver menu or use the menuitems in there (GPS receiver / get / …).

To connect the device hit the check button. If it got connected successfully it
turns green and the label changes to online.


The receiver window

The GPS receiver menu allows you to connect to your device and up- and download data.

To import waypoints simply click on WPoint in the get (download) area (this works the same for Tracks and Routes)
If you want your waypoints to show up in the map check Display items on map (you can also
change this for any item later)

The downloaded items now show up in the Lists menu.
It is accessible via Data / Lists but should pop up automatically
when you start up GPSMan.

Save/export data

To save your Tracklog to display it on maps, share it with friends or simply archive it for computing statistics later on, you can use the save and export functions.


Lists Menu

The Lists menu lets you manage, edit, import and export waypoints, routs and tracks.

I don’t know which is the standard format for sharing waypoints and tracks. GPX which is an XML format seems very promising to me but i don’t know if GPSTrans, MapSend or Ozi are wider used.

For the time being i use save instead of export to save the files in a gpsmap internal format to not lose any information when exporting.

GPSbabel

GPSbabel is an application to convert spatial information given in
a certain data and projection into various other formats.

You can download the most recent version on the homepage http://www.gpsbabel.org.

I’m using GPSbabel not only for conversions but also to upload waypoints
of http://geocaching.com caches to my garmin.

The format of these *.LOC files is geocaching.com-specific
and supported by gpsbabel too: http://www.gpsbabel.org/htmldoc-1.3.0/fmt_geo.html

I used gpsbabel to convert the file GCN8A0.loc and upload it to my garmin device:

gpsbabel -i geo -f GCN8A0.loc -o garmin,power_off=0 -F /dev/ttyUSB0

to get a waypoint file you can use with GPSMan try something like:

gpsbabel -i geo -f GCN8A0.loc -o gpsman -F GCN8A0-gpsman.wpt

you can also download data from your device using gpsbabel, see http://www.gpsbabel.org/htmldoc-development/fmt_garmin.html for more details.

Outlook

Now that i found a way to transfer data from my garmin to the PC,
i want to play with this data and my GPS device a little more.

Statistics

I’m interested in some statistical information on my bike tours such as
distance, average speed, vertical meters up/down, elevation charts.

GPSMan can do a lot of this already:
In the Lists view double click a track. In the track menu that will
pop up you can click the Compute button to get some statistical information
and a menu for generating plots such as elevation sideview by time or by distance.

Maps

GPSMan can display waypoints with different icons and tracks in a map (see screenshot).


The receiver window

GPSMan can also use maps in the background, but i need to find out how to obtain and
use maps for this.

There are also some online services that let you upload tracks and generate
a map with your track.
One of these is http://www.gpsvisualizer.com/map.

There is a tutorial how to integrate a route in google maps on http://www.marengo-ltd.com/gps/.

MapSource

I did not yet play with the map support of the 60Csx (eTrex Summit did not have it).

I need to check if there are any applications for linux, of if i can use it with an
emulator.

According to the Garmin FAQ garmin does not plan to ship mapsource for
other operating systems than windows atm.

First look on InstanceManager

Currently I’m using a lot of different bash-scripts for managing zope
instances.

I was planning to try out InstanceManager (IM) since Reinout’s first post in may 2006,
but it took me until today to have a look at it.

Warm up

I read the documentation on the product page,
Reinout’s first post and another post describing some new features

that really made me excited about trying out IM.

So i immediately checked out svn trunk skimmed through README.txt and INSTALL.txt
and created my first IM-managed instance.

Expectations

There are some expectations i have and certain scenarios IM
has to support to make me forget my old setup and make me use
IM daily instead.

Backups

IM has the backup and restore options. Probably they
do very much the same as my bash wrapper scripts for repozo
i’m currently using.

Since some of our projects are running on the same instance
using differnt databases and IM seems to assume one instance per project
i’m interested if i can find a way to backup them properly.

Collaboration
IM could potentially support collaborative work of different
developers on a certain product.
One can define the Product versions used for this project in the
project configuration file.
Other developers simply check out this config file
and create an identical instance for developing and testing on their
machine.
Development
looks like IM can speed up development a little.
it can update products, [replace the modified database if a default one],
restart zope and optionally quickinstall some products you defined
the configuration
Zeo and Multi-Database support

Our current instances are using zeos and have differnt databases mounted
into main database.

Additionally one ZEO might be used by multiple zope instances.

Let’s see if I can get IM working for out setup.

Evaluation

Talked enough, let’s get our hands dirty on InstanceManager.

Create a Instance

Installing IM is well documented in INSTALL.txt and works like a charm.
It takes some time to find your way in the master configuration file
~/.instancemanager/userdefaults.py and the derived project configuration
file (~/.instancemanager/plone25.py in this example)
However, all settings are well documented - you just need to read a lot at
the beginning.

Backups

instancemanager –backup plone25 creates an incremental
backup of the database to the backup_basedir_template set in
your configuration. if the database has been packed (instancemanager –pack)
in the meantime, a full backup will be created.
(you can create full backups by invoking instancemanager –repozo=full plone25)

As assumed in the beginning, IM acts as a handy wrapper around repozo
similar to the bash wrapper scripts for repozo i use by now.
You can use these actions for your backups by invoking them in a
cronjob (a pack every weekend, and a daily incremental backup).

Restoring backups is damn easy, too: just use the restore action
to get back to the state of the last backup.
you can even choose to restore the database state of a certain
date and time.

restoring a database will stop the zope and zeo server and restore the database
(you have to start zeo and zope again after that):

frisi@frisi:~/instances> instancemanager --restore-date=2007-01-01-21-10-43 plone25
INFO  zope: stop on port 8102.
daemon process stopped
INFO  zeo: stop on port 8100.
daemon process stopped
INFO  Restoring database file /home/frisi/instances/zeo/plone25/var/Data.fs from /home/frisi/instances/zeo/plone25-backup.

I don’t know yet if the transactions within the database are taken into account, or if repozo
is just taking the backup file that fits best, the only thing i tried was to provide the date
and time of the backup file in the format above and the backup file with the given filename
was chosen: 2007-01-01-21-10-43.deltafs was used although 2007-01-01-21-44-27.deltafs existed
in the backup directory.

Support for multiple databases

Neither the –backup nor the –repozo option help me to manage multiple projects (with
differnt databases) within one instance. IM really seems to have a "one instance per project"

policy. Creating a config file for every project that is using the same zopeconf_template and
zeoconf_template and the same sources for products etc. could be a solution, but requires to do
every change (eg when adding or removing a product) on all config files.
Probably i understood something wrong here, and there is a way to support
mutliple projects on one instances. If there is, please tell me!

Collaboration

To really exchange an instance (including products) with other developers,
it should not be necessary to have the product packages downloaded to the
same directories on the probably different machines.

the following only works if every developer has the plone2.5.1 bundle
in the directory /home/<developer>/download/plone:

archivebundle_sources = [{'source':'Plone-2.5.1-final.tar.gz',
                          'productname':'', # not relevant for bundles
                          'develop':False,
                          'droplist':('Five'), # drop Five from this bundle
                          },]
archivebundle_basedir_template = ‘%(user_dir)s/download/plone/’

If IM would support urls here, i could just exchange the
product configuration file and other developers could create exactly the same
instance on their machine.

I’ve had a short chat with reinout and he pointed out that there actually
is a - yet undocumented - way of doing this using
the ‘url’ key in archivebundle_sources. It should work somehow like this, but i did not get
it working in my testsetup:

archivebundle_sources = [{'source':'Plone-2.5.1-final.tar.gz',
                          'productname':'', # not relevant for bundles
                          'url':'http://surfnet.dl.sourceforge.net/sourceforge/plone/',
                          #download it from here, if source is not available in archivebundle_basedir_template
                          'develop':False,
                          'droplist':('Five'), # drop Five from this bundle
                          },]

Development

For the next Zope2 project i work on, I’ll use IM.
I’ll have to figure out a nice structure in svn to check in
configuration files in the repository.

The one button product update and quickinstall multiaction:

instancemanager fresh <projectname>

and the possibility to transfer databases from development to production instance:

instancemanager --repozo=sync <projectname>

will save a lot of time, i think.

There are handy actions for intermediate "I’m going to do something dangerous right now" backups:

instancemanager --repozo=snapshot and
instancemanager --repozo=restoresnapshot

The testing facility will not completely replace the Makefiles I’m currently using
for product development. (see my TemplateProduct for more information)

svn browser does not work yet, but you should be able to check the product out:

svn co https://svn.lovelysystems.com/repos/public/lovely-products/TemplateProduct/trunk TemplateProduct

Zeo and Multi-Database support

Defining multiple mount points using instance manager is possible
in principle:

IM does support custom configuration files. I’m anxious whether
settings influencing zope.conf and zeo.conf such as port are completely
ignored when using zopeconf_template or zeoconf_template

and invoking instancemanager –upgradezope.

In the next days I’ll try to find a way to have different projects
using the same zope instance and report my findings in this blog.

Summary

I think most developers have their custom scripts (be it bash, python or whatever)
that help them to backup databases, setup instances or install certain python modules and so on…

With InstanceManager Reinout started an initiative that offers a documented and standardized
way to manage zope instances.

I’ll try to use IM in the future and try to contribute by writing feature requests, bug reports,
or also coding the functions i’m missing (if they make sense for other ppl too).

If other zope and plone developers do the same InstanceManager will even be more useful
in the future.

I’m planning to contribute for the support of multiple projects within one instance..