Mapserver Debug Logging

Daniel Morissette spills the beans on the mapserver-users list:

IIRC, LOG only logs some info on the mapserv request status at the end of its execution. I don’t use it much and don’t know much about it.

To get debugging output, with MapServer 5.0+, set:

CONFIG “MS_ERRORFILE” “/var/tmp/ms.log”

… and then set DEBUG level (ON, or number between 1 and 5) at the top-level in the mapfile and in each layer for which you want debugging output.

More details are available in RFC-28: http://mapserver.gis.umn.edu/development/rfc/ms-rfc-28

If there is something definitively “bad” about modern Mapserver it is the migration of configuration directives into “magic string” blocks of the map file, which are much less well documented that the “official” elements of the file.

CONFIG, PROCESSING, METADATA, that’s right, I’m looking at you.

Mapserver and Lat/Lon

One of the problems with open source is how much interesting stuff hides beneath the surface, only visible to those willing to read the source code… interesting features you do not even know are there!

On the bright side, you can find these Easter Eggs, if you look.

For example, today I found a case where Mapserver renders projected maps even when the extents you send in are in lon/lat!

My map file looks like this (note the output projection is defined as Mercator):

MAP
  SHAPEPATH "/Users/pramsey/Code/mapserver/msworldtest/"
  IMAGETYPE GIF
  PROJECTION
    "proj=merc"
  END
  LAYER
    NAME continent
    PROJECTION
      "init=epsg:4326"
    END
    TYPE POLYGON
    DATA continent
    STATUS DEFAULT
    CLASS
      OUTLINECOLOR 10 10 10
      COLOR 200 200 200
    END
  END 
END

My request URL looks like this (note the mapext coordinates are lon/lat):

http://localhost/cgi-bin/mapserv?map=~/Code/mapserver/msworldtest/reproj.map&mode=map&layers=continent&mapext=-90+45+0+80&imgsize=500+250

And the output looks like this:

So my request was in geographic coordinates, but my output was still in Mercator.

This is, of course, a brutal bug-in-waiting for someone with a projected coordinate system that happens to include valid requests in the range of (-180,-90 180,90). Mercator does, but a 180x180 meter patch of the Atlantic ocean will probable never be zoomed in on – if it is, the user will suddenly see the whole world, to their great surprise.

That's Billion with a "B"

This article on scaling PostgreSQL to support Skype’s operations is well worth a read for anyone running a high-end PostgreSQL installation.

PostgreSQL is used “as the main DB for most of [Skype’s] business needs.” Their approach is to use a traditional stored procedure interface for accessing data and on top of that layer proxy servers which hash SQL requests to a set of database servers that actually carry out queries. The result is a horizontally partitioned system that they think will scale to handle 1 billion users.

Snapping Points in PostGIS

Fun question on the #postgis IRC channel today, just hard enough to be interesting and just easy enough to not be overwhelming:

Given a table of points and a table of lines, snap all the points within 10 metres of the lines to the lines.

My first thought was “PostGIS doesn’t have that snapping function”, but it actually does, hidden in the linear-referencing functions: ST_Line_Locate_Point(line, point).

OK, that returns a measure along the line, but I want a point! No problem, ST_Line_Interpolate_Point(line, measure) returns a point from a measure.

Great, so now all I need are, for each point within 10 metres of the lines, the nearest line. Yuck, finding the minimum. However, with the PostgreSQL DISTINCT ON syntax and some ordering, it all pops out:

SELECT DISTINCT ON (pt_id)
    pt_id,
    ln_id, 
    ST_AsText(
        ST_line_interpolate_point(
            ln_geom, 
            ST_line_locate_point(ln_geom, vgeom)
        )
    ) 
FROM
(
    SELECT 
        ln.the_geom AS ln_geom,
        pt.the_geom AS pt_geom, 
        ln.id AS ln_id, 
        pt.id AS pt_id, 
        ST_Distance(ln.the_geom, pt.the_geom) AS d
    FROM 
        point_table pt, 
        line_table ln 
    WHERE 
        ST_DWithin(pt.the_geom, ln.the_geom, 10.0) 
    ORDER BY pt_id, d
) AS subquery;

The sub-query finds all the points/line combinations that meet the 10 meter tolerance rule, and returns them in sorted order, by point id and distance. The outer query then strips off the first entry for each distinct point id and runs the LRS functions on it to derive the new snapped point.

Snapperiffic!

See?

One of the things I wanted to do after moving on from Refractions was get back into technology in a “hands on” way again, and the place I most want to get my hands dirty is with PostGIS. It’s all very nice to be a technology evangelist, but very frustrating to have to depend entirely on others to get things implemented. I have to be my own staff now, and that means if I want to play with the guts of PostGIS, I have to learn C.

So that’s what I’m doing. I have my book. I work through exercises. I read the PostGIS code. It’s a slow process, but rewarding as my understanding grows.

For those of you who, like me, have mostly worked in higher level languages, I want to share my C “wow” moment for the week. C has arrays. The syntax is the same as (surprise) all those other languages (Java, Perl, Javascript, PHP) that ape C syntax. Want to iterate through an array? No problem, very familiar, we print out the contents of our array:

for( i = 0; i < sizeof(array); i++ ) {
   printf( "%d\n", array[i] );
}

Now, I knew C pointers were much less abstract than Java pointers, they actually point to memory addresses. Even so, there’s knowing and then there is KNOWING. This routine, that also prints the contents of the array, blew my mind:

for( i = 0; i < sizeof(array); i++ ) {
     printf( "%d\n", *(array + i) );
}

WTF!?!

First, it turns out that the value of the bare “array” variable is just a pointer to the front of the array (how efficient). But the icing on the cake is that you can do math on the pointers! I add 1 to the pointer, and now it’s pointing at the next element, so when I dereference the pointer (with that *) out pops the next value!

All you CompSci majors can have a laugh at my expense (“technopeasant!”), but I’m self-taught, and I have been living in other people’s (Perl, Java, PHP, Avenue (!!!), Javascript) interpreters for many years. This stuff is too cool.