FastCGI Hint

I’m preparing to benchmark / profile Mapserver and PostGIS for the upcoming code sprint in Toronto, and setting up Mapserver as a FastCGI is a requirement to get good profiling results. The JMeter bench marks run multiple threads of load, so having multiple Mapservers running makes things faster.

However, trying to get “FastCgiConfig” to dynamically spawn the required instances was a real pain. Setting the “updateInterval” nice and low made extra Mapserver processes come online a little faster, but in a kind of chunky way. It seemed to kill the existing process before flipping on the new ones. The config line looked like this:

FastCgiConfig -appConnTimeout 60 -idle-timeout 60 -init-start-delay 0 -minProcesses 2 -maxClassProcesses 6 -startDelay 5 -restart-delay 1 -killInterval 30 -singleThreshold 5 -updateInterval 1

In the end, I opted to just statically start the number of processes that made sense for my dual core system (4, in my estimation) using the “FastCgiServer” directive. The config line is a blissfully simple:

FastCgiServer /Users/pramsey/Sites/cgi-bin/mapserv.fcgi -processes 4

Throughput for simple tests (style-free roads from PostGIS, 4 threads of execution) is running as high as 48 maps per second.

Small Mercies

One of the “nice” things about having already lost most of ones investment portfolio is that further drops in the market aren’t nearly so emotionally distressing. 5% of the original investment, that was worth fretting about. 5% of what’s left? Meh.

I have become, in the words of the rock dude, comfortably numb.

Update: Good thing I’m in it for the long term. Oh, wait a minute.


What is this “Cadcorp” of which you speak?

We in the long-homogenized markets of North America find it easy to forget that there is any vendor other than the One True Vendor, but in regional markets all over the world there are strong alternatives available. Cadcorp is a strong regional UK/EU company, GeoConcept is a French company, there are lots of others in markets I know nothing about.

(As recently as 10 years ago, BC still had its own regional GIS software, “PAMAP GIS”, used by the Ministry of Forests and others for spatial analysis. After a valiant (and fatal) attempt to crack into the US Forest Service, PAMAP was purchased by PCI and eventually moth-balled.)

Unlike the “leading brand”, Cadcorp has made it a goal to interoperate with as many other systems, in as many ways, as possible. They were the second proprietary product to include native PostGIS support (the gods of interoperability, Safe Software, beat them to it by a nose), they have been active in the OGC since forever, and they added support for things like GeoJSON, GeoRSS, and the Tile Mapping Specification almost before they existed.

I’ve got Cadcorp on the brain because they recently won a pretty big contract to provide software a bunch of Irish local governments, and the database that is going to underlie this installation is PostGIS.

All that interoperability adds value – one of the things that differentiates Cadcorp’s product is how easily you can use it to directly edit and manipulate PostGIS data. Fulton County, Georgia, was one of the earliest users of PostGIS, and not-coincidentally is also a Cadcorp customer.

Cadcorp has also been hearing from customers that they really, really, really, want to put their rasters into the database (poor buggers), so Cadcorp has taken the next step down the slippery open source slope. They have funded (with cold hard cash to this PostGIS developer) some early work on defining raster storage in PostGIS, and have also allocated staff time to support the development. The WKTRaster project has now got types defined and is starting to implement data importers/exporters. It will be a hot topic of discussion among the PostGIS team at the Toronto Code Sprint next week. If you are interested in rasters in the database, the project is still only partially funded and looking for more backers.

Reading the Signs

WiReD Thin!If you and your circle of acquaintance are still gainfully employed (and be happy, very happy, if you travel in such a fortunate circle) how can you discern that we are in the early stages of an historically significant economic slowdown? What are the physical manifestations of a recession? Here’s one: the copy of Wired I bought when starting out my travel day is fully 50% thinner than the one I bought on my last trip (I tend to only read Wired on airplanes).

What other visible signifiers are there? In the long run, Paul Krugman provides a depressing theory as to what will end this slump – the eventual decay of our physical infrastructure will become so extensive that the effort involved in replacing it will initiate recovery.

WKT Itch

Originally in GeoSpeil.

Eric Raymond says that open source software comes from developers “scratching their itch” — an unpleasant bodily metaphor for “solving their own problems” — a side effect of which is software being released and usable by everyone.

My current itch is attacking the “well-known text” (WKT) representation of spatial reference systems. In theory, an Open Geospatial Consortium standard spatial database like PostGIS stores spatial reference information in the SPATIAL_REF_SYS table, and the actual information about reference system parameters is serialized in a “well-known text” string, stored in the SRTEXT column of that table. In practice, what PostGIS really uses for coordinate transformations is a PROJ4TEXT string in a spare column of the table, and the SRTEXT is just window dressing — we carry it, but we don’t use it.

Using the WKT representation directly is attractive, because it drops needless duplication of information and allows more direct interoperation with things like ESRI “prj” files, which are themselves just WKT serializations. Unfortunately WKT is not as “well-known” as the name would have us believe. Every vendor has used slightly different naming for things like projection operations, parameters, datum names, and so on.

So my itch is multi-fold: I want to be able to parse WKT, I want to learn the technologies necessary to parse WKT (bison and flex), I want to be able to standardize WKT (to strip out the vendor-specific bits) and I want to be able to turn my parsed form into PROJ4 projection objects, because I’ll still be using the PROJ4 engine for transformations at the end of the day. I’m an itchy guy.

So far, I have achieved the parsing and learning-how-to-parse goals, and placed my results in a spike in the PostGIS SVN repository. Next up is standardization, and finally creating PROJ4 objects. Then I’ll try to hook the whole thing into PostGIS.