Stop Mapping the World

The Thematic Mapping Blog has been on a 3D chloropleth jag lately, exploring all the ways to display global variables on a map of the world. The technical achievements are all very nice, but the actual display of the useful numbers, is as usual totally screwed by the intractable fact that countries are radically different sizes!

The trouble with chloropleth maps, is that you’re trying to display numerical data visually, but one of the most visually arresting features of your display is a variable that you do not care about, the size of the regions.

What do I get out of the global map display of world data, that I don’t get out of a simple rank-ordered table? A and B are high, and A and B are on the same continent. That could be done by coloring the rank order table by continent. A and B are high, and A and B are adjacent. That’s harder.

I’ve really become fond of the cartogram approach, as a partial solution to this problem, but it has it’s own problems.

We’re in for a whole slew of this kind of stuff, with the US election offing, since the Republicans uniformly win the large less dense counties, every chloropleth map visually overstates the results in one direction, no matter what the cartographers do. How do you show San Francisco County on a map of the continental USA? Does it get 1/2 a pixel even?

X my L

Uh oh, the most unkillable meme in the OSGeo firmament is back, the XML map file. Even more unkillable than the “rasters in PostGIS” meme, the XML map file waits out the winters under the thick bark of .Net and Java programmers and blossoms every six months or so, until knocked back by the harsh frost of having-to-do-it-yourself and a Skeptical Development Community.

Me? I’m all for it. Check out my shopping list for today.

For me, it’s all about human readability.

Government and the Web

Matt Ball recently pointed out an article coming to press at the Yale Journal of Law and Technology, titled “Government Data and the Invisible Hand”. Since I just got back from San Francisco, where I gave a talk titled “Robocop: Public Service in the Internet Age”, it feels like the universe is vibrating on a particular wave length right now, saying

“free your data, let the world play in your garden!”

My talk was itself organized around the ideas presented in the UK “Power of Information” report, which I learned about from a high level civil servant in the BC government.

Vibrate, universe, vibrate!

Coles notes of my talk:

  • public service
  • on the internet
  • is a new medium
  • it requires a new approach
  • serve online communities
  • serve alternative users of information
  • provide open access to your information
  • remove policy barriers
  • expect re-use, encourage it
  • use standard formats
  • become part of the internet

Why use ArcSDE?

Those not on the PostGIS users mailing list will have missed an interesting thread that pulled together some good comments from both the usual suspects and some unexpected participants.

Some choice quotes:

  • “Yes, using SDE effectively castrates the spatial database. It still walks and talks, but it’s a shell of the man it was before. “* – yours truly

  • “The ‘economical’ route for those that want to use PostGIS and have edit capabilities inside ArcGIS desktop would be to use the ZigGIS professional which (from my understanding) implements an editable PostGIS layer in ArcGIS desktop.”* – Rob Tester

  • “If you compare [pricing] with Oracle then yes probably so… But honestly if you are talking about SQL Server vs. PostgreSQL, I think the savings you get from running PostgreSQL would be dwarfed by the cost of SDE.”* – Regina Obe

  • “Spatial databases are being commoditized, but SDE is not a spatial database, it’s a revenue enhancer for ArcMap, and ArcMap is not being commoditized.”* – yours truly

  • “A few disclaimers and background : Until last year, I worked at ESRI. I contributed with some of bug fixes for the PostgreSQL/PostGIS support in the ArcObjects side of things.”* – Ragi Burhum (read this one!)

  • “There is one thing to note in a mixed-client environment and that is this: if a non-ESRI client writes an invalid geometry to a the database then when ArcSDE constructs a query over an area that includes this feature ArcSDE will discover the error (as it passed all queried features into its own shape checking routines) and stop the query”* – Simon Greener (read this one!)

I’m thinking of becoming an instigator. Apparently there’s money in p***ing people off.

Elephant vs Dolphin

Elephant crushes dolphin, but dolphin drowns elephant?

More folks doing real work needing a real spatial database, this time RedFin a real estate information company.

Specifically, we were having some major performance problems with queries that were constrained by both spatial and numeric columns, and all of our attempts to squeeze more performance out of MySQL (including hiring expensive outside consultants) had come to naught.

Guys, if you’re going to hire expensive outside consultants to help with your spatial database problems, they should be (a) me and (b) conversant in PostGIS! :)

Update: It is worth explaining that their main problem, the slow results when the spatial clause was weak, is a direct result of MySQL having an inferior query planner. PostGIS is fast because PostGIS provides good statistics on spatial index selectivity to the planner, and the PgSQL planner is flexible enough to accept selectivity information from extended types. I know this is a big performance problem because before PostGIS had a decent selectivity system, performance bottlenecks like RedFin’s happened all the time.