WKT Itch

Originally in GeoSpeil.

Eric Raymond says that open source software comes from developers “scratching their itch” — an unpleasant bodily metaphor for “solving their own problems” — a side effect of which is software being released and usable by everyone.

My current itch is attacking the “well-known text” (WKT) representation of spatial reference systems. In theory, an Open Geospatial Consortium standard spatial database like PostGIS stores spatial reference information in the SPATIAL_REF_SYS table, and the actual information about reference system parameters is serialized in a “well-known text” string, stored in the SRTEXT column of that table. In practice, what PostGIS really uses for coordinate transformations is a PROJ4TEXT string in a spare column of the table, and the SRTEXT is just window dressing — we carry it, but we don’t use it.

Using the WKT representation directly is attractive, because it drops needless duplication of information and allows more direct interoperation with things like ESRI “prj” files, which are themselves just WKT serializations. Unfortunately WKT is not as “well-known” as the name would have us believe. Every vendor has used slightly different naming for things like projection operations, parameters, datum names, and so on.

So my itch is multi-fold: I want to be able to parse WKT, I want to learn the technologies necessary to parse WKT (bison and flex), I want to be able to standardize WKT (to strip out the vendor-specific bits) and I want to be able to turn my parsed form into PROJ4 projection objects, because I’ll still be using the PROJ4 engine for transformations at the end of the day. I’m an itchy guy.

So far, I have achieved the parsing and learning-how-to-parse goals, and placed my results in a spike in the PostGIS SVN repository. Next up is standardization, and finally creating PROJ4 objects. Then I’ll try to hook the whole thing into PostGIS.

Chesterfield

Some terms start particular and become generic. “Chesterfield”, “kleenex”, “hoover”. Marketers love it when things move the other way, co-opting generic terms. One that drives my up the wall is the shortening of “Microsoft SQL Server” to “SQL”. As in, “we use SQL, what do you use?”. Or “UNIX” to refer to the full sweep of systems, from Sun, IBM, HP, SGI, “that program runs on the UNIX”. I have also spotted “let’s go get a Starbucks” in the wild, which is a disturbing innovation.

Update: In fairness to ignorant abbreviators of “Microsoft SQL Server”, every individual component of the term is pretty generic. Microsoft. SQL. Server. If you didn’t know any better, you might think that “SQL” was the particular portion of the phrase. However, that just proves you don’t know any better. “SQL Server” or “MSSQL” is as short as it goes, suck it up.

Googlesoft Redux

Some might think that there is a subtext (Microsoft is Evil! Google is becoming like Microsoft! Ergo! Google is Evil!) underlying my previous post, so let me disabuse that now: Microsoft is not Evil. Follow the reasoning from there.

However, Microsoft is Big. And Microsoft has a Lot of Money. And Microsoft is looking for New Sources of Earnings. Because Microsoft makes all of its money in One Place (well, two). Sound familiar?

When an organization gets as successful as Microsoft or Google in one market, it has lots of cash and time on its hands to play around in other markets. Ever see a cat play with a mouse? It’s fun… for the cat.

What twigged me was the Google Latitude announcement (incidentally, thanks CBC, for the scary technology story last night). Hey, mobile social networking… just like Loopt!

How did it go down? Like this?

Google: Hm, we’ve got maps on Android and iPhone, we can get location, we should use this for social networking. Hmm, Loopt is already doing that.
Google: Hey Loopt! Wanna be bought?
Loopt: Sure, that’ll be $100M.
Google: Are you insane? In this market? Try $10M.
Loopt: But our investors are already into us for $10M…
Google: Sorry, thems the breaks. We can build it ourselves for less than $10M, if you know what I mean, wink, wink, nudge, nudge, say no more, say no more.
Loopt: sob

Or maybe like this?

Googler A: Let’s build a mobile social networking site!
Googler B: And then play some beach volleyball!
Googler A: Yeah!
Loopt: sob

Really, no matter how the story goes, it ends up with the notional value of Loopt shares falling by 50-80% on the day Google Latitude is announced.

And the investors who dropped their millions into Loopt? What are they going to think the next time someone comes with a great technology idea, maybe a little ahead of its time, but a chance to build out and be ready to rock when the pieces fall into place. Maybe they’ll think: yeah, this could grow into something huge, but as soon as the market matures enough to be valuable, Google is just going to steal it anyways. Pass.

Googlesoft

It’s been obvious for a while, but it took the release of “Google Latitude” to finally shock me into conscious awareness. Google is now the Microsoft of internet services. (Microsoft remains the Microsoft of desktop software. Apple hopes to be the Microsoft of smart phones.)

I’m not referring necessarily to full-on bring-in-the-DoJ monopoly here, but to a level of leadership in market consciousness, and the bottomless pockets, that allow for a form of monopolistic behavior. I remember reading articles in the late nineties about the death of innovation in desktop software, and one of the factors blamed was the “Microsoft effect” – investors stopped supporting innovative new desktop software out of fear of Microsoft. Microsoft would allow you as a third-party developer to have a neat little application, to write some shareware, but if you invented anything strategic of of potentially large market, Microsoft would make its own version, inferior at first, but would eventually crush you and take your market.

And now we have Google, doing the same thing. It’s been a while since Google brought out anything truly innovative, but they sure have shown themselves willing to copy the services of upstart companies and try to snatch their markets away. Sometimes they win (Google Docs) and sometimes they lose (Google Video, Orkut) but this kind of aggressive barging in will eventually dry up the investment ecosystem for web services.

PostGIS and the Public Sector

The EU’s Open Source Observatory and Repository reports on the ESLAP conference in Portugal. Looks like some big public agencies are moving to PostGIS as their spatial data store. We already knew about France’s Institut Géographique National, but to that add Portugal’s Instituto Geográphico Português.