18 Aug 2006
Over the past month, I have been trying to compile a list of good case studies of organizations using PostGIS in their daily business. So far, it seems hard to get people out of their shells and say what they are doing – even when you promise to do all the work of writing up the story!
I know from things like the membership of the postgis-users mailing list that there are some big companies using PostGIS. Big names in the geospatial world use it for all kinds of production oriented tasks. But apparently they do not want their stories told.
This is not a problem unique to PostGIS. Other open source projects suffer from the same “shy user” syndrome. I read the postgresql-advocacy list and often see comments to the effect that “my client is a huge company, and they love the performance they are getting from PostgreSQL, but they do not want to be publicly named”.
What will it take to get big organizations to “out” themselves?
03 Aug 2006
It has long been rumoured that ESRI might move their “database neutral” ArcSDE to the ultimate “neutral database”, PostgreSQL. I have heard versions of this idea since around 2003, but I never thought they would come to pass. So, mea culpa, all the people I told “it will never happen”… it has!
Yes, ESRI is currently in the process of developing support for PostgreSQL. We have done all the necessary testing to ensure that this will continue to be a viable product in the future. We plan to release this capability sometime after ArcGIS 9.2.
So, what does this mean for PostGIS? Same thing it means for Oracle Spatial – not very much. ESRI may, or may not, support using PostGIS native spatial geometries as the geometry type in ArcSDE. For Oracle, the default ESRI position has always been their SDEBINARY performs better than SDO_GEOMETRY, so it does not sound like using native types holds any particular allure for ESRI.
Even if ArcSDE does support PostGIS types, the ArcSDE versioning model means that all changes to the geometries will have to be done through the SDE API, in order to ensure the versioning metadata remains consistent.
Still, from a read-only perspective, if ArcSDE does support PostGIS as a geometry type, then the following architecture becomes possible, which could represent a big opportunity for some jurisdictions:
- (DBMS) PostgreSQL Database
- (ESRI Pound of Flesh) ArcSDE for PostgreSQL using PostGIS geometries
- (Desktop Editing / Cartography) ArcGIS
- (Desktop Viewing) QGIS
- (Analysis Engine) GRASS
- (Web Map Publishing) Mapserver
- (Web Feature Publishing) Geoserver
If, on the other hand, ArcSDE on PostgreSQL only supports SDEBINARY, then this will be a non-event from an open source interoperability point of view. I look forward to hearing some reports from the ESRI UC – someone button-hole those ArcSDE developers and find out what the plan is!
29 Jul 2006
About a week ago, my colleague Jody Garnett posted a link to a paper of mine on the OSGeo mailing list. From there, some other folks passed it on, and finally the cycle closed again, as I found a posting-about-a-posting on James Fee’s excellent blog.
The paper itself is a backgrounder for a talk I have been trying with varying degrees of success to shoehorn into 30-minute time slots for the past several years. Any student of the open source GIS world will soon realize that in order to try and talk about it coherently, a great deal will have to be left out. That said, my criteria for taking and leaving have hardly been scientific.
James guesses that my choices have something to do with focussing on “the open source GIS clique that revolves around Refractions and OSGeo”, which is not correct. I include references to projects (QGIS, gvSIG, GMT, TerraLib, OpenEV, OpenMap, Mapbender) which our company has never touched, or which have no association with OSGeo. Like everyone, I write through the frame of my experience, but I do try to get beyond it occasionally.
The original post took issue either with my failure to include a “.Net tribe”, or perhaps with the whole “tribe” concept entirely.
I won’t defend the lack of a “.Net tribe” section. The document was largely written about three years ago, and updated more or less incrementally since then. When I did the initial survey, there was no .Net GIS community that minimally met the standards I was holding the other tribes to. That has changed now. Attendees at my talks this year will have heard me point out that gap in my materials.
I do defend the “tribe” concept, as a way of organizing a talk about open source GIS, because I believe that the “community” is more important than the technology. Do a little association graph of high profile members of the communities and you will find they all clump around particular groups of projects, and those groups tend to be defined by their implementation language.
The same seems to be true of the .Net tribe. As I read the contributor names to those projects, I find few points of reference to contributors in the other tribes, with the exception of a spome people who have gotten involved in the Java side just enough to port things to .Net.
Anyhow, if people are going to read my work and take it seriously, I have a responsibility to update it a little more carefully and frequently. Among the things I noted while doing talks this spring:
Frankly, even with my arbitrary cutting of whole swathes of projects from my “survey” the survey itself is getting unwieldy, which seems to call for yet another approach. I did the survey as a sort of reaction to the FreeGIS approach, which while egalitarian has resulted in an undifferentiated mass of entries. Comprehensiveness can be a curse.
I think my next series of talks or documents will focus on particular combinations of projects that can be stacked into solutions.
10 Feb 2006
A couple months ago I wrote a piece about how uDig made use of a simple web services catalogue to complete the web services publish-find-bind chain of being in a nice clean transparent way. I was very proud, because at the time it was the only example of a decent desktop interface with a proper web services hook.
The times, the are a-catching-up to me! About the same time we were cobbling together our first catalogue of OGC WMS and WFS services, Jeremy Bartley at the University of Kansas was doing a similar thing, except he was mining Google for ArcIMS services. (It should come as some disappointment to boosters of the OGC that he found about 10 times as many ArcIMS servers as we found OGC WMS servers of all types (including ArcIMS).)
Bartley took the results of his mining, and like us, stuffed them into a database for searching in useful ways: by layer and keyword. He exposed the result as Mapdex. For a while, Mapdex was just an HTML web user interface, which made it an interesting novelty, but not exactly an integrated experience.
Then Bartley added a (again, simple!) web services API to the Mapdex database. Now the doors were wide open, and along came the final piece: an ArcMap toolbar that allows direct searching of Mapdex and adding of services to the ArcMap application in real time. So now uDig is not longer unique in providing this particular capability.
I hope this stuff is not being lost on the OGC: facts on the ground are being established, and things that work now will be adopted in favour of things that do not. Complex standards have utility, but if you want them to be adopted you need to provide a real reference implementation that people can deploy easily, without necessarily understanding the standard itself. I would suggest BSD- or MIT- licensed open source code for things like GML parsing, Catalogue client, WFS client, WMS client, OpenLS client, and so on.
The existence of freely-usable reference client code would allow people to quickly enrich their particular client applications with OGC web services ability, making the protocols more relevant and allowing the server makers (who form the bulk of the standards writing pool) to sell more of their product.
Ding! Ding! Ding! Time to wake up, the future is here, and it is not using OGC standards!
06 Feb 2006
I got home from Chicago this afternoon, and though I am a little loopy from travel, I had better write something down before the moment is gone.
It was a very interesting experience to have all these open source people in the same room at the same time. While I had met all of them (with the exception of a couple of Autodesk folks) at different times and places, this was the first time that particular combination of people were all in the same place at one.
The discussion was good, and surprisingly on point the whole time, which is great for a group of 25 opinionated people. Gary Lang of Autodesk did an excellent job ensuring that all the important topics were covered, while letting the discussion move at more-or-less its own pace. The IRC channel was a great forum for carrying on parallel discussions to the verbal points being made. Rather than interrupting a speaker, folks could jot a point into IRC, and if it was trenchant enough it would come up verbally shortly anyways. An interesting example of a meld of old and new forms of discourse.
The summaries at Directions Magazine and Mapping Hacks cover the details of decisions quite well, so I won’t go over that ground again.
One thing that the recaps do not cover is that I was under some pressure on a couple of occasions to bring PostGIS (and to a lesser degree uDig) to the Foundation right away. I recognize that PostGIS is considered a core piece of the open source GIS product stack right now, and that is what is generating the pressure – the desire to have all the “big guns” under the Foundation roof right away. But we have spent a lot of effort to get PostGIS to where it is, and I am not inclined to jump into the Foundation without giving a good deal of thought about the benefits and risks.
As I talked about a couple months ago, our open source work is a company calling card with respect to our abilities: our abilities with the products we make; and our abilities in general. Our projects let potential clients understand the quality of our work and and our enthusiasm, even if they have never met us or even heard of us before: the projects are tangible proof that we are a good company. If we move our projects into the Foundation, the direct link between the projects and ourselves gets severed. Now they are Foundation projects, not Refractions projects, and the “halo” effect is somewhat diminished. Sure, smart folks who read the email lists or the commit logs will figure out that we are doing the work, but that is a long ways from the current fairly obvious linkages between the projects and the company.
The flip side is that taking the projects away from the (one, smallish) company and into the Foundation (with Big Important Companies as funders and backers) adds a new halo of legitimacy to the projects themselves. And therein lies the calculation. Does making the projects look better help us enough to offset the loss of direct project/company linkage?
I do not know.
One of the things I will be doing is talking to people who have some distance from the whole issue, and see how they perceive both ourselves, our projects, and the foundation. I certainly do not have enough distance from the issues to have an undistorted view of the issue, and neither do the people in Chicago who wanted us to join right away. So it will take some time and some thought.
Strangely, it seemed that we were the only organization with this particular quandry. Autodesk actually wants to break the linkage between their open source project and their company. Most of the other projects are classic multi-individual projects that came into being without corporate gestation. One exception is ka-map, which was gestated by DM Solutions, and will eventually put them into the same should we/shouldn’t we cycle as I am in now. But ka-map is still a new project, so people were not pressuring them to jump into the pool right out of the gate.
