09 Oct 2005
Web services are a “big deal” these days, garnering lots and lots of column inches in mainstream IT magazines. The geospatial world is no exception: after years of stagnation the number of number of Open Geospatial Consortium (OGC) web services publicly available on the internet is starting to really explode, going from around 200 to 1000 over the last nine months.
Great news! Except… now we have to find the services so we can use them in clients. The web services mantra is “publish, find, bind”. “Publish” is going great guns, “bind” is working well with good servers and clients, but “find”… now that is another story.
The OGC has a specification for a “Catalogue Services for the Web” (CSW), which is supposed to fill in the “find” part, but:
- It is 180 pages long;
- You also need a “profile” for services, another 40 pages; and,
- The profile catalogues information at a “service” level, rather than at a “layer” level.
The OGC specification is designed to handle a large number of use cases, most of which are irrelevant to spatial web services clients currently in action. By designing for the future, they may have foreclosed on the present, because there are much easier ways to get at catalogue information than via CSW.
For example, if you want a quick raw listing of spatial web services, try this link:
http://www.google.ca/search?q=inurl%3Arequest%3Dgetcapabilities
Instant gratification, and you did not have to read a single page of the CSW standard!
With a little Perl, some PostgreSQL and PostGIS you can turn the results of the above into a neat layer-by-layer database and allow people to query it with a very simple URL-based API:
http://udig.refractions.net/search/google-xml.php?keywords=robin&xmin=-180&ymin=-90&xmax=180&ymax=90
This kind of simple catalogue access became a clear necessity while we were building uDig – there were no CSW servers available with a population of publicly available services, and in any event the CSW profiles did not allow us to search for information by layer, the fundamental unit of interactive mapping.
By using this simple API and properly populated catalogue, we turned uDig into a client that implemented all the elements of the publish-find-bind paradigm, and exposed them with a drag-and-drop user interface. A user types a keyword into the uDig search panel, selects a result from the return list, and drags it into the map panel, where it automatically loads.
So where does this leave the OGC catalogue effort? Still waiting for clients. There are server implementions from the vendors who sponsored the profile effort, but the client implementations are the usual web-based show’n’tell clients, not clients cleanly integrated into actual GIS software. All web services standardization efforts have a chicken-and-egg quality to them, as the network effects of the standard do not become apparent until a critical mass of deployed clients can hit a critical mass of populated servers. Both halves of the equation are required: useful clients, and useful servers.
09 Oct 2005
I will write a technically oriented entry eventually, but right now there are some business issues I still want to explore!
Because most open source proponents are developers themselves, they tend to cast the adoption problem in terms of technical features. The theory goes: “once a product achieves a particular feature level, it will be rapidly adopted, because it is better”. This view has a couple of problems, in that:
- it ignores switching costs entirely (both monetary and emotional); and,
- it assumes the open source product can provide every feature required by the market.
Switching costs are almost insurmountable, regardless of the technology, and regardless of whether the alternative is proprietary or open source. Generally speaking, once a piece of technology is deployed, it is not going anywhere, until it hits end of life. Exceptions abound, of course, but it is a good rule of thumb.
Looking at the open source deployment success stories, most back up this theory: Linux and Apache grew like crazy in green fields, they were rarely replacing existing systems, they were filling in gaps and providing new capability. Again, exceptions abound, but keep your eye on the larger picture.
Feature completeness is a different beast, and it is where open source has a hard time competing with proprietary. Once an organization has gotten over the switching hump, for whatever reason, they have to decide what to switch to. They tot up the lists of features they “need”, and then put out a request to vendors to propose products that meet their list of needs. Bzzt! Strike one for open source, there is no dedicated vendor! Which means either the organization themselves needs to take the initiative to bring open source to the table, or some other company who sees a side opportunity to make money has to bring it in.
Now they compare their feature “needs” with the products. Almost invariably, no product meets 100% of the needs, but that is OK, because vendors can provide a “roadmap” of future development, which hopefully covers their requirements.
Bzzt! Strike two for open source. Open source projects generally make no promises about particular features by particular dates. At best a wish list of future features (“subject to interest and time”) is provided. So open source projects which provide only partial coverage of requirements are rejected. Do not pass go, do not collect any dollars.
The problem is that the technology evaluation and standardization process in many large organizations has been shaped over the years to optimally match the properties of the existing proprietary software market:
- products are provided “finished”, for an up front purchase price;
- products are marketed by vendors, they do not exist in isolation;
- products are owned and supported by a single organization;
- the onus is on the vendor to supply most of the effort in proving the worth of the product (“respond to the following 256 categories in the Request for Proposals”), not the client to evaluate the product independently.
For an organization to start to make optimal use of open source, habits developed over decades have to be stood on their head:
- products are only as good as the client makes them (through the strategic application of money or effort);
- great products must be sought out by the client;
- support may be provided by many vendors, or by the client itself, if their pockets are deep enough to hire a pool of expertise;
- the onus is on the client to evaluate the worth of the product.
But that hardly seems like an improvement! Now the client is doing way more work! And that is a big hurdle in getting people to invert their thinking about product evaluation and adoption. In order to reap the privilege of receiving software without licensing costs, they must shoulder more responsibility for their own technological health. With great privileges come great responsibilities.
08 Oct 2005
People seem surprised when I tell them that our company does open source development and has a couple of large open source projects (PostGIS and uDig). They have a very valid reason for their surprise: there seems to be no reasonable way such projects could make money. In fact, they seem destined to lose money. Lots of it.
Superficially, I cannot help but agree. When I think back on the profile of clients we have added over the past several years, there does not seem to be that much work that ties back directly to either uDig or PostGIS. And yet, the company has been growing pretty steadily over that same time period, and the number of people working on the open source code in one way or another has also been growing. So clearly something is afoot.
I think the answer is in the nature of a consulting company, which is what we are. Ordinarily (open source excluded) our business procedes lock step with the relationships we can develop with current and potential clients. Those relationships lead to referrals, to more work, and so on. The currency of the relationship is competence and trust. We demonstrate our competence and trustworthyness on a project, and gain a referral.
What the open source effort is buying us, over time, is a proxy source of referrals and trust. By using our software for free, clients are gaining a sense of our competence. By asking us questions on mailing lists, they are also gaining a sense of trust in our reliability. So even though our direct revenue stream from PostGIS consulting is pretty small, the indirect stream of trust leads clients to contact us directly for partially related or unrelated work. We don’t necessarily book the work as “PostGIS derived” or a “uDig lead” and then over time we forget the source.
There is something mildly corrosive about this process, because it can potentially lead us to under-invest in our open source work. After all, it appears to be primarily a cost center. Recognizing the benefit requires extra work: noting where we save money by using PostGIS instead of OracleSpatial for project work; sourcing leads back to the open source work, recording it, and recognizing the sales expense that would have been required to generate that lead in a conventional way.
Eric Raymond spelled out a number of business models for “open source companies” in his paper The Magic Cauldron and none of them really fits us well. We use open source as a loss leader (9.1), but not for proprietary software. We offer services around the open source projects (9.3), but they are not our principal source of revenue, nor are they the primary benefit the open source software provides us.
Our model is what I call the “business card model”. Our open source software projects are tokens of our wider capability in understanding and solving geospatial problems. The projects provide us with an introduction to a large number of potential clients, and a (very) small percentage will eventually contact us with work, but the cost of doing the development will still be much smaller than the conventional sales effort needed to generate the same leads. Our open source projects act like a virtual salesman, flying around slapping backs and handing out business cards. And, as an added bonus, we get cool software too!
Open source: it’s cheaper than a salesman, and you can burn it to a CDROM (try that with a salesman).
