Tuesday, April 24, 2007

REST Feature Server

Piling on this meme! Pile!

Chris Holmes has taken a stab at some of the semantics of a REST feature server, and of course, Chris Schmidt has already written one. (!!!!!)

I would like to take issue with one of Chris Holmes' design points:
The results of a query can then just return those urls, which the client may already be caching

http://sigma.openplans.org/geoserver/major_roads?bbox=0,0,10,10

returns something like

<html>
<a href=’http://sigma.openplans.org/geoserver/major_roads/5′>5</a>
<a href=’http://sigma.openplans.org/geoserver/major_roads/1′>1</a>
<a href=’http://sigma.openplans.org/geoserver/major_roads/3′>3</a>
<a href=’http://sigma.openplans.org/geoserver/major_roads/8′>8</a>
</html>
Ouch! Resolving a query that return 100 features would require traversing 100 URLs to pull in the resources. What about if we include the features themselves in the response? Then we are potentially sending the client objects it already has. What is the solution?

It is already here! Tiling redux! Break the spatial plane up into squares, and assign features to every square they touch. You get nice big chunks of data, that are relatively stable, so they can be cached. Each feature can also be referenced singly by URL, just as Chris suggests, but the tiled access allows you to pull more than one at a time.

What about duplication? Some features will fall in more than one tile. What about it? Given the choice between pulling 1000 features individually, and removing a few edge duplicates as new tiles come in, I know what option I would choose. Because each feature in the tile includes the URL of the feature resource, it is easy to identify dupes and drop them as the tiles are loaded into the local cache.

Tiling on the brain, tiling on the brain...

Monday, April 23, 2007

Everybody Loves Metadata

Or, more precisely, everybody loves <Metadata>.

Hardly is KML into the OGC standards process and already folks are getting ready to standardize what goes into the anonymous <Metadata> block.

Ron Lake thinks that ... wait for it ... wait for it ... GML would be an "ideal" encoding to use in <Metadata>.

Chris Goad at Platial thinks that we should be doing content attribution (who made this, who owns this) in <Metadata>.

Even Google is getting into the game. The explanations of how to integrate your application schema for <Metadata> extensions into the KML schema are a nice reminder of the sort of eye-glazing details that have made life so hard for GML. Doing things right is hard.

It is particularly delicious that the very thing that makes adding information to <Metadata> fiddly is the preparation of schemas: you need metadata about the metadata you are adding to <Metadata>.

Where will this all end? I think it will end with the Google Team picking one or a few <Metadata> encodings to expose in their user interfaces (Earth and Maps). At that point all content will converge rapidly on that encoding, and the flexibility of <Metadata> will be rapidly ignored.

Thursday, April 19, 2007

OGC Apologies

I owe an apology to OGC, and the participants of the Mass Market working group. The post below on KML includes information about preliminary decisions that are not the policy of the OGC until they are officially approved by the higher governing bodies of the OGC. So to Mark and Carl and anyone else who has more stress to deal with today than they did yesterday because of me, a heartfelt "I am very very sorry".

Tuesday, April 17, 2007

KML @ OGC

Well, the Open Geospatial Consortium (OGC) Technical Committee (TC) meeting is once again proving to be the Most Boring Meeting in the Known Universe. The best came on the first day, with the plenary session talks, and the Mass Market meeting, with the entrance of Google Earth's KML into the OGC.

I attempted to capture Michael Jones' (CTO Google) explanation verbatim, but am not a professional stenographer. So take the below as a loose quotation:
"Why go this far [submitting KML to OGC]? So we could go further. A lot of people are using Google Earth. It is thrilling to us. It [KML] is a pretty standard format, even though it is vendor controlled. It seemed to us like the Microsoft Word .doc format. Will become more and more a de facto standard. If we could put it into a standards body, they could shape it. It would be a better outcome. We wanted [KML to be] HTML for the geospatial web. Our fledgling efforts have got us this far. We don't want to bail on this [continuing development of KML], but will support a standards effort in the future."
"We thought we could codify the current state [of KML]. And then over the next while work together to build a new KML that is a lot like the current KML, but would be built according to the standards body process. Between here and there, could be mid-points, stops along the way, but that is the path. This [submitting KML 2.1 to the OGC] is not the whole activity, it is the first step along the way."
The quotation captures Jones' excitement about the potential of KML as an interchange format, a way of sharing and linking geospatial data. The process of bringing KML into the OGC will be:
  • The OGC will develop a "position paper" explaining how KML relates to the OGC standards baseline.
  • KML 2.1 will be re-published as an official OGC "best practices document".
  • The OGC Mass Market working group will begin working on KML 3.0.
  • In the meantime, Google may add changes to the KML 2.X stream, and the OGC Mass Market group will decide whether they are worthy of propagation into KML 3.0.
The whole process is anticipated to take less than a year.

The pivot on which all this turns is the idea that KML is similar to HTML, as a content encoding that will be consumed by multiple clients ("spatial browsing software"), not just Google Earth. The market already seems to be well on the way to that end state — there are lots of KML consumers (ArcGIS, FME, World Wind) and producers (ArcServer, PostGIS, Mapguide). Given such an emerging market, KML will be healthier and grow even faster under third-party control (OGC) than under corporate control (Google).

There were some questions after.

The OGC has spent a good deal of time and effort carefully separating concerns in their standards: there are content encoding standards (GML), and styling standards (SLD), and filtering standards (Filter), and behavior standards (WMS/WFS), and they have been carefully kept separate. KML merrily stuffs all those concerns into one pot. Asked about this disconnect, Google reps were unapologetic, saying that the very combination of concerns allows for the simple user experience which makes Google Earth (and by extension KML) so compelling. I agree. Bundling the concerns together makes deployment simple, and does not preclude the over-riding of styling, etc, by the client software.

I had to ask the question that was on my mind: if KML 2.1 was going to be an OGC "best practices" document, but Google was going to continue to add features to the 2.X series of KML, how would the KML 3.0 work re-converge? Basically, was the OGC just going to end up rubber-stamping whatever features Google felt like tossing into the 2.X series in the period prior to the 3.0 release.

Michael Jones answered that vendor extensions are a time honored part of the standards development process, that the innovation around standards is what drives them forward. Citing his time in the OpenGL process, Jones noted that companies would add proprietary extensions to their hardware that were not in the standard, but that the worthy extensions would end up pulled into the standard at the next revision. I replied that there was only one vendor in this case, Google, which made for an unhealthy balance of power. Jones said he had approached Microsoft and ESRI to join in the standardization process of KML, and will continue to invite them to the table. In the end, an OGC process with Google and invitations to the others is better than no process at all. I find it hard to disagree with that.

A similar process has driven development of HTML — it is not always pretty, and sometime people yell at each other, but in the end the standard moves forward and everyone has a known baseline to work against. With more and more applications using and creating KML, it is important for companies to have a solid baseline they can trust.

Tuesday, April 10, 2007

FOSS4G 2007 Registration Open!

Another milestone past! The online registration for FOSS4G 2007 in Victoria is now open, and the conference hotel blocks are available.

Firstly, let me plead with folks who are planning to come to the conference to register early. Knowing the attendance makes all the planning so much easier. So don't wait until the last day before Early Bird rates end to register, just do it now! Please, please, please!

Secondly, let me warn folks who really like to put things off until the last minute. When we booked our hotel blocks, the hotels curiously did not ask for any penalties in the case of our not filling the blocks. In fact, they have a schedule for taking the rooms back away from us and putting them back into their general pool. The implication is that they expect to have no trouble filling all their rooms in that time period, at higher rates than we got for our block. So book your hotel early.

Finally, for those flying in from the USA via Seattle, note that there is a relatively modest amount of seats available each day from the SEA->YYJ run (I calculated about six daily flights of a 40-seat plane). Which is enough probably for all our US travelers coming to the conference — but of course, you won't be the only ones going through SeaTac to Victoria that day. I will not be surprised if all those SEA->YYJ flights end up full quite early during our conference start/end periods.

Happy travels!

Friday, April 06, 2007

Ask the OGC

I'm going to Ottawa in 9 days time, to attend the Open Geospatial Consortium technical working group meetings. I expect to randomly blog things of interest, including of course the submission of KML to the OGC. Anyone have any OGC questions they want asked and answered? No guarantees of answers given, mais oui, we can only do what we can do.

CalGIS Notes: Day 3

The last day of CalGIS was on Good Friday, which was a bit of a bummer. However, it had some redeeming features, notably the brunch, and the diverse set of presentations from 3D world bigwigs to the closing plenary sessions.

Erik Weiker of Microsoft, Michael Jones of Google Earth, and Patrick Hogan of NASA World Wind all gave talks, pushing their various globes within the context of talking about what the future of such technology holds.
  • Arik Weiker of Microsoft had the weakest talk, presentation-wise, as I cannot remember much of it now (typing in the airport bar in Vancouver, waiting for my flight to Victoria). He showed off some nice mash-ups in Virtual Earth, gave a little tour of the app, and pointed up the different between "GIS as practiced by GIS'ers" and "GIS as experienced by ordinary folks". His grandma figured prominently as a user story.
  • Michael Jones of Google gave a clean, crowd pleasing talk that was long on laughs but short on content. The core was similar to Arik's, pointing up the massive size of the audience represented by the aggregate number of Google Earth downloads (over 200 million at this time). His take away message was that the size of this new audience of geospatial consumers represents a huge opportunity for geospatial content providers. And since the audience consisted of a lot of folks who manage and create geospatial content (government agencies largely), it was a worthy message. GIS — it ain't just about printing maps anymore.
  • Patrick Hogan of NASA was the final speaker, and had a different message, though with similar overtones. Again, the idea was the virtual globes are bringing geospatial to the masses. Hogan highlighted the role that World Wind can/is playing in providing an open platform for delivering this content. Because World Wind is open source, and leverages open standards (like WMS), it is (hopefully) leading the virtual globe market to the same standards convergence that NCSA Mosaic led the web browser world through a scant 15 years ago.
Sam and I left before the talk from Sean Walsh, advisor to the Governator, and headed off. And here I am, cooling my heals in an airport bar, stealing WIFI from the executive lounge (in YVR, gate C34 is where you want to be, for reference, on your way to FOSS4G 2007) waiting for the last flight to Victoria.

And then to bed.

Thursday, April 05, 2007

CalGIS Notes: Day 2

The first sessions in the morning were mostly two-hour sessions by the premier sponsors, so I just hung around the booth with Sam. Mid-morning, the short sessions started again.

Justin Lokitz, formerly of Oracle, now of Acquis, gave what must be his standard talk on "enterprise geospatial". For someone who understands the technology, his talk is a bit disconcerting, because he kept talking about moving spatial features from device to database to on-screen map and saying "this does not involve GIS, you do not need a GIS to do this". After a while though, I understood that while he was saying "GIS", he really meant "ESRI". By using the generic term, he was attempting to inoculate against the strong bias in the (standard US spatial user/manager) audience to think that any kind of spatial data work requires ESRI products. His general point was that you could build an application that gathered and managed spatial data using (gasp) only a spatial database a rendering engine and some user interface glue. The examples were Acquis apps, but of course the concept carries over the any number of other mixes of technology.

Next up was another (!!) Farallon consultant, with an app that appeared to automate data interchange between a cost modeling application, GIS database, and scenario manager. A bit of a hair-ball, to say the least, but apparently quite successful at streamlining an ugly prioritization process for road maintenance dollars.

Completing my tour of the Farallon (!!!) staff, was a talk from Adam Lodge, which recapitulated the points they raised on their day one talks, about the many benefits of building business apps against the Google/Yahoo/Microsoft APIs. Adam added some points about the usefulness of open source components that his colleagues largely omitted from their talks. I spoke with him briefly after the talk, and he is a recent refuge from county GIS — he seems to be enjoying his new freedom to work with a range of tools.

A panel discussion on the "future of GIS in government" with folks from San Francisco, counties, and CalTrans was notable for how often "open source" was raised as a point of future investigation and priority. For the managers with keen young staff, doings lots of hands-on exploring, I think great things are ahead. For those who are going to download some stuff, dump it on a server and "see what it does", I predict disappointment. Much of this flows from organizational dynamics, vibrant interesting people do vibrant interesting things, and attract like-minded colleagues.

Sam gave his conflation talk at the end of the afternoon sessions (the "golden hour" *cough*) and was well received. Conflation does not get the respect it deserves. Operational GIS folks thinks that you just need to apply enough clicking desktop monkeys. Algorithmic GIS folks think that it can be automated away into a cloud of digits. In fact, it inhabits a nasty gray area, with unanticipated special cases, the politics of "who's data is 'right'", and algorithmic possibilities all colliding in an interesting stew. Sam's talk is about how to cook that stew so that it's possible to consume it and survive.

The evening included a stroll down to Jack London Square, not worth writing home about (oops), and then dinner at a super fabulous Mexican tapas restaurant (mmmmmmm) just a couple block from the conference center, very much worth writing home about.

And so, to bed.

Wednesday, April 04, 2007

CalGIS Notes: Day 1

I am attending CalGIS, the California GIS conference, this year in lovely Oakland. Sam Smith and I are manning a Refractions booth on the trade show floor, or rather, Sam is manning the booth most of the time and graciously allowing me to go to whatever talks strike my fancy.

Counting on my thumbs, attendance appears to be around 400 to 500, with about 100 of those folks associated with the sponsors on the exhibition floor.

Quick notes on talks:
  • Keynote from Dave Sonnen, IGN analyst. Sonnen is the big name in industry analysts for geospatial, is suitably well connected, and has big generic things to say. Analysts are supposed to predict the future, so the big takeaways are the five things Sonnen says will change the direction of the industry over the next few years.
    • Full spatial data search, the first glimmerings of which we are seeing with the new Google KML search
    • Entry of old-line IT firms into geospatial with "good enough" products to hold onto their existing customer base while adding spatial functionality. Oracle, SAP, IBM, etc. We're already seeing this with Oracle adding their own map renderer, their own slippy map user interface, etc, to their spatial offerings
    • Shortage of skilled geospatial workers
    • Spatial data quality becoming a core concern (I imagine lots of people taking the georeferencing of base maps from the consumer mapping providers as gospel will do wonders for creating data of poor quality)
    • Open source changing the relationship between vendors and customers
    Sonnen really talked up open source, not just in that part of the talk, but others as well, mentioning World Wind as an example of a must-see open source geospatial app. Needless to say, I made sure to button-hole him and invite him to FOSS4G 2007.
  • I caught the tail end of a talk from a Microsoft rep on Virtual Earth. The most interesting bit was a question at the end, asking what he thought made VE better than Google Maps/Earth. He said oblique photos, web browser integration of 3D, roof-top geocoding (said they got data from corporate partners, Fedex, UPS, DHL, who GPSed addresses at delivery time), and better routing (because their data was NavTeq, driven, not TeleAtlas, compiled). The latter points feed into some of the (none-too-subtle) language coming out of the Virtual Earth for Government blog, which avers that Microsoft is doing great work on data quality. We'll see.
  • The Autodesk Map3D product manager gave a good talk on open source, focusing mostly on FDO, and how being open source was benefiting Autodesk and its customers, by providing a faster release cycle and new features from the community. A good talk, it seem well pitched to me, but some of the audience still seemed confused about the idea of a data access abstraction library. It's hard to talk software library concepts to folks who understand software as something that appears on your screen, no matter how carefully you pitch the message. One question: "does FDO run on the desktop, or on the server"? Both, the presenter gently explained.
  • Very nicely presented talk from a Farralon consultant on a basic Virtual Earth web mapping project they did for Boston Redevelopment. Good thoughts on resisting creeping featuritis, focusing on the real end user and stopping there. Good demonstration of the value-adds you get for free building on the VE API (or, as the presenter noted, the GMaps or Yahoo APIs). Good free base data, geocoding, nice UI. Some very nice shots of how VE provides multiple views of the same data. They put some red data dots on their map (yawn), flipped to oblique and there the dots are on the side of the buildings (huh?), flipped to 3D and there the dots are on the tops of the 3D buildings (nice!). Very impressive demonstration of how VE ties together the three views under one API.
  • Afternoon talk from CalTrans on their use of Google Earth. They bought the Whole Enchilada, a complete Google Enterprise license ($200K, he said). Looks like they have a good quantity of their live data accessible to their staff via the interface now, some good pretty pictures. Interesting, gave some reasons for not going with other 3D options: ArcGIS Explorer, too vertically integrated, tied into other ESRI things, trying to get their IT more diversified (I guess you can have too much of a good thing); World Wind, harder to use, more "scientist oriented"; Virtual Earth, more business-to-business or business-to-customer oriented. I just report the news, ma'am.
  • Finally, a talk from a Google employee on KML Regions. Nothing I did not know already, or you could not learn from the API documentation, but a nice overview of what can be done. I'm looking forward to playing with this technology more, particularly in the relatively untapped field of publishing large amounts of vector data progressively via super-overlays.
Took the BART over the San Francisco for a little evening stroll through the city, had dinner at a "gourmet burger" place on Polk, and BARTed back. In the words of Samuel Pepys, "And so, to bed."

About Me

My Photo
Victoria, British Columbia, Canada

Followers

Blog Archive

Labels