Large shapefile performanceThis been a problem for as long as Mapserver has been around, but Mapserver has been so damn fast that for the most part the performance fall-off as files got larger was ignored (if you can render your map in 0.12s on a 2M record file, that's still pretty acceptable).
However, during FOSS4G2007, Brock Anderson reported that Mapserver was actually several times slower than Geoserver for the particular use case of rendering a small map off a large file.
This could not be borne.
The problem turned out to be the way Mapserver handled the SHX file, loading it all into memory for each render. For a very large file, loading the whole SHX file just to pull less than 1% of the records out is a very bad performance bargain. So I re-wrote the SHX handling to lazily load just the bits of the SHX file needed for the features being rendered.
A secondary problem was that Mapserver kept the list of "features to draw" in a bitmap with as many entries as the shape file had records. Then it iterated through that list, at least twice for each render. Counting to several million twice when you only want a couple hundred features is a waste of time. Replacing the bitmap would have been a lot of work, so I replaced the iteration with one about 10 times faster.
The net result was a several-times improvement in speed for small maps rendered on big files. My reference render of 20 features from 1.8M went from a respectable 0.120s to a screaming 0.037s.
Tile-based map access"How do I put my Mapserver layers into Google Maps?"
A fair question. Here's this great mapping user interface, and this great map renderer, they should go together like chocolate and peanut butter. It's possible to do with a relatively thin script on top of Mapserver, but requires some extra configuration steps.
This upgrade cuts the steps down to:- author map file; and
- author Google Maps HTML page.
See the tile mode howto for some examples. It boils down to using the GTileLayer and setting the tileUrlTemplate to point at a tile-enabled Mapserver.
WMS client URL encodingThese were minor patches, but issues that had been bugging me for a while.
The WMS client URL encoding brings Mapserver intro strict compliance with the WMS specification and that will allow it to work with strict servers, of which the ER Mapper Image Server is one.
HTTP Cache-control headersThe HTTP patch allows the user to configure Mapserver to send a
Cache-control: max-age=nnnn header with WMS responses. For clients like OpenLayers, that fetch images in a tiled manner, this should hopefully promote a more cache-friendly behavior, and faster performance.
4 comments:
You never *needed* a thin wrapper around MapServer to use it with Google Maps: it's been possible for years, and there have been a number of published/documented ways of doing it, and people who are actually doing it, going back to before I was even involved in the geoweb.
The new API makes things "dead simple" for idiots, but make no mistake: for the most part, there isn't a significant benefit here for the user who is technically competent enough to copy paste.
Wow, nice job Paul.
While you're at it, want to take a crack at thematic map performance creation as well? I did some test lately that suggested MapServer was n times slower than GeoServer when creating thematic maps, where n is the number of classes used in the thematic map. For example, try to make a one on one comparison doing a topp:states equivalent map with MapServer (and remember to disable antialiasing in geoserver, or else enable AGG in MapServer).
If you want more data, I can give you the mapfiles I used for the test.
Reading back, I see that I wrote just "idiots", which makes it seem like I'm insulting people. What I meant was "idiots like me" -- people who don't have any knowledge of projections, MapServer configuration, etc. and want to set this up without learning any of it. Sorry that I was somewhat... rude there.
Paul, it is great to see you tackling Mapserver with renewed energy! Wow.
Post a Comment