08 Jan 2015
Some very interesting news out of the USA today, as the GSA (the biggest, baddest procurement agency in the world) has released a Request For Information (RFI) on agile technology delivery.
To shift the software procurement paradigm, GSA’s 18F Team and the Office of Integrated Technology Services (ITS) is collaborating on the establishment of a BPA that will feature vendors who specialize in Agile Delivery Services. The goal of the proposed BPA is to decrease software acquisition cycles to less than four weeks (from solicitation to contract) and expedite the delivery of a minimum viable product (MVP) within three months or less.
In a wonderful “eat your own dogfood” move, the team working on building this new procurement vehicle are themselves adopting agile practices in their own process. Starting small with a pilot, working directly with the vendors who will be trying the new vehicle, etc. If the hidebound old GSA can develop a workable framework for agile procurement, then nobody else has an excuse.
(The reason procurement agencies have found it hard to specify “agile” is that agile by design does not define precise deliverables in advance, so it is damnably hard to fit into a “fixed cost bid” structure. In places where time-and-materials vehicles are already in place, lots of government organizations are already working with vendors in an agile way, but for the kinds of big, boondoggle-prone capital investment projects I write about, the waterfall model still predominates.)
30 Dec 2014
Brian Timoney called out this Tom Macwright quote, and he’s right, it deserves a little more discussion:
…the client side will eat more of the server side stack.
To understand what “more” there is left to eat, it’s worth enumerating what’s already been eaten (or, which is being consumed right now, as we watch):
- Interaction: OK, so this was always on the client side, but it’s worth noting that the impulse towards using a heavy-weight plug-in for interaction is now pretty much dead. The detritus of plug-in based solutions will be around for a long while, inching towards end-of-life, but not many new ones are being built. (I bet some are, though, in the bowels of organizations where IE remains an unbreakable corporate standard.
- Single-Layer Rendering: Go back almost 10 years and you’ll find OpenLayers doing client-side rendering, though using some pretty gnarly hacks at the time. Given the restrictions in rendering performance, a common way to break down an app was a static, tiled base map with a single vector layer of interest on top. (Or, for the truly performance oriented, a single raster layer on top, only switching to vector for editing purposes.) With modern browser technology, and good implementations, rendering very large numbers of features on the client has become commonplace, to the extent that the new bottleneck is no longer the CPU, it’s the network.
- All-the-layers Rendering: Already shown in-principle by Google Maps, tiled vector rendering is moving over the last 12 months rapidly from wow-wizzy-demo to oh-no-not-that-again status. Rather than rendering to raster on the server side, send a simplified version to the client for rendering there. For base maps there’s not a lot of benefit over pre-rendered raster, but there’s some: dynamic labelling means orientation is completely flexible, and also allows for multiple options for labelling; also, simplified vector tiles can serve a wider range of zoom levels while remaining attractively rendered, so the all-important network bandwidth issues can be addressed for mobile devices.
- “GIS” operations: While large scale analysis is not going to happen on a web page, a lot of visual effects that were otherwise hard to achieve can now be pushed to the client. Some of the GIS operations are actually in support of getting attractive client-side rendering: voronoi diagrams can be a great aid to label placement; buffers come in handy for cartography all the time.
- Persistence: Not really designed for long-term storage, but since any mobile application on a modern platform now has access to a storage area of pretty large size, there’s nothing stopping these new “client” applications from wandering far and completely untethered from the server/cloud for long periods of time.
Uh, what’s left?
- Long term storage and coordination remain. If people are going to work together on data, they need a common place to store and access their information from.
- Large scale data handling and analysis remain, thanks to those pesky narrow network pipes, for now.
- Coordination between devices requires a central authority still. Although, not for long, with web sockets I’m sure some JavaScript wizard has already cooked up a browser-to-browser peer-to-peer scheme, so the age of fully distributed open street map will catch up to us eventually.
Have I missed any?
Once all applications are written in 100% JavaScript we will have finally achieved the vision promised to me back in 1995, a write-once, run-anywhere application development language, where applications are not installed but are downloaded as needed over the network (because “the network is the computer”). Just turns out it took 20 years longer and the language and virtual machine are different (and there’s this strange “document” cruft floating around, a coccyx-like evolutionary remnant people will be wondering about for years).
03 Dec 2014
I haven’t had to build CUnit myself for a while, because most of the systems I work with have it in their packaged software repositories, but for Solaris it’s not there, and it turns out, it’s quite painful to build!
- First, the latest version on sourceforge, 2.1.3, is incorrectly bundled, with files (config.h.in) missing, so you have to use 2.1.2.
- Second, the directions in the README don’t include all the details of what autotools commands need to be run.
Here’s the commands I finally used to get a build. Note that you do need to run libtoolize to get some missing support scripts installed, and that you need to also run automake in “add missing” mode to get let more support scripts. Then and only then do you get a build.
wget http://downloads.sourceforge.net/project/cunit/CUnit/2.1-2/CUnit-2.1-2-src.tar.bz2
tar xvfj CUnit-2.1-2.tar.bz2
cd CUnit-2.1-2
libtoolize -f -c -i \
&& aclocal \
&& autoconf \
&& automake --gnu --add-missing \
&& ./configure --prefix=/usr/local \
&& make \
&& make install
Update: The bootstrap file does provide the required autotools flags.
02 Dec 2014
.gif)
Most users of PostGIS are safely ensconsed in the world of Linux, and their build/deploy environments are pretty similar to the ones used by the developers, so any problems they might experience are quickly found and removed early in development.
Some users are on Windows, but they are our most numerous user base, so we at least test that platform preemptively before release and make sure it is as good as we can make it.
And then there’s the rest. We’ve had a passel of FreeBSD bugs lately, and I’ve found myself doing Solaris builds for customers, and don’t get me started on the poor buggers running AIX. One of the annoyances of trying to fix a problem for a “weird platform” user is just getting the platform setup and running in the first place.
So, having recently learned a bit about vagrant, and seeing that some of the “weird” platforms have boxes already, I thought I would whip off a couple vagrant configurations so it’s easy in the future to throw up a Solaris or FreeBSD box, or even a quick Centos box for debugging purposes.
I’ve just been setting up my Solaris Vagrantfile and using my favourite Solaris crutch: the OpenCSW software repository. But as I use it, I’m not just adding the “things I need”, I’m implicitly choosing an environment:
- my
libxml2 is from OpenCSV
- so is my
gcc, which is version 4, not version 3
- so is my
postgres
This is convenient for me, but what are the chances that it’ll be the environment used by someone on Solaris having problems? They might be compiling against libraries from /usr/sfw/bin, or using the Solaris gcc-3 package, or any number of other variants. At the end of the day, when testing on such a Solaris environment, will I be testing against a real situation, or a fantasyland of my own making?
For platforms like Ubuntu (apt) or Red Hat (yum) or FreeBSD (port) where there is One True Way to get software, the difficulties are less, but even then there is no easy way to get a “standard environment”, or to quickly replicate the combinations of versions a user might have run into that is causing problems (libjson is a current source of pain). DLL hell has never really gone away, it has just found new ways to express itself.
(I will send a psychic wedgie to anyone who says “docker”, I’m not kidding.)
21 Nov 2014
Update: New York and Chicago are exploring exactly this approach, as reported in the New York Times: “Regulators in Chicago have approved a plan to create one or more applications that would allow users to hail taxis from any operators in the city, using a smartphone. In New York, a City Council member proposed a similar app on Monday that would let residents “e-hail” any of the 20,000 cabs that circulate in the city on a daily basis.”
Nick Denton has a nice little article on Kinja about Uber and how they are slowly taking over the local transportation market in cities they have been allowed to operate.
[
](http://fs01.androidpit.info/a/52/86/taxi-limo-ambulance-test-bc-5286f6-w192.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;)
it’s increasingly clear that the fast-growing ride-hailing service is what economists would call a natural monopoly, with commensurate profitability… It’s inevitable that one ride-sharing service will dominate in each major metropolitan area. Neither passengers nor drivers want to maintain accounts with multiple services. The latest numbers on [Uber], show a business likely to bring in nearly $1bn a month by this time next year, far ahead of any competitor
BC has thus far resisted the encroachment of Uber, but that cannot last forever, and it shouldn’t: users of taxis in Vancouver aren’t getting great service, and that’s why there’s room in the market for Uber to muscle in.
Like Denton, I see Uber as a mixed bag: on the one hand, they’ve offered a streamlined experience which is qualitatively better than the old taxi service; on the other, in setting up an unregulated and exploitative market for drivers, they’ve sowed the seeds of chaos. The thing is, many of the positive aspects of Uber are easily duplicable by existing transportation providers: app-based dispatching and payment aren’t rocket science by any stretch.
As an American, Denton naturally reaches for the American solution to the natural monopoly: regulated private enterprise. In the USA, monopolists (electric utilities, for example) are allowed to extract profits, but only at a regulated rate. As Canadians, we have an additional option: the Crown corporation. Many of our natural monopolies, like electricity, are run by government-owned corporations.
Since most taxis are independently owned and operated anyways, all that a Crown taxi corporation would need to do is provide a central dispatching service, with enough ease-of-use to compete with Uber and its like. The experience of users would improve: one number to call, one app to use, no payment hassles, optimized routing, maybe even ride sharing. And the Crown corporation could use supply management to prevent a race to the bottom that would impoverish drivers and reduce safety on the roads.
There’s nothing magical about what Uber is doing, they are arbitraging a currently inefficient system, but the system can save itself, and all its positive aspects, by recognizing and reforming now. Bring on our next Crown corporation, “BC Dispatching”.
