One of the crazy things about actuarial probability is that if you get a large enough population of people together, the odds that something unlikely will happen to any one of them really go up fast. They can get hit by lightning, crushed by a vending machine, killed by a tiger, or more prosaicly suffer a stroke at the age of 49 as Democratic Senator Ben Ray Luján did today (apparently and hopefully he will be fine).
The news last week that US Justice Breyer will resign, and that his replacement can only be confirmed if the entirety of the US Democratic Senate Caucus remains healthy, led me to dust off an old macabre political analysis of group mortality: “what are the odds that the one or more members of the Democratic Senate Caucus will die in the next year”.
Basically the whole thing turns on a pretty simple statement of group probability. The odds that one or more members of a group will die is equal to the inverse of the odds that all members of the group will stay alive.
And the odds of N independent events all happening (like all members of a group staying alive for a period of time) is just the product of all probabilities of each individual event.
To answer the question for the Senate, we just need a few easily sourced pieces of data:
The ages and sexes and parties of all the Senators, from Wikipedia.
Then we can easily construct a spreadsheet to answer the pressing actuarial question of the day.
Probability of one or more Senators dying in next 12 months
Population
Probability
All Parties
87.8%
Democratic
63.0%
Republican
67.1%
D Senator + R Governor
32.3%
R Senator + D Governor
18.4%
It’s been a while since I took probability, but this strikes me as an easy one so I don’t think I got it wrong.
Long story short, a caucus made up of 50 quite old people has a really surprisingly high probability of suffering a death in any given year. The population level Social Security tables probably overstate the mortality of the Senate members though, since Senators are all wealthier than the average American and thus have access to better medical care and generally healthier living conditions.
I have not done the year-end blog post very often, but it’s cold and snowy outside, I have my toes under a knit blanket, and like Oprah I feel like sharing some of my favourite things.
I spent a lot of time in the gym as a young man, and for a while I had the shoulders to prove it, but as is the nature of things over time I invested less time in my physique (and more in family and work) so the muscles mostly melted away.
I always missed the post-gym feeling, but while I accumulated the start of a home gym over the years, I never gave it a lot of use, until Covid came. Working from home, the garage gym is only ever a flight of stairs away, so putting in 3 or 4 hours a week is not too hard.
My overarching goal is to exercise 7 hours a week, so I do 3 or 4 hours of strength training, and fill in the remaining hours with yoga, rowing, walking or (until an injury this spring) running.
In terms of positive returns, this has been one of the best investments I have made in the past 10 years. I sleep better, my formerly sore back is no more, and I have a body that would make my 35-year-old self green with jealousy.
It also turns out I’m in the vanguard of the conventional wisdom! The New York Times seems to publish a newstudy on the superior health benefits of strength training every day.
There is a huge amount of information online about getting started, and I have only one non-conventional piece of advice for new folks: no matter what your age, look at the “weight training over 50” articles. The routines for older people tend to stress “full chain” movements (squats, pull-ups, push-ups) and slower progressions to avoid injury. The result is a stronger core and back which are a key foundation to build general fitness. (And supplemental, be sure to read Zeynep Tufekci’s amazing article on the lies the exercise industry tells women and how everyone can get strong.)
Finally (this has gotten long) doing yoga regularly has allowed me to strengthen all kinds of tiny muscles I never knew I had, but that turn out to be critical to core strength useful in other strength training moves. Since Covid shut my local yoga studio I have been using the Down Dog app on my phone to guide my sessions, which has been money well spent.
Revolutions Podcast
There’s probably an entry for this in the Stuff White People Like (the Stuff Older White Guys Like?) but c’est la vie: Mike Duncan’s Revolutions Podcast has been a constant companion both in the garage gym and while out walking Victoria’s lovely coastline.
Duncan is currently wrapping up his tenth revolution (the Russian revolutions) and this series has been the work of multiple years so there are hundreds of episodes of content.
English Revolution (16 episodes)
American Revolution (15 episodes)
French Revolution (55 episodes)
Haitian Revolution (19 episodes)
Spanish American wars of independence (27 episodes)
July Revolution (7 episodes)
Revolutions of 1848 (33 episodes)
Paris Commune (8 episodes)
Mexican Revolution (27 episodes)
Russian Revolution (70+ episodes)
I found that for historical periods I knew something about, like the French Revolution, the podcast filled in the 90% that I was missing, and it was enthralling stuff. The Revolutionary wars in Europe, the ebb and flow of the power of the Parisian street, the whole period of the Directory! Turns out that just reading Tale of Two Cities doesn’t give you enough backgrounding in the Revolution.
Revolutions would be worth a listen just for the extra context and detail on things I already knew, but the real eye openers have been the revolutions I knew basically nothing about.
OK, I knew there was a Haitian revolution. A slave uprising, right? That’s the sum total I went in with. Revolutions has 19 episodes on the Haitian revolution, and the whole thing is amazing. Tragic, uplifting, depressing, hair-tearing, absolutely worth knowing about.
Similarly the Mexican revolution. Sure, there had to have been one, right? Pancho Villa, he’s a guy who did some stuff? I’m in the midst of it now (Revolution number 9) and the historical echoes into the present day are eerie.
Anyways, the Revolutions Podcast, give it a try. Middle aged white guys and everyone else.
Ursula Le Guin
Not for the first time, but returning to Le Guin this year has been very comforting. Her prose is so clear and unfussy and easy to read, yet also contains so much beauty and observation. In particular I have enjoyed revisiting:
The Dispossessed, which structurally owes a lot to Siddhartha or The Glass Bead Game. The life story of a philosopher finding his way to understanding. But this philosopher lives in a world of perfect equality, a world-wide anarcho-syndicalist society, and as in all (good) sci-fi the fun is in unravelling the ways humans live in this totally foreign millieu.
The Found and the Lost, a huge collection of novellas, ranging from sci-fi pieces set in the Hainish universe to fantasy pieces in Earthsea. They are all excellent, but the one that sticks with me the most is Matter of Segri. A world in which there are 16 women for every man, and the men are kept separate in “castles” from the age of 12 on, “castles” within which they form elabourate dominance heirarchies, torment each other, and practice displays of strength. So, like, a total fantasy cough.
Art Tatum
I am a half-assed jazz pianist, and my interest in the details of the art form comes and goes, but at some point I heard someone say that, if you want to understand jazz piano you have to listen to Art Tatum. In particular, listen to the embellishments.
So I did.
The first thing that hits you is the technical perfection of his playing. Every scale, every arpeggio, the incredible regulation of his touch. I mean, listening to professional piano players, of course the technique is amazing, but Tatum is a notch above.
The second thing, particularly in the early work, where he’s just banging out popular tunes of the time, is the ad-libbed fills and transitions, the embellishments. The harmonic structure he uses often feels appropriate to recordings from 10 or 15 years later on. I totally get where the commentator I read was coming from: Tatum is a precursor to later players.
Check out this jaunty stride rendition of Tea for Two. Crazy fills and asides, and getting increasingly harmonically fun as the piece goes on. Three minutes of happiness.
Anyways, make some time in your streaming playlist for Art Tatum.
It turns out that it is possible to get an indexed n-nearest-neighbor (KNN) search out of PostGIS along with a distance using only one distance calculation and one target literal.
See that?!? Using the column-name syntax for ORDER BY, the <-> operator pulls double duty, both returning the distance to the target list and also forcing an index-assisted KNN ordering.
It's slightly more subtle because our query is finding the nearest object and its distance: `SELECT id, $point <-> geom FROM geoms WHERE foo ORDER BY 2 LIMIT 1` It's in the order by clause but we're also returning the distance as a by-product (and taking advantage of the index)
A couple weeks ago, former BC Liberal MLA and current radio host Jas Johal delivered an epic twitter rant trashing his former party’s record on diversity.
Among other things, he pointed out the distinctive rule the 2018 BC Liberal leadership contest was run under, which granted 100 points for each riding in the province and distributed those points based on the votes within each riding.
Even your 100 points per riding for leadership, is to reduce the impact of South Asian signups.
The net effect of the “100 point rule” is a “one riding one vote” system, rather than a “one person one vote” system.
For those of you joining from outside BC: the South Asian community in BC has an admirable history of political engagement and can wield outsize influence in contests (like leadership elections) which place a premium on signing up and activating new party members. Because the community is concentrated in a few regions, this extra activity results in a some ridings having a lot of new members during leadership contests. Normalizing votes on a per-riding basis has the effect of devaluing votes in these highly active ridings.
Johal’s rant made me wonder though: sure, the (often unspoken) conventional wisdom was that the point system devalued South Asian votes, but in the final analysis, did it?
Incredibly, the riding-by-riding vote results for the 2018 BC Liberal leadership contest are all available in Wikipedia, so it is possible to do a loose analysis.
We cannot know exactly what the visible minority make-up of the BC Liberal membership was in 2018, but we can approximate it by using the 2016 Census profiles for the ridings.
The three largest visible minority categories in BC for 2016 were “Not a visible minority”, “Chinese”, and “South Asian”, so I did analyses for these three groups.
Here’s the overall provincial distribution of those three visible minority categories:
Here’s the distribution of those categories within the BC Liberal membership, calculated by weighting using the riding level distributions against the membership in each riding:
And here’s the scaled BC Liberal membership distribution, where each riding is given equal weight, as in the 100-point leadership vote counting rules.
So, yeah, Johal wasn’t just making this up, the 100-point system is not only anti-democratic in conception (“one vote per riding” instead of “one vote per person”), the system disadvantages the South Asian community in particular.
Having done this lightweight analysis, it is super tempting to try and figure out the counter-factual–who would have won the 2018 leadership under a one-member-one-vote rule–but unfortunately that’s not possible with the data available.
This article (PDF) was published in the August 2002 edition of GeoWorld. I came across my hard copy cleaning my desk and thought it might be of historical interest.
They say you can have your software good, cheap or soon, but you can’t have all three. Information technology (IT) project managers have assumed since the dawn of microchips that any improvement in one measure of software quality must inevitably be accompanied by a reduction in others.
Last year, Ross Searle of the Department of Natural Resources and Mines, Queensland, Australia, faced this “three-horned” IT dilemma. He had a problem, and the solution had to be cheap, quick and good. Searle wanted to create an online permitting application that allowed resource officers in his department to quickly evaluate the environmental consequences of tree-clearing permits.
“In Queensland, our state government has legislation controlling the clearing of trees,” says Searle. “If a landholder wishes to clear trees, then he or she has to apply for a permit. A permit will only be issued if the clearing does not cause environmental degradation. The state government has the role of assessing the permits. To do this, officers need access to a broad range of clatasets all generally held in GIS form.”
The three-horned dilemma loomed. The application had to be cheap—budgets for the department were shrinking, no discretionary funding was available, and all the existing licenses for proprietary Web mapping software were tied up in the departmental head office. The application had to be good—data volumes were huge, encompassing several spatial coverages of more than 500MB apiece, so lightweight solutions weren’t going to work. The application had to be ready soon—Searle didn’t have the time or money to program a complex system from scratch.
Searle slew his three-homed dilemma with a combination of “open-source” tools, using the University of Minnesota (LIMN) MapServer to provide Web mapping capabilities and PostGIS/PostgreSQL as the spatial database backend.
“We are using PostGIS to deliver large amounts of natural resource information via a MapServer interface,” adds Searle, “The MapServer/PostGIS application allows users to quickly search for a parcel of land and bring up all the relevant information in a standard format.
“Apart from the cost factors, I believe that the open-source software for this particular purpose is every bit as good—if not better—than the solutions offered by commercial vendors. We have found that the developers of open-source software are responsive to bugs/suggestions/inquiries, etc., more than commercial vendors. In fact, one of the biggest problems we have found in using open source is keeping up with all the improvements.”
What’s Open Source?
Unlike “freeware” or “shareware,” open-source software provides users with more than just a program and some documentation. As defined by the Open Source Initiative, open-source software “must be distributed under a license that guarantees the right to read, redistribute, modify and use the software freely.”
Open-source programs are distributed along with their “source code,” i.e., the programming instructions that control how the software works. Using open-source software is like eating at a restaurant where the recipes are served alongside the meals—you can simply enjoy the food, but you also have the option of taking the recipe home, changing the seasonings and serving the result to your friends.
Successful open-source projects attract developers interested in improving the software. Sometimes their motives are personal, but often they’re professional—the software helps solve a problem, and improvements to the software make doing their job easier. Through time, success breeds success. The projects with the most development activity attract more developers and become more active, improving and addling features at a rapid rate.
After users become accustomed to having complete access to the inner workings of the software they use, proprietary software begins to feel a little limiting, even unnatural. Bob Young, co-founder of the successful open-source company Red Hat, likes to compare purchasing proprietary software to “buying a car with the hood welded shut.”
“We demand the ability to open the hood of our cars, because it gives us, the consumer, control over the product we’ve bought and takes it away from the vendor,” notes Young.
In Young’s view, the software market should be one in which consumers don’t purchase software per se, but instead purchase whatever services they need to effectively use the software they choose.
Rather than purchase a proprietary database system and then purchase support from the proprietary database company, customers instead choose an open-source database and purchase support from an array of support companies with expertise in the chosen database. The net effect is the same—customers have functioning and supported products—but the balance of power is shifted in favor of customers.
An Open-Source Economy
In a healthy open-source economy, every successful open-source software project should have an accompanying set of companies prepared to offer support and consulting to customers who choose to implement systems with the software. DM Solutions Group, for example, is one of the companies supporting the LIMN MapServer—the open-source Web mapping application Searle used to implement his online permitting application. DM Solutions started as a traditional systems integrator, providing consulting services that implement proprietary software packages.
“We were frustrated with the fact that we were dealing with ‘closed boxes’ that magically did all the work for us,” says Dave Mcllhagga, president of DM Solutions, “If it didn’t do it the way we wanted it to, we couldn’t change it or would have to depend and wait on a third party to take care of any problems.
“Now that we are using open-source software, we’re in full control of the situation and can offer not only consulting services, but also free and open software to base it on. We then can guarantee that if there are any problems in the base software, we can fix them. The word ‘workaround’ no longer is part of our vocabulary. If it doesn’t work, we fix it.”
In addition to providing MapServer consulting services, DM Solutions soon became actively involved in MapServer development, adding new features like OpenGIS Web Map Server, Macromedia Flash and GML support. Mcllhagga notes that effort spent on development actually promotes consulting skills, demonstrating that “we are the industry leaders in use of the product, have a high level of expertise and can therefore offer a premium service to our clients.”
Refractions Research occupies a similar position with respect to PostGIS/PostgreSQL, Searle’s other key application component. As the original developers and current maintainers of PostGIS, Refractions Research occupies a market niche and can offer special expertise to the growing community of PostGIS users. Dave Blasby, principal developer of PostGIS, wonders where it’s all leading.
“If you had told me last year that we would be working for clients in Germany, Florida, Montreal and Los Angeles during the next 12 months, I would have thought you were crazy,” says Blasby.
An Unlocked Hood
Products like UMN MapServer and PostGIS/PostgreSQL gain leverage by building on the prior efforts of other open-source projects. For example, by building on the capabilities of PostgreSQL, PostGIS gains all the strengths of an existing industrial-strength database: transactional integrity, write-ahead logs, Structured Query Language (SQL) and standard application programming interfaces.
Similarly, MapServer garners much of its GIS file format compatibility by using OGR and Geospatial Data Abstraction Library (GDAL) file format libraries. Shared code such as software libraries is extremely important to open-source projects, because it allows all projects to improve together in lockstep as enhancements are made to base libraries.
The author of the OGR and GDAL libraries is Frank Warmerdam, an independent contractor in Ontario, Canada. Warmerdam provides customizations of his many geospatial libraries to software companies and system integrators, who bundle the libraries in their products.
“In many cases, the clients gain substantial leverage from building on an existing open-source library, only needing to pay for the specific improvements they require,” explains Warmerdam. “Clients funding initial work on libraries often gain from testing and improvements provided by later users.”
Ironically, Warmerdam’s open-source TIFF image-format library is the basis for TIFF support in several well-known proprietary GIS products as well as many open-source projects. Even proprietary software vendors can take advantage of the group source libraries provide.
Warmerdam has been working in the software industry for many years and says one of the reasons he now works mainly on open-source contracts “is the sense that I’m building something that will outlast the commercial decisions and market success of any one company.”
Future Solutions
As the foundation of several open-source projects, Warmerdam’s libraries will be used for many years. The sheer quantity of open-source GIS projects available can be appreciated by browsing the entries at FreeGIS.org, a clearinghouse Web site for project information.
Unlike Linus Torvalds, the author of Linux, open-source GIS promoters don’t talk of “world domination”—not even in jest as Torvalds did. Instead, they point to the flexibility that’s the hallmark of open-source software and predict increasing ubiquity behind the scenes.
Dave Mcllhagga points to MapServer as an example of open-source GIS infrastructure with strong momentum, and notes the importance of providing a viable alternative to the status quo.
“There’s a reason why MapServer’s user base has been growing,” notes Mcllhagga. “With the availability of user-friendly tools complementing some technically robust technology, there’s reason to believe it can play a substantial role in this business. If open-source alternatives can continue to improve and at least keep commercial vendors honest, they will be successful.”
Regardless of whether open-source OS ever lands on desktops and workstations, it likely will play an increasing role in meeting the specialized needs of the OS community. Wherever people have problems to solve and a willingness to share their solutions with others, open source will continue to flourish.
