A brilliant polyglot-historian-political-scientist friend of mine (who is, incidentally, as stateless in fact as I am in spirit) brought an article to my attention that requires comment. The New York Times is pitching Bruce Bueno de Mesquita as the Svengali of political prognostication. He doesn’t really need their help, though, ‘cause he’ll do it himself if you give him a stage and mic.
The article briefly describes his agent-based model of political coalition-building (without using scary terms like ‘agent-based’) and sings the praises of his many and illustrious successes in predicting outcomes for various firms and government agencies. Allow me to debunk, if I may.
-
They say in the article that 40% of the papers in the American Political Science Review use ‘modelling’. I’ll buy that, but very little of it is comparable to what Bueno de Mesquita does. His is agent-based, so it doesn’t begin by trying to come up with an equation that will tie all the relevant variables together in a nice curve on a graph. Instead, models like his start with little agents in a computer simulation, and somebody inputs rules for their interaction and their qualities. You let them figure out how to interact on that basis, and you can play with the parameters to simulate different scenarios. I would buy that 40% of APSR’s papers use either statistical/econometric modelling (cramming variables onto a curve) or two-player game analogies, like the prisoner’s dilemma. If you want to see how different agent-based modelling in international politics can be from either of these approaches, check out Lars-Erik Cederman’s earlier work or Armando Geller’s recent work.
-
He’s presented in the article as if he’s some kind of alchemist who’s stumbled across the philosopher’s stone and has privileged access to underlying political truths of the universe. Ever wanted to be an alchemist? Download
Netlogo (for free!), spend a day or so doing the tutorial and getting familiar with the software, and you’ll be able to construct models just like his. I’ll grant that the trick was more impressive back in the ‘70s, when you’d probably have had to use punch-cards, but the mechanics of the process are the same. Alchemy for everybody!
-
There’s been loads of
criticism directed at the models used by financial firms, economists and regulators lately for ignoring the shaky ground on which they stood. I’ve
touched on this tangentially before. The NYT article, though it presents scepticism about Bueno de Mesquita’s approach in the title of the article, spends almost no space engaging with the model’s limits. Sure, the financial and economic models are more analytic (making inferences based on real data), while Bueno de Mesquita’s are more synthetic (trying to simulate something real based on principles), but is there any reason to believe one is more inherently accurate than the other? I don’t see why there would be.
-
If Bueno de Mesquita can model the next big decision in, it would seem, any given context, why does he stop there? I mean, that would give him the input to model the decision after that, which lets him model the next decision, and the next, and so on. Given enough computing power, he would have a crystal ball able to see arbitrarily far into the future. The article doesn’t even mention why this is a problem. (Even assuming his assumptions make sense and we’re all rational actors, if I know that you are using a model like his to make your decisions, I will use a model that includes your use of a model as a parameter. You will then build my model into your model, which I will then build into my model, etc. Eventually, we have expectations of each other’s behaviour of an
arbitrarily high order (read the first couple of chapters of that link) and put each other out of the modelling business in the process, if you’re interested in that sort of thing.)
-
One of his customers said his model has “intellectual rigor”. In model-guy-code, this means “I don’t understand it, but I take that to be a sign of its refinement and sophistication! (Just please don’t say I have blind faith).”
When faced with such nonsense back in the ‘90s, oh that hallowed decade, we used to say “Dont believe everything that you breathe
You get a parking violation and a maggot on your sleeve”
