Data Worship and Duty

If you spend more than a few minutes working as an analyst--operations, program, logistics, personnel, or otherwise--it is almost inevitable that some wise military soul will offer trenchant historical lessons about undue trust in analytics for decision making derived from the performance of Robert McNamara as Secretary of Defense. Too often, these criticisms are intended to deflect and deflate criticisms and conclusions of analysis without addressing the analysis itself (an ad hominem approach without so much of the hominem). But that doesn't mean there aren't common mistakes made in the conduct of analysis and worthwhile lessons to be learned from McNamara.

This short article from the MIT Technology Review is a bit old, but it also makes a number of useful points. The "body count" metric, for example, is a canonical case of making important what we can measure rather than measuring what's important (if what is important is usefully measurable at all). Is the number of enemy dead (even if we can count it accurately) an effective measure of progress in a war that is other than total? So, why collect and report it? And what second-order effects are induced by a metric like this one? What behavior do we incentivize by the metrics we choose, whether its mendacious reporting of battlefield performance in Vietnam or the tossing of unused car parts in the river? 

There's something more fundamental going on in the worship of data, though. We gather more and more detailed information on the performance of ours and our adversaries' systems and think that by adding decimals we add to our "understanding." Do we, though? In his Foundations of Science, Henri Poincaré writes:

If we could know exactly the laws of nature and the situation of the universe at the initial instant, we should be able to predict exactly the situation of this same universe at a subsequent interest. But even when the natural laws should have no further secret for us, we could know the initial situation only approximately. If that permits us to foresee the subsequent situation with the same degree of approximation, this is all we require, we say the phenomenon has been predicted, that it is ruled by laws. But this is not always the case; it may happen that slight differences in the initial conditions produce very great differences in the final phenomenon; a slight error in the former would make an enormous error in the latter. Prediction becomes impossible and we have the fortuitous phenomenon. 

Poincare is describing here what would later be dubbed the butterfly effect for nonlinear systems (with the comparison to predicting the weather made explicit in a later chapter). In systems such as these, chasing data is to pursue a unicorn and the end of the rainbow. Rather, it is structure we should chase. Modeling isn't about populating our tools with newer and better data (though this may be important, if secondary). Rather, modeling is about understanding the underlying relationships between the data.

We often hear or read that some General or other should have fought harder against the dictates of the McNamara Pentagon, but one wonders if perhaps such a fight is also the duty of a military analyst.

Know your history ...

A friend who knows my leanings toward math and statistics -- and who understands my professional inclination to read, study, and apply them to military problems -- recently sent me a link to a wonderful article from The Economist, "They also served: How statisticians changed the war, and the war changed statistics."

Aside from the laudatory mention of George Box, whose assertion that "all models are wrong, but some are useful" has done more damage to the profession than any other single statement, this should be essential reading for the members of our smallish profession. Off the top of my head, I can think of at least two other works (other than the marvelous titles already described on this blog as "essential reading" and "books of interest") that should be part of our essential education as military analysts:

The Science of Bombing: Operational Research in RAF Bomber Command by Randall T. Wakelam. Much of our professional identity as a community comes from the mythology of operations (or Operational) Research and its application to the problem of civilizational survival in the Second World War. It seems a good idea to read the actual history of the people, techniques, politics, decisions, and decision makers involved in that history.

Thinking About America's Defense: An Analytical Memoir by Glenn A. Kent, David Ochmanek, Michael Spirtas, and Bruce R. Pirnie. Whether it's the mathematical techniques, the influence of political/historical context on problems of interest, or something more personal, this is an important work for military (especially Air Force) analysts.

There are more, of course. It's hard to tell what the next problem posed to a military analyst will be, so our educations must be necessarily broad. The study of mathematics, statistics, PPBE, doctrine, military history, international relations, leadership, management, theories of innovation, etc., are all important. In this case, though, the question is about the central and defining history of our communal story, our mythology.

What else should we read?

The Analytic Profession in One Tweet

Last week, a blog post by an Army strategist appeared on The Bridge (a marvelous blog that I highly recommend to military professionals of all stripes) that posed the following question:

"How would you define the art and science of our profession in one tweet?"

In this case "our profession" referred to the profession of arms, and the author put forward a compact solution with an attendant explication of his reasons that answered the challenge nicely. There is, I think, more than a little value in an effort like this one, and cutting away the chaff and getting to the heart of who we are, what we do, and why we do it is more than just an interesting intellectual exercise. If done well, it provides a clear and memorable vision that communicates to those on the outside what we do and to those on the inside why and how we do it (whatever "it" might be), in this way creating a professional community centered on the vision. This clarity of vision then has any number of second-order effects on prioritization, training, recruiting, etc., and the effort to create it can pay incredible dividends.

As a member of more than one professional community, though, this line of thinking led me to wonder, "How would you define the art and science of our military analytic profession in one tweet?" I frequently use the phrasing below when discussing the career field among the analysts with whom I work, though I can't claim credit for its composition. Those who know Mike Payne will recognize it and have likely been part of the ongoing conversation that led to it, but the words are his:

"Analysts learn how things work and explain it to others, usually in relation to other things and often quantitatively."

This definition (with 22 characters to spare) captures several critical characteristics of the analytic profession. 

1. It is general. In many cases, we don't have the luxury to consider ourselves as ISR analysts, force structure analysts, operational assessment analysts, etc. Rather, our particular skills will be applied to whatever question is relevant to leadership. 
2. Learning how things work is interesting as a standalone activity, but productive of nothing. Communicating the things we learn to those responsible for making decisions is a critical element of who we are as a community. 
3. Not all analysis is quantitative. There are some tools available to the community of military operations research professionals (mathematical, simulation, etc.) that are in some sense unique, but these are not a sine qua non for analysis. Consider, for example, the analysis given by Graham Allison and Philip Zelikow in their iconic book Essence of Decision. Nary an equation is to be found, but it's difficult to dispute that they are seeking to understand a system and explain it to others who will make decisions. Analysis is something done by analysts, and it is independent of the tools used (except for tool between the analyst's ears). 
4. The systems we study are not isolated, and understanding how they are coupled to other systems is vitally important, both to understand the constraints and restraints imposed by the environment and to illuminate non-proximate effects that may result from changes in the system under investigation. 
5. It's worth noting what this definition does not do. The word "answer" does not appear, for example. Most questions of interest, do not have clean and precise answers, for example, or they have multiple answers that each have merits making them equally palatable but qualitatively different. So, it is generally not possible for analysis of an interesting problem to produce a single, incontrovertibly trues, and perfectly optimal answer. Thus, we explain to senior leaders how the system works to help them better understand the decision space before them, but we rarely provide answers and to chase these chimerae is ... problematic.

That's my 140-character contribution.
Merf

PS ... There's a fairly robust discussion of this question on Facebook among some of the participants in this thread. You can find it here.

Physics of the Future and Force Structure

I've just finished Michio Kaku's Physics of the Future.  In this work, Kaku describes the state of science, technology, and engineering in the areas of computation, artificial intelligence, medicine, nanotechnology, energy, space travel, the meaning of wealth, and the future of human civilization.  He then extends this discussion to speculate on the shape of each discipline and the consequences for life and civilization in the long (2070-2100), mid (2030-2070), and near (present to 2030) terms.

I don't want to dwell on the aspects of Kaku's work that bother me, but I can't resist (very) briefly touching on a few.  

• I'm always a little troubled by the inveterate optimism of some folks (especially those with an unshakable faith in science to make the world better).  This is some of my own pessimism and misanthropy coming through, so my judgement should come with a grain of salt.  I'm not asking for dystopic visions of the future, but Kaku is really unbalanced in his approach.
• My teeth always start to ache when physicists start talking about international relations, psychology, economics and other fields in which they are (at best) dilettantes.  (To be fair, I'm similarly troubled when experts in international relations betray their misunderstanding of science.)  Kaku has a lot to say on these subjects and the pedigree of his conclusions sets off alarms in my little dilettante brain.
• Kaku completely ignores a fundamental aspect of human interaction--war.  (Of course, my calling this element of human interaction fundamental betrays my Hobbesian outlook, but "to thine own self be true.")  This gap is filled, to an extent, by others.  (E.g., The Next 100 Years: A Forecast of the 21st Century by George Friedman is a nice look at the interaction of technology, social change, geopolitics, and war.  Robert Kaplan does some of the same things, though with a lens that doesn't seek to see quite so far.  Etc.)  I just wish a survey of science as wide-ranging as Kaku's touched on military science as well.  (Discursive aside...I think it bears mentioning that any discipline using "science" as a noun modified by some discipline-related word--in a quest for the illusion of rigor that only comes from science in our post-enlightenment minds--will never be an actual science.  Just saying.) 

It's this last that brings me to the reason for my post here.  (The questions that follow are not new, I suppose, but in the interest "read, think, write"...)  What are the implications of the radical changes Kaku foresees and military science, warfare, and therefore the force structures we project and for which we plan?  These changes include: quantum leaps in expert heuristics and expert systems enabled by advances in artificial intelligence and computation; widespread use of driver-less cars, with implications for all sorts of remotely piloted, semi-autonomous, and autonomous vehicles; nano-machines performing medical miracles (and, as Kaku does not note, acting as weapons); profound changes in sources of energy (including the side-effect of nuclear proliferation); expansion of space programs (manned and unmanned...with implications for the military domain, something not noted by Kaku); etc.  Note that he predicts these changes in the near term (by 2030), the first period of his speculation (the mid and late century predictions are much more extreme); this period obviously falls squarely in line with our Air Force long-range force structure timelines.

So...1) Can we meaningfully incorporate these speculations into force structure analysis?  2) Should we do so?  3) If so, how would such speculative force structure analysis work?

Merf

The Point of Know Return...Running Out of Prime Numbers

In April of this year a relatively obscure mathematician named Yitang Zhang set the world of mathematics on it’s ear when he proved that no gap between two prime numbers will ever exceed 70,000,000.  That’s a pretty audacious claim and it must be wrong.  But mathematically he seems to have proven something that many others have tried...and failed to do.  And he appears to have the support of many an esteemed mathematician.  If prime numbers continually get larger, but the gap between them ceases to increase for very large values...and I mean very large values (the largest prime number currently known is 2^57,885,161 − 1, to put that into your calculator you would have to punch 2 x 10 to 57,885,161 and then subtract 1)...will prime numbers eventually run out? Yitang says no...but since prime numbers are the building blocks of all other numbers, it stands to reason that once you run out of gap, you eventually run out of prime numbers between the gap, and then ultimately you run out of numbers.  Does this mean we can stop counting? Can you imagine the end of the number line?  Kind of like the end of the Earth.  There’s a giant waterfall and over you go.  Or, perhaps like the Mayan calendar, maybe we just start counting all over again...it actually isn’t the apocalypse, it’s just a Y2K scare. 

If it’s true, God will have to go back to the drawing board because it seems, he meant to refer to infinity as simply a concept since personally he was never actually able to count that high.  Since Yitang’s proved we still have infinite pairs of prime numbers, even though the gap between them grows not larger then 70 million, his math must be wrong.  Regardless he has created quite a stir since many are believing his math to be correct.  In situations such as these, I tend to cast a skeptic's eye on the situation.  If he’s reached an upper bound, no matter how good his math might be, if it means the end of the number line, either there is no such thing as the infinite or he has made a error. Personally, I hope there is an end to the number line.  That will end our search for things that don’t matter.  Professionally, however, I sense there must be some mistake.  Kind of like neutrinos traveling faster than the speed of light.  Go back and check the math there is an error in the assumption.

What’s left to do, however, is to prove, or disprove, that the number of twin pairs of prime numbers, prime numbers separated by a gap of 2, could be infinite.   The two seem quite different yet are clearly tied together.  If it turns out that prime numbers separated by a gap of 2, are infinite, then again Yitang’s math has to be wrong.  If it turns out that they are not infinite, then his math is correct but his assumption that prime pairs are infinite has to be incorrect, and therefore we eventually run out of numbers. Either way it has to be wrong because we can’t run out of numbers...they are an artificial abstraction that we can always increase by 1.

Therefore the error here could be the limit theorem.  If you chose a number such as “infinity” to approach, you have chosen a bound. Therefore you can find another bound, inside that bound, if you look far enough.  What this means, simply, is that it’s time to expand our definition of infinity...  Instead of the world getting smaller, since we reached the end of the number line under our current definition of infinity, the world just got bigger.  The point of know return, therefore, just got a whole lot stranger.

Reading Lists

Perhaps I should be ashamed of "re-blogging" an item from Tom Ricks' blog on Foreign Policy, but I can never resist a new reading list.  You'll find the reading recommended by the UK's Chief of Defence Staff here.  Of this list, he writes,

"I cannot predict the future. But I can predict that it will test our intellectual mettle. We will have to deal with uncertainty and ambiguity, to decide how best to achieve the necessary outcomes, and to persuade others of the need to act in a timely and effective fashion. We will have to do this in ways that reflect and advance our national interest and make best use of the resources that we are provided with.  This will increasingly require a breadth and depth of contextual understanding, an ability to interpret the lessons of history, agile and creative thinking, and a dedicated professional approach to all that we do, be that on operations or in the office. This web page is designed to tempt readers into developing such attributes. It contains lists of books and articles that will provide intellectual stimulus for those who work in or with Defence, be they military or civilian."

This is not necessarily the most original sentiment, but it is perhaps as close to truth as one can get (at least in this regard).  Mooch has started a nice list here (of both required and interesting reading), but I wonder...

• What is NOT on these lists (the UK's and Mooch's) that should be?
• What IS on the list that should not be?  Why?
• What is it that shapes our views on what comprises the "right" list of recommended reading?  For analysts?  For military professionals?

Essentially--in the spirit of the first element of "read, think, write"--I wonder what our cannon is, what it should be, and why.

STORM and EW?

I received an oddly specific question today, and wondered what this community thinks.  The question dealt with the utility of STORM--my favorite "campaign" model--for assessments involving electronic warfare.  I'm generally not a fan of campaign models for a variety of reasons (OK..."not a fan" may understate the case), but it has been about two years since I dealt seriously with STORM in general and its application to electronic warfare in particular.  So, I'm willing to plead a certain amount of ignorance.  What's the state of this art?

Any thoughts?

Merf