Right, Wrong, and Relevance

I'd like to introduce a young man whose studies in chemistry at the University of London were interrupted by the Second World War. As a chemist, he was assigned to a chemical defense experimental station with the British Army Engineers and undertook work determining the effect of poison gas. There, he was faced with mountains of data on the effects of various doses of various compounds on rats and mice. Since the Army could provide no statistician, and since this bright lad had once read R.A. Fisher's revolutionary Statistical Methods for Research Workers, he was drafted to do the work. Thus was born a statistician who would become the Director of the Statistical Research Group at Princeton (where he married one of Fisher's daughters), create the Department of Statistics at the University of Wisconsin, and exert incredible influence in the fields of statistical inference, robustness (a word he defined and introduced into the statistical lexicon), and modelling; experimental design and response surface methodology; time series analysis and forecasting; and distribution theory, transformation of variables, and nonlinear estimation ... one might just as well say he influenced "statistics" and note that a working statistician owes much of his or her craft to this man whether they know it or not. Ladies and gentlemen, I'd like to introduce George Edward Pelham Box.

George Box

But Box hardly needs an introduction. You already know him and no doubt quote him regularly even if you are not a professional analyst or statistician. Let me prove it to you. George Box, in his work with Donald Draper on Empirical Model-Building and Response Surfaces, coined the phrase, "Essentially, all models are wrong, but some are useful."

We've all heard this aphorism, even if we do not know it springs from George Box. While true in the most fundamental sense possible and a profoundly important insight, what isn't widely understood or acknowledged is how incredibly dangerous and damaging this idea is and why, despite its veracity, we should ruthlessly suppress its use. True but dangerous? How is this possible?

Listen carefully the next time you hear this mantra produced. The key is the manner in which most use the statement, emphasizing the first half as exculpatory ("It doesn't matter that my model is wrong, since all models are") and the latter half as permissive. The forgiveness of intellectual sins implicit in the first half of the statement requires of the analyst or programmatic and planning partisan no examination of the sin and its consequences; we are forgiven, for we know not what we do ... though we should know and should not be forgiven for turning a blind eye.

Once forgiven, the utility of the model is elevated as the only criterion of interest, but this is a criterion with no definition. As such it admits all manner of pathologies that contradict the intent of Box in framing this discussion of statistical methods. Consider a somewhat more expansive discussion of the same concept. In the same work quoted above, Box and Draper wrote,

Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.

And earlier, in a marvelous 1976 paper, Box averred,

Since all models are wrong the scientist cannot obtain a 'correct' one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so over-elaboration and over-parameterization is often the mark of mediocrity.

In each case, the question of utility is tied explicitly to the question of how wrong the model is or is not. Similarly, this is precisely the emphasis in Einstein's famous injunction, often quoted "as simple as possible, but no simpler." He actually said,

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.

In all cases, it is the phenomenon under investigation, the datum of experience, that forms the basis for evaluating the relative utility of the model that is, of necessity, not a perfect representation of the phenomenon. The question is not whether the model is right or wrong, useful or useless. The real question of interest is whether the model is right in the ways that matter. The two concepts, separated by a lowly conjunction in Box's famed quote, are not separate in his intellectual construct. Yet they are very much so in the the framing of the quote as typically (ab)used.

A Spherical Chicken and a Vacuum?

Why does this matter? As a separate idea, "useful" is applied without considering the meaning of the word. Does it mean useful in the sense that it illustrates and illuminates some natural phenomenon, as Box would have it? Or does it mean that the model is simple enough to be understood by a non-technical audience? On the other hand, it may mean that our tools appear to deliver answers to life, the universe, and everything, a chimera some see as worth chasing. Or perhaps it means that the model is simple in the sense that Herbert Weisberg uses to characterize "willful ignorance" which "entails simplifying our understanding in order to quantify our uncertainty as mathematical probability." And it is no great stretch to extend this notion of willful ignorance to the ways in which we frame underlying assumptions regarding the structure of the elements and interactions in a model to facilitate their mathematical and/or digital representation. There is a great physics joke in this vein involving a spherical chicken in a vacuum, but that's a story for another day. If any of these begin to affect our assertions regarding utility, we have crossed over into a territory where utility becomes a permissive cause for intellectual failure, and that is a dangerous territory.

So why write about these things? The answer is simple. These questions affect every aspect of nearly every problem a military analyst will face--whether that analyst is an operations research analyst, an intelligence analyst, a strategist engaged in course of action analysis,etc. Examples abound.

Consider the ubiquitous 1-n list, a model of decision making that problematically imposes a strict, transitive order in preferences, treats cost and benefit as marginal with all the path dependence possibilities that entails, and does not typically account for interaction and dependencies across the list, all of which compromise the utility of the list as a tool. The model is, however, simple, easy to explain, and conveys some sense of rigor in the construction of the list ... even if none exists. Useful indeed.

Or consider the notion of risk as an expected value expressed via the product of probability and consequence. With no meaningful characterizaion of the underlying distribution in the probability half of this formula, risk degenerates to a simple point estimate with no consideration of the heaviness of the probability tails and the relative likelihood of extremity. Or worse, it is implicitly viewed as a Gaussian distribution because that is what we've been taught to expect, and extreme outcomes are unwittingly eliminated from our calculus. On a related note, when considering a given scenario (within the scope of the various Defense Planning Scenarios) and speaking of risk, are we considering the likelihood of a given scenario (by definition asymptotically close to zero) or the likelihood of some scenario in a given class? This sounds a bit academic, but it is also the sort of subtle phenomenon that can influence our thinking based on the assessment frame we adopt. As such, characterizing the output of such a model as a description of risk is specious at best. 

John Maynard Keynes

This isn't the end of the issue vis-à-vis probability, though, and there are deeper questions about the model we use as we seek some objective concept of probability to drive our decisions. The very notion of an objective probability is (or at least once was and probably still should be) open to doubt. Consider A Treatise on Probability, a seminal work of John Maynard Keynes--a mathematician and philosopher of long before he became one of the fathers of modern macroeconomics--or Risk, Uncertainty, and Profit by Frank H. Knight, both first published in 1921. Both, in the formative days of the modern theory of probability, put forward a notion that probability is inherently subjective. Knight, for example, includes in his notion of risk (i.e., probability) the question of confidence: "The action which follows upon an opinion depends as much upon the confidence in that opinion as upon the favorableness of the opinion itself." But if subjective consequence is inherent to assessments of probability and risk, we enter into all manner of human cognitive shenanigans. Does increasing information increase the objective assessment of probability, the subjective assessment of confidence, both, or neither? There is some evidence to suggest the second and not the first, with all manner of consequences for how we conceive of risk (and for notions of information dominance and network-centric warfare). But these are central questions for models of decision making under risk.

Frank H. Knight

Further, the question of consequence is no less problematic. What do we mean by consequence and how do we quantify it (because the probability/consequence model of risk demands an ordered consequence)? And how does the need for a quantifiable expression of consequence shape the classes of events and outcomes we consider? Does it bias the questions we ask and information we collect, shifting the world subtly into a frame compatible with the probability/consequence mode of orienting to it? What are the consequences of being wrong in such a case?

Continuum of Conflict,
2015 U.S. National Military Strategy

There is an interesting corollary relationship between the the numerical output model of risk and 1-n lists in the sense that the numerical output provides a de facto list. Et voila! The model is useful, at least in one sense.

It offers another kind of list, though, based on the Defense Planning Scenarios. Since each scenario is assigned a numerical value, and since real numbers are well ordered we suddenly have a continuum of conflict. This model may be useful--it certainly makes the complex simple--but is it right in the ways that matter? The continuum makes each of the types of conflict shown effectively similar, differing only in degree. Even the implication of such a continuum is dangerous if it leads military planners to believe the ways and means associated with these forms of conflict identical or that one form of conflict can be compartmentalized in our thinking. Perhaps some room should be made for the notion that more is not always simply more; sometimes more is different, but this is an idea explicitly excluded from an ordering like that presented here.

Blind Men Building Models of an Elephant

Another interesting question arises from the ways in which these conflicts are modeled as we seek to develop computations of the consequences in them or to develop recommendations for the force structures best aligned with the demands of give scenarios. How will we represent the scenarios, our forces, the forces of the adversary, and their respective strategies? Will attrition define the objectives, and, if so, what is the model for attrition we will use and how does that model for attrition apply across the continuum of conflict? Will our enemies be volitional, dynamic, and devious or static and inanimate? Will we make simplifying assumptions of linearity, an assumption that sounds esoteric but matters in the sense that a nonlinear model exhibits behaviors a linear model cannot replicate, may be more difficult to develop and interpret, and is also generally more reflective of reality. Stainslaw Ulam's adage--"Using a term like nonlinear science is . . . like referring to the bulk of zoology as the study of non-elephant animals”--is a trenchant reminder of this principle.

Modeling Counterinsurgency
in Afghanistan

But this does not mean linear representations are necessarily inappropriate or without value, and precise emulation can be taken too far. Will we proceed down a path of non-linear interactions and voluminous detail, toeing Box's line of "excessive elaboration," as we often do with large-scale campaign simulations or the (perhaps unfairly) infamous effort to model the dynamics of counterinsurgency in Afghanistan? What does utility mean in each of these cases, and what does "right in the ways that matter" mean here?

Or what about our models of human nature and the international system. Are we classical realists, structural realists, institutionalists, Romantics, Marxists, or something else? The structural realism of Kenneth Waltz is famously parsimonious, abstracting a great deal into billiard balls that interact on the basis of power alone (a description that is itself more parsimonious than is fair). But this leaves us with a model that cannot explain critical phenomena and necessitates expansion and refinement--see Stephen Walt's balance of threat, for example, a socially constructed concept outside the Waltz model. In the end, we are faced with a model and not with reality, with approximations of truth and not with truth itself.

This notion is particularly important in thinking about the veracity and utility of our models. They are, in fact, models. In all cases, the intent is an "adequate representation of a single datum of experience." But in studying our models we can become detached from experience and attach ourselves to the models themselves, associate our intellectual value with their form and behavior, and make them into things worthy of study unto themselves. In short, we are wont to reify them, a process Peter Berger and Thomas Luckman describe as

... the apprehension of the products of human activity as if they were something else than human products-such as facts of nature, results of cosmic laws, or manifestations of divine will. Reification implies that man is capable of forgetting his own authorship of the human world, and further, that the dialectic between man, the producer, and his products is lost to consciousness. The reified world is ... experienced by man as a strange facticity, an opus alienum over which he has no control rather than as the opus proprium of his own productive activity.

Auguste Rodin, The Thinker

This suggests an important remedy to the problem of models that are wrong in ways that matter. If we recognize them as the products of human activity, as opus proprium, and not as handed down from authority, then they can be superseded by new products of human ingenuity. They are not sacred, and when we say a model is wrong, our next though should never be to apologize for the model ("but it is useful"). Rather, our thoughts should turn to whether the model is right in the ways that matter. This is the only proper way to defend our work.

Finally, if the model is wrong, we must demand a new model more closely aligned to the question of interest, a model right enough to be useful. And this is not just a task for analysts and mathematicians, though it is our duty. This is a task for planners, strategists, operators, decision makers, and everyone else. We must seek the truth, even if we may not find it.

First, however, we should probably scrub Box's exculpatory and damaging aphorism from our decision-making discourse.

Belief, Dissonance, and Difficulty in Analysis

René Descartes

In a recent conversation on The Constant Strategist, an acquaintance offered the insightful observation that a lot of reading is really important, but a little reading is actually harmful as people may be taken with the belief that the one book they've read on a subject is the last word rather than what should be the first word on the subject. In this case, the particular subject was the importance of cultural and contextual understanding as an important (if not necessary) prerequisite for effective strategic engagement with another society or nation. But this thought led to a broader reflection on the theory of knowledge (or at least one aspect of the theory of knowledge), cognitive dissonance (with all its myriad side effects), and what these two things mean for analysts.

Baruch Spinoza

Several years ago, I first read a marvelous paper by Daniel Gilbert titled "How Mental Systems Believe" (that you can find online here). The gist of this article is a contrast between theories of learning described by René Descartes and Baruch Spinoza. In a nutshell, Descartes believed that one must first comprehend an idea before one can assess the truth of that idea. In other words, "comprehension precedes and is separate from assessment." Spinoza, on the other hand, dismissed the Cartesian distinction between comprehension and assessment, arguing "that to comprehend a proposition, a person had to implicitly accept that proposition." Only once an idea has been comprehended and believed is it possible to engage in a secondary and effortful process to either certify the idea as true or actively reject its truth. The evidence presented by Gilbert suggests that human beings are, for any number of reasons, Spinozan systems rather than Cartesian systems (or, at the very least are not Cartesian and may be some other type of system in which acceptance is psychologically prior to rejection).

This is all very interesting, but why does it matter? Perhaps the most important answer to that question is an oddity of human cognition commonly known as cognitive dissonance, the mental stress that results from holding "two or more contradictory beliefs, ideas, or values at the same time or" confronting "new information that conflicts with existing beliefs, ideas, or values." How do humans respond to cognitive dissonance? Robert Jervis has a good deal to say on the effects of cognitive dissonance in the milieu of international relations, noting that dissonance "will motivate the person to try to reduce dissonance and achieve consonance" and "in addition to trying to reduce it, the person will actively avoid situations and information which would likely increase the dissonance" so that "after making a decision, the person not only will downgrade or misinterpret discrepant information but will also avoid it and seek consonant information." This gives us such favorite phenomena as the Dunning-Kruger Effect (where the uninformed and unskilled rate their knowledge higher than is objectively accurate), the Backfire Effect (where in the face of contradictory evidence beliefs get stronger rather than weaker), and oh-so-many-more. So, if we must accept as true a concept if we are to understand it, as Spinoza indicates, subsequent rejection of the newly-learned concept is not just effortful but in some sense super-human. This means that reading more may not be useful, and for an inveterate reader this is disheartening. But ...

In another recent post, I raised the idea (shamelessly copied from an analyst far more insightful than I) that the essence of the analysis profession is to understand things and explain them to others. If the very act of learning and understanding drives us to error, though, what are we to do? Does this mean we should throw up our hands and abandon the search for truth? Of course not. But when my acquaintance suggested that we read more, he was only half right. We must absolutely read and study more. But we must also:

• Actively seek out positions different from our own. This includes red-teaming ourselves and exploring the results if each, every, and any combination of our assumptions are wrong. Since, by definition, each assumption we make must be necessary for planning or analysis, changes in those assumptions should change our analysis (else we would state them as facts and not assumptions) and generate a better view of the decision space.
• Train our analysts (and ourselves) as early and as constantly as possible that the mental models we have of the world are themselves assumptions, and then refer to the previous point. This should go a long way toward mitigating the Law of the Instrument (when I have a hammer, problems seem to resemble nails).
  
• Take nothing personally in the search for truth on which no one among us has a monopoly. 

There are probably other things to do, but this seems a good start. The natural question, then, is how we go about doing these things. One answer is simple, but (to shamelessly appeal to the authority of Dead Carl), "Everything is very simple in war, but the simplest thing is difficult." 

I submit that the first, second, and third steps in this journey of a thousand steps are, in the words of a favorite maxim from ADM Stavridis, to read, think, and write. Reading widely brings us new ideas, providing new positions, information, and perspectives (if we consciously seek non-confirmatory writings). Thinking is all about taking in the new information and new models, acting on the assumption that our own might be wrong, and looking for new and informative results. Writing facilitates both of these by putting our thoughts out in the world where they are subject to criticism from those not subject to our own biases, and it is these contradictory views we must learn to cherish since it it easy to find agreement (via confirmation biases if in no other way) but hard to find and use constructive disagreement. Public writing is a way (though not the only way) to find this input. (A disciplined red team can do so as well, as can other well meaning and trusted colleagues.) The final injunction, to take nothing personally, is important. This is an iterative process. If we read, think, and write we start on the right (write?) path, but if we then allow offense to drive use from the debate we will lose the gains we seek and must have.

Vincit omnia veritas ... but analytic truth is first a foe to conquer.