Some words on science...

Recently, I've been re-reading Chaos: Making a New Science by James Gleick. I read it first about 20 years ago, and I'm fascinated by what a difference those years make.  It's not as though this thought of mine is original, of course.  Once upon a time—about 2500 years ago—there lived a man named Heraclitus of Ephesus who said, “You can never step into the same river; for new waters are always flowing on to you.”  Just because we know the idea isn’t new, though, doesn’t detract from the frisson I felt yesterday.  The effect this book had on me as an undergraduate—studying English literature at the time—was electrifying.  In some ways—perhaps romanticizing my own life (a bit)—this book is why I moved from English to Math for my graduate studies.  Yesterday, I received a different kind of inspiration, though.  As I read the second chapter, REVOLUTION, it gave me heart.  Again romanticizing (a bit), I hope I’m in Tolstoy’s camp. 

Of course, I'm at the same time reminded of a quote from Carl Sagan: "But the fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."  So, while I take heart I will also try not to take myself too seriously.

All that by way of prologue, I’d like to share the following from Chaos…

*****

The historian of science Thomas S. Kuhn describes a disturbing experiment conducted by a pair of psychologists in the 1940s. Subjects were given glimpses of playing cards, one at a time, and asked to name them. There was a trick, of course. A few of the cards were freakish: for example, a red six of spades or a black queen of diamonds.

At high speed the subjects sailed smoothly along. Nothing could have been simpler. They didn't see the anomalies at all. Shown a red six of spades, they would sing out either "six of hearts" or "six of spades." But when the cards were displayed for longer intervals, the subjects started to hesitate. They became aware of a problem but were not sure quite what it was. A subject might say that he had seen something odd, like a red border around a I black heart.

Eventually, as the pace was slowed even more, most subjects would catch on. They would see the wrong cards and make the mental shift necessary to play the game without error. Not everyone, though. A few suffered a sense of disorientation that brought real pain. "I can't make that suit out, whatever it is," said one. "It didn't even look like a card that time. I don't know what color it is now or whether it's a spade or a heart. I'm not even sure what a spade looks like. My God!"

Professional scientists, given brief, uncertain glimpses of nature's workings, are no less vulnerable to anguish and confusion when they come face to face with incongruity. And incongruity, when it changes the way a scientist sees, makes possible the most important advances. So Kuhn argues, and so the story of chaos suggests.

Kuhn's notions of how scientists work and how revolutions occur drew as much hostility as admiration when he first published them, in 1962, and the controversy has never ended. He pushed a sharp needle into the traditional view that science progresses by the accretion of knowledge, each discovery adding to the last, and that new theories emerge when new experimental facts require them. He deflated the view of science as an orderly process of asking questions and finding their answers. He emphasized a contrast between the bulk of what scientists do, working on legitimate, well-understood problems within their disciplines, and the exceptional, unorthodox work that creates revolutions. Not by accident, he made scientists seem less than perfect rationalists.

In Kuhn's scheme, normal science consists largely of mopping-up operations. Experimentalists carry out modified versions of experiments that have been carried out many times before. Theorists add a brick here, reshape a cornice there, in a wall of theory. It could hardly be otherwise. If all scientists had to begin from the beginning, questioning fundamental assumptions, they would be hard pressed to reach the level of technical sophistication necessary to do useful work. In Benjamin Franklin's time, the handful of scientists trying to understand electricity could choose their own first principles—indeed, had to. One researcher might consider attraction to be the most important electrical effect, thinking of electricity as a sort of "effluvium" emanating from substances. Another might think of electricity as a fluid, conveyed by conducting material. These scientists could speak almost as easily to laymen as to each other, because they had not yet reached a stage where they could take for granted a common, specialized language for the phenomena they were studying. By contrast, a twentieth-century fluid dynamicist could hardly expect to advance knowledge in his field without first adopting a body of terminology and mathematical technique. In return, unconsciously, he would give up much freedom to question the foundations of his science.

Central to Kuhn's ideas is the vision of normal science as solving problems, the kinds of problems that students learn the time they open their textbooks. Such problems define an accepted style of achievement that carries most scientists through graduate school, through their thesis work, and through the writing a of journal articles that makes up the body of academic careers. "Under normal conditions the research scientist is not an innovator but a solver of puzzles, and the puzzles upon which he concentrates are just those which he believes can be both stated and solved within the existing scientific tradition," Kuhn wrote.

Then there are revolutions. A new science arises out of one that has reached a dead end. Often a revolution has an interdisciplinary character—its central discoveries often come from people straying outside the normal bounds of their specialties. The problems that obsess these theorists are not recognized as legitimate lines of inquiry. Thesis proposals are turned down or articles are refused publication. The theorists themselves are not sure whether they would recognize an answer if they saw one. They accept risk to their careers. A few freethinkers working alone, unable to explain where they are heading, afraid even to tell their colleagues what they are doing—that romantic image lies at the heart of Kuhn's scheme, and it has occurred in real life, time and time again, in the exploration of chaos.

Every scientist who turned to chaos early had a story to tell of discouragement or open hostility. Graduate students were warned that their careers could be jeopardized if they wrote theses in an untested discipline, in which their advisors had no expertise. A particle physicist, hearing about this new mathematics, might begin playing with it on his own, thinking it was a beautiful thing, both beautiful and hard—but would feel that he could never tell his colleagues about it. Older professors felt they were suffering a kind of midlife crisis, gambling on a line of research that many colleagues were likely to misunderstand or resent. But they also felt an intellectual excitement that comes with the truly new. Even outsiders felt it, those who were attuned to it. To Freeman Dyson at the Institute for Advanced Study, the news of chaos came "like an electric shock" in the 1970s. Others felt that for the first time in their professional lives they were witnessing a true paradigm shift, a transformation in a way of thinking.

Those who recognized chaos in the early days agonized over how to shape their thoughts and findings into publishable form. Work fell between disciplines—for example, too abstract for physicists yet too experimental for mathematicians. To some the difficulty of communicating the new ideas and the ferocious resistance from traditional quarters showed how revolutionary the new science was. Shallow ideas can be assimilated; ideas that require people to reorganize their picture of the world provoke hostility. A physicist at the Georgia Institute of Technology, Joseph Ford, started quoting Tolstoy: "I know that most men, including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives."

Science

Since engineering is the corner stone of analysis, for the most part, most quality analysis is conceived and executed in this domain. What’s lacking within the engineering domain is the desire to look beyond what has been done before. Fortunately we have science.  The scientific method lies at the heart of discovery and is so important it is almost shameful that it is the most often over looked aspect of good analysis. To seek the truth, by definition, we have to cast the widest net across the solution space as possible. To cast the widest net we must look beyond the solutions that are obvious, because quite frankly, those solutions are obvious. Fortunately there is a scientific method available to us. Again, we could not do justice to the scientific method a few blog essays, however, we can briefly outline what should be accomplished.

First establish a hypothesis for what it might be that you want to discover or at the least a hypothesis that suggests there might be something out there that you wish to discover.

Next run back to first principles. What are the governing physical laws that either constrain or open up your solution space to other things? The best way to know the truth is to know if the physics work. If the first principles work, by physical law, whatever you might be studying will also work, or at least be in the decision space. If you reduce it to first principles and are solid here, you most likely will be solid in your solution.

Next comes the investigation. This is the fun part and really consists on putting on your lab coat and becoming a scientist. Poking and probing at things. Taking them apart and putting them back together in different ways. Measuring and taking temperatures. Watching for color changes and checking to see when things either boil or break. And then documenting all the results. This will be the process that leads to the discovery of new things. In relentless pursuit of the truth, you must have the curiosity to explore these heights and depths. This is your chance to sail on the HMS Beagle and document and name yet undiscovered life in your decision space. Why would anyone ever give up this part.

If you discover new life, publish your findings. Perhaps your new life will ultimately not be the truth or the solution to the problem you were seeking, but it is part of the process that will add to your job satisfaction. This is also not the place to take all the credit. This is the time when the intellectual credit should be shared with your team members. Everyone will want to share in the discovery of something new.

Through publication you are naturally putting forth your work for peer review. This also is a major factor in the scientific method and should not be avoided. Although sometimes in the heat of battle it is difficult to find the time to get this done. You will be happier if your discovery begins to have credibility in a larger circle. Or, if it can’t stand up to peer review, you can gracefully back down. If you have found the truth, you should not fear others either working to found the same truth – because they will in fact discover the same thing. If they do not, then you have not either, and this is a good thing to know.

It will be reassuring to note that the truth will hold up if you have logically arrived at your assertions and your first principles are firm. If there is a dispute of the truth at this level, logically speaking, someone is in error, and that can be proven with empirical fact. It just means the experiment should be repeated. Again, the truth should never be feared. If you have been in error, you want to know this. If the other guy is in error then you want to know this as well.

Engineering

Engineering is the foundation for good analysis. You do not necessarily have to be an engineer to be an analyst, and sometimes engineers do not make good analysts because they lack other important skills required in the process. But engineering principles form the basis of what we would like to call formulaic analysis. In engineering the basic equations for various applications have already be derived and committed to books or tables to be picked up and reused when an engineer is confronted with a problem. It is the engineer’s job to pick what formula applies and then to use it correctly. To do this incorrectly, as a Primitive Pete, results could be catastrophic. This is why engineering has levels of expertise and in some disciplines, such as civil engineering, a Professional Engineering licence is required. These are when applications of safety and human life depend on the engineer doing his job correctly.

Nothing of this sort exists in the wide open discipline of analysis. We are free to call ourselves analysts with absolutely no certification whatsoever. We are not calling for certification or stronger safeguards in the problem solving profession because we believe analysis is so much more than the engineering/formulaic side to the field. It is, however, at the end of the day, the best approach for cranking through a known problem and getting to a solution. So once the problem is properly defined and we know how to approach it, let the engineers take over and crank out the solution. It is important to point out that there are a few rules that should be followed.

This blog is not the rule book for how to solve problems or to do the correct engineering but we should mention a few good habits to an effective formulaic approach in order to do the job correctly. If we fail in applying the right technique and then fail in executing the technique correctly, we will never find the correct answer and hence we will produce nothing close to the truth. So here just a few rules you should follow:

Establish the objectives of your study as early as possible. As engineers we learn to state the problem, state what is given, then list our assumptions, then you can begin listing tools for consideration.

Plan your study. Your study plan does not have to be perfect. We always say a study plan will be complete when the study is complete.

Decompose the study into bite size pieces. And then break each piece into the smallest component for which a solution might be possible.

Obey physical laws. Rule out early that which falls into the realm of the supernatural. These might very well have been considered during the scientific and artistic phases of your work, but when it’s time to make something fly, engineers will have to design and build it.

Put processes into place that are both verifiable and and can be repeated many times. This distinguishes you from the artists who may only have to build something once or the scientist who many only have to repeat his experiment a few times.

Always have some form of review in place where a Grey Beard analyst can take a look at what you plan to do.

Finally, document everything that has been done so it can be repeated and stand up to scrutiny later.

Science, Art and Engineering: Part II

My response to this earlier post was too long to put in a comment:  

Mooch, I'll take your Brooklyn Bridge and raise you one Hagia Sophia! Granted there are many examples like this from throughout the centuries, but my personal experience of standing inside it's great dome and imagining, this place was built in the early 6th century, is quite inspiring; it's a magnificent achievement and a beautiful place. They employed new building theories, new building materials, they were operating on the cutting edge of engineering. Oh and they didn't compromise on making it beautiful! According to my guide when I was there (though not documented on wikipedia per se) the dome did initially collapse -- it was the largest span anyone had ever built -- but they went back to the drawing board and tried again. (There are a number of other documented collapses, though these were do to natural disasters -- earthquakes, fires.)

When I was 9 I started taking art classes at The National Academy of Design (they have a nice museum a block from the Guggenheim in a beautiful old mansion -- overlooked, but highly recommended). Every Saturday morning from 9 - 12 I would draw and paint. Once I started in Highschool I started taking one of the adult figure drawing classes -- the human body is one of the most challenging, beautiful "objects" to draw. It was a wonderful outlet. For highschool, I applied and was accepted to one of the best performing and visual art schools in the country, Laguardia and always wonder who I would be today had I attended, versus staying at the school I did. The other day, I noticed Tim Booher sketching in our branch meeting and it motived me a bit; I haven't drawn in years but am motivated to start again, to exercise that other half of my brain...

Anyway, I think Mooch touched on a touchy point for me: I think our universities and colleges are doing a great diservice in regards to creating diverse graduates. When I went to The Johns Hopkins University I was looking forward to not only studying math and science, but history and philosophy as well. I was in for a rude awakening: the curriculum was just not geared for cross-disciplinary study. Of course there were a number of cookie-cutter humanities classes we were expected to take in order to get our "H" credits, but none of them were particularly interesting, challenging or rigorous -- the humanities equivalent of "rocks for jocks". Hopkins is a great school and there are some amazing intelligent people there, but I don't think I fit in well there (or did well there) because there seemed to be very little cross-over, academically. When I approached my advisor about taking a mid-level Philosophy class she balked. The course would be too competitive, she said, I would not have the background. Granted she did have a point -- I didn't have the background a lot of other students had in Philosophy, but why did that matter I asked her? Well, you wont do on the exams she said. I'm not interested in taking the class just to get a grade and pass go! I'm taking the class to learn, to expand my knowledge!

Luckily, not all Universities are so myopic. Take Columbia for instance, where they have a core curriculum -- irregardless of whether you are a Physics or Philosophy major, you will take a core series of proven, interesting, non-fluffly courses, across the disciplines. One of my best friends from highschool, David attended and spent his time not only on studying harmonic functions, but reading Plato and Homer. And I know for a fact that this has helped in his career: he obtained his Doctorate in Physics from this school across the pond - Cambridge. Oh and got to sit down with Dr. Hawkin's a few times. He is currently working on cutting edge string theory with one of it's "creators", Brian Greene. (And to think, I was better at math than him back in junior high...) I believe that without studying art, history, literature, in addition to mathematics and science, his research would not have been as successful. (He has told me so much over the years.)

This is why I am a major advocate of inter-disciplinary study; I just wish I had more of it in undergrad...

I just picked up a copy of Dr. Ken Robinson's book, The Element: How Finding Your Passion Changes Everything" -- I've watched a number of his TED talks and I highly recommend them as I think he has a lot to say on how we are educating (or in this case, mis-educating) our kids. Changing Education Paradigms has been wonderfully illustrated by RSA and I think is quite brilliant.

Why We Do What We Do

Behavioral economics under attack

Here is an interesting article. I have not yet finished reading Nudge (though enjoying it) and Predictably Irrational is next on my to-read list, so it’s difficult for me to refute or support this piece.

http://www.slate.com/id/2283063

I will say that I don’t think it was Thaler & Sunstein’s intent to offer anything more than evidence (at this stage) that the classical economics models aren’t adequate in determining how people actually make decisions. (Have we forgotten LTCM?) I would however be interested in hearing their response to this challenge. I would also add that Thaler & Sunstein do point to “the why”: our Automatic and Reflective Systems (as discussed in the first chapter). We can describe these systems quite well, in fact – they do in the text – and there is considerable evidence to believe they are accurate models of how we think

http://www.scientificamerican.com/article.cfm?id=of-two-minds-when-making

Curious if Pinker would agree, but I digress.

Gigerenzer and Berge are not wrong in their complaint; we do not have a complete description of our decision making system – there is no equation we can write down that says this is exactly how we behave. I guess the question is, do we need it to gain insights? (I would say no.) Is it sufficient to design experiments to test these black boxes that are our Automatic and Reflective Systems? (I would say I don’t know.)

Anyway, Gigerenzer seems to have something interesting to say: “Gut Feelings” looks like an interesting read and will end up on the wishlist:

The key concept—rules of thumb serve us as effectively as complex analytic processes , if not more so—is simple to grasp. Gigerenzer draws on his own research as well as that of other psychologists to show how even experts rely on intuition to shape their judgment, going so far as to ignore available data in order to make snap decisions. Sometimes, the solution to a complex problem can be boiled down to one easily recognized factor, he says, and the author uses case studies to show that the Take the Best approach often works.

I’m going to have to add Basic Instincts to my wishlist, too, since it was mentioned in the article along with Nudge and Predictably Irrational, besides also looking interesting. (I am humored that this article actually “nudged” me to add another book on the exact topic it’s trying to refute!) If any of you would like to borrow Nudge (or anything else on this or related topics, for that matter), let me know.

Cheers,

Ken

Science, Art, and Engineering

Thus far this has not been an exactly optimistic comment on truth. I actually do have a much higher opinion of human nature and believe that the vast majority of us are, in fact, in favor of the truth. Why then have I been so negative? I fundamentally believe that there is truth to be found. There are so many examples of things that were not don’t properly, they are unfortunately never really reported and heralded as something that was done right. We really only primarily here about the things that have gone wrong. It's typically said that that one ah shit erases a dozen or so atta boys. And this is true. Here is an example.

How many suspension bridges does the world have in place? These numerous spans are both beautiful and are the epitome of engineering marvels, but they are more than engineering. Most of us can name the more significant ones in our country, Brooklyn, Golden Gate, Verizono, but very quickly the list gets shorter. There are more, many more world wide. But if you are an engineer you also know the name of the Tacoma Narrows bridge. Has that mistake ever been repeated? But we still hear about it today and it is used to describe an engineering mistake. At the time, however, I think it was probably more of a scientific mistake.

I use the suspension bridge as an example since more than anything, a project of this immense scale epitomizes the combining of three areas, namely science, art, and engineering. All three were necessary in order for the bridges, as we know them, to be standing therefore giving us functional examples of the success of truth, as we know it. When man first undertook construction of these grandiose extremes, the Brooklyn bridge, for instance. There were many things that were not understood. Steel and its properties had to be examined to get the cabling right. Significant additional work was necessary in order to understand how to get the foundation for that structure correct, let alone how to construct it. Yet throughout, there was a need to construct something aesthetically appealing. All three disciplines played together to get it right - John A. Roebling got all three right despite the efforts of many to throw him off.

What I would like to present is an understanding that when seeking the truth in our profession as well, a commitment to all three is vital in the search for the truth. True analysis, as is the case with a bridge, is a convergence of all three disciplines. Most often analysis is treated as an engineering problem, something to be solved by formula and repeated over and over. Formulaic analysis ignores 2/3’s of the true potential of analysis. It is not the analyst fault necessarily. This too was learned in school, which is most typically aligned with an engineering discipline and certainly not aligned with the school of science or art. This is not to suggest that a case could be made for creating an analysis curriculum in the other places, it is definitely in the right school. It is an understanding that analysis and engineering should not be done in isolation.

The Primitive Pete

I don’t have the greatest recollection of my days in middle school but there is one thing that stands out in my mind, Industrial Art’s class. I don’t remember my instructors but I remember it was my first experience bending metal. We fabricated ash trays which we proudly brought home and presented to our parents. Aside from the interesting social commentary it was the first time I had actually produced a useful product with my hands. My dad, however was not too interested in my profession in the ash tray fabrication business. I remember him telling me on numerous occasions to get into a business where I could use my head... where I could think and get paid at the same time.

There is one more thing I remember.  I was introduced to a character who has lived with me ever since. His name was Primitive Pete. Primitive Pete had a staring role in an educational film produced in collaboration between General Motors and Walt Disney Productions known as the ABC’s of Hand Tools. A quick search on the Internet and you can view some of what remains of the Primitive Pete material. Pete was an interesting character; he of course was a caveman intent on using all the available hand tools he could find, in the wrong way. Using a screwdriver as a crowbar for instance and always, always using a hammer for just about every application. Now my memory of Pete is most likely flawed and having found the material on the Internet I intentionally did not go back and change my fond perception of him after 30 years – so please forgive me if there are some inaccuracies as I report him to you.

The important part of this topic is that there is a right way and a wrong way to use a tool. And there are certainly many tools available to the professional analyst. Tools, models, spreadsheets, simulations, programs, algorithms, heuristics, hierarchies, etc. all must be used correctly in order to have a remote chance of doing the job correctly. Unlike Primitive Pete however, if you use the wrong hand tool you will do a lousy job and even worse someone could loose an eye. For us, the use of a wrong tool will probably go unnoticed by most, and in the main no one will get hurt or killed in the process of doing the analysis. The end result will be the same, however, a bad job.

The bad job will manifest itself at some later time following the equally bad decision that could be made as a consequence of the influence. It’s certainly not inconceivable that someone could get hurt or die as a result of a bad decision – although this essay should not be so bleak as too suggest that product safety or sound engineering principles in the fabrication of a product wouldn’t be assumed – the other guys always to their job correctly. So we are professionals in are trade at the office, not shade tree mechanics or Norm Abram’s carpenter want-a-bees as we might be at home. Professional means we know what we are doing and we will do the job correctly with the right tools at our disposal.

So why do we fail? Typically it is because we use the tools that are in our toolbox just like Primitive Pete. If the tool is in there we will probably find a use for it. It saves us a trip to the hardware store. Also, our tools tend to be very expensive software applications – and it is difficult to retool the factory without a major investment. Another reason is training – we know how to use the tool that’s in the box – if there is another one, we may not know about it or may not know how to use it. In addition, even if the tool in the box is the correct tool for the application, very frequently we don’t use it correctly or we don’t bother to sharpen it, to draw yet another analogy with hand tools.

I’m fairly certain that an unsharpened tool is about the most dangerous one of them all. That’s the wood chisel that when sharpened will carve wood like soft clay but when dull you will invariably drive deep into the palm of your hand. So here are a few examples from our profession. Using a linear program to optimize a non-linear behavior. Using old data or data that has been prepared by another group that is not fully reviewed and understood in the context of your analysis. Adding too much complexity to a spreadsheet thereby loosing either the cause/effect traceability or fixating on a portion of the problem that is not the main concern. Or conversely, over simplifying a problem that requires more detail.

The list goes on and some of the right ways to approach problems will be addressed in further installments of this blog...please stay tuned.