The Right Picture is Worth a Thousand Words

Why Visualization?

We often hear a picture is worth a thousand words.  There is a simple reason this is true.  Our eyes evolved to be the best way to put information into our brains (speed and amount).  The amount of information that can be absorbed through our eyes is staggering and only limited by our ability to train ourselves to use our eyes to collect information.

Conversely, if we want to convey information to other people, we should also strive to do so visually.  Hence we must train ourselves to think visually so that we can then figure out how to transmit information visually to our audience.

Here is a short list of rules to help transmit information and concepts visually.  Special thanks to Glenn V and Brian R for assisting me with this list yesterday.

1) A visualization is not simply a picture.  A visualization must be information or a concept conveyed through sight.

2) Pictures that convey no meaning detract from the information being presented and should be eliminated.

3) The most casual observer should understand what's being presented even if they don't understand what's being presented.

4 ) Any process can be visualized...think flow chart, they are simply and easy to draw.

5) Data visualization starts with a simple graph, rarely, if ever, is it a table of data.

6) Data keys are key...never forget to use them.  However, keep simplifying your key until most of the information is embedded in the graphic.  Then you might have the option to eliminate the key altogether.

7) Never try to convey more information then the human brain has the capacity to remember... if it's important to be remembered.

     If the concepts are new they should be in the range of  3  (1st, 2nd, and 3rd law of thermodynamics)
     If the concepts are simple or familiar maybe it's  5 +/-2  (Directorates in your organization)
     If the concepts are extremely familiar or can be grouped, you might get to 7 +/-2 (Football teams)


I run the risk of violating rule number 7 if I go any further.  And it might already be too late...

IBM Has Many Eyes

Glenn gave me permission to post his email along with many of the comments he received.  Glenn wrote:

Last week I attended a series of talks sponsored by WINFORMS and IBM. The speakers were Brenda Deitrich -the leader of IBM's math department (now branded as Business Analytics) - and her lead visualization guy, Stephan Jou. It was exciting to see where IBM's math team has gone, considering that when I graduated I was impressed by the work they did but was unsure about moving there due to industry changes (cf. Bell Labs, Kodak).

But for this discussion, I'd like to think about the visualization ideas. Jou gave a tour through their web site IBM Many Eyes, as well as IBM Many Bills which focuses on depicting US legislation. Tim - what was the web site you were using with lots of visualization techniques? This looked very similar, where the public can upload their own data and try different and creative ways of visualizing their data.


Here are two of the IBM sites for a closer look:

Many Eyes

Many Bills


It got me thinking again about how Tim and Swish provided A8 with an interactive map, and how Brian developed interactive maps and sliders. These techniques are a major change in how we tell the story to decision makers and provide information to the rest of the community. We're used to building or using tools to make maps and other visualizations for ourselves, because we know how valuable they are to analysis. But each time we do, it usually takes a lot of effort, is hard to adapt on the fly (like FalconView), or is not presentable beyond the analyst (STORM w/ OpenMap).

I'm certain there is value in having a warm-base of graphics capability ready to support our analysts and AO's. But how can we get there cost-effectively? IBM sees a profitable market there - they bought the makers of SPSS not for statistics but for business analytics - data manipulation and visualization. Jou cited Leland Wilkinson's classic tome "Grammar of Graphics" about rule-based graphics language. Instead of building an Excel-like library of charts that often falls short (because "there will always be a need to tweak"), they aim to allow users to quickly build their own library with these rules.

Short of licensing IBM's new project when it's built and contracting their specialists, how can we do that? Do we need to keep reinventing the wheel (or settling for what Excel gives us) each time? Is any in-house effort doomed to follow our attempt to bypass Sharepoint with MIS? Is there any open source work that gets us most of the way there, and could we use it on the networks that matter? How much up-front and ongoing effort would be required to make a significant impact on A9, AF, DoD visualization?

I know some of you have pondered down these lines already. What do you think?

Regards,

Glenn

Analytic Foundations

While this blog on Analytic Truth is a philosophical treatment of the subject of finding truth in analysis, a complementary blog has sprung up in which we will treat in much greater depth the foundations of good analysis, namely, the correct application of engineering and mathematical principles. The following is an excerpt from the new Blog.

ANALYTIC FOUNDATIONS...

The purpose of this site is for the analysts ... and friends to keep our analytical spears sharp. We are currently focused on learning Radars.

A small group of us have decided we’ll try to establish lunchtime seminars to learn/review/discuss RADAR and eventually other core competencies. Any/all of you are welcome to join us. Per our discussions today, we plan for our next seven sessions to cover the following topics:

RADAR Range Equation

Parameter Estimation

Waveform Design

Detection Theory

Clutter, Multipath

Key RADAR Components

Electronic Attack/Electronic Protection

TB will send out a few problems or tasks related to the weekly topic NLT the Monday prior to our session, and we will all try to get together on Thursday to share our experiences/challenges/epiphanies. After session 7 we will move onto different topics, intending to work our way through some set of “core knowledge” that our fearless leader would like for us to understand/master. If you don’t manage to complete the task/problems, don’t let that dissuade you from attending on Thursday. The interaction and discussion is most important. The Lincoln Lab lecture series and RADAR tutorial websites below and the Merrill Skolnik Introduction to RADAR Systems text are our core resources, but please share any other materials/links that you find useful.

This is a follow-up to TB’s “Let’s Learn RADAR” proposal: “I propose we go through the first lesson in this course on our own. Then we meet at 1500 next Friday and discuss, break into teams and spend an hour going through relevant problems. As we go through the course, let’s ask questions and create an atmosphere of dialogue .”

TB had asked BT to come up with some questions/problems we could work through in order to help set the first lesson’s material firmly in our minds. We’ll also use this session as a kick-off to discuss how we want to do this in the future. (Many of us have the Skolnik text on our desk, so maybe we work through that with the MIT lectures and materials. Again, anyone who wants to join us is invited.

You can participate in this new activity at http://analytical-foundation.herokuapp.com

The Passion of Analysis

As we continue to struggle with the essence of what makes good analysis and how we labor with uncovering truth by merging the physical disciplines of engineering with the mental curiosity of science and the spontaneous creativity of art we build a passion for our career field. Dr. Henningsen speaks freely about passion within the field of analysis and often dons a red business suit to accent her own “passion” for the subject.

Interestingly a Marine Corps Press publication that hit the streets in 2007 called, “The Passion of Command; The Moral Imperative of Leadership”, by Colonel B. P. McCoy, delves into “Passion” from the stand point of leading soldiers into combat. Whereas the field of operations research should never be construed to approach anything near the life and death circumstances of a combat infantryman, Col McCoy’s instructions in leadership are useful tools across countless professions and other human endeavors. Analogies can be drawn to many of the necessary aspects of combat, like cleaning and firing a weapon, and those analogies are sound. Because, as with repetition and practice and knowing your M-16A2 rifle like the back of your hand, also knowing the tools of your trade, in any trade, so well that their operation becomes second nature does not stretch the analogy and is easy to understand.

To become a professional means to become not just proficient but to move beyond the apprentice and journeyman stage. Becoming a master in your craft is what must be sought. On the battlefield it means the expert and unflinching application of violence against ones adversary. In analysis it means the courage to seek that which is accurate verses that which is favored by political or personal bias. But yes, accuracy does not come easy. Illuminating accurate criteria for choices that must be made by those who are in a position to decide does not come easy. Napoleon’s glance, coup de oeil, cannot be confused with intuition when the matter is complex. Therefore the analyst must become a master a deconstructing the problem and then reconstructing the issues that surround the problem in a way that is logical, consistent, and obeys the laws of physical science.

If it is accurate to say that some soldier as better than others, than it is also accurate to believe that some doctors, some lawyers, some teachers, and some analysts are also better than others. In fact even on the battlefield, unfortunately we must live according to the 3,000 year old law of Heracles which is quoted by McCoy in his book, but is worth repeating for another 3,000 years. “Of every one hundred men in battle ten should not even be there. Eighty are nothing but targets. Nine are the real fighters. We are lucky to have them since they make the battle. Ah, but the one—one is the warrior—and he brings the others home.”

How then to we strive to be the one? Or, at least one of the ten? That, is the essence of what, this blog is all about. This blog may not be perfect…but it is dedicated to those of us who want to do better. But it takes more than dedication, more than education, more than belief. It takes passion. Are you passionate about analysis?

The Whistle Blower

Can we know the truth if we do are jobs competently? Sometimes...but also one big question remains...can we handle the truth? And the truth shall set us free...right? If that were true we wouldn't have to deal with the term, "Whistle Blower". The truth might hurt, it might even cost us our jobs. But we would be free. John Boyd didn't ask for a salary in his post retirement years so he could remain completely independent of bias. Most of us don't have that luxury. But by seeking the truth, we could be labeled a profession of whistle blowers. I've done very little analysis where during the process someone didn't have a little secret or fact they would rather not have surfaced. And that's the dirtiest little secret of all.

Whether we like it our not, we are the ones who have to tell our bosses the bad news. Now, we are not saying that our work cannot produce answers that are not bad news. Indeed our work can produce good news – but that news of marginal value. Those decisions are easy to make. Everyone is on board; this idea is self evident, everyone will make money and every body wins. The analyst is hardly noticed at all. We have referred to this person as the marketer. If you know the truth and sell out, you are a charlatan or worse. If you say nothing, you are a coward. It is the bad news to which everyone takes notice. And there is only one person in the room speaking of the bad news – the analyst.

We can chose to do so quietly or we can chose to tell the media. I would suggest telling your boss first. If your boss shoots the messenger then perhaps you were in the wrong place and you should go tell the media. If you boss hugs you and tells you he will implement all of your recommendations immediately, please call me, I would like to send you my resume. If, what’s more likely to happen occurs, your boss tells you he appreciates your work, thanks you, and sends you back to your office feeling elated and some real job satisfaction that is the best the analyst can hope for.

Unfortunately, in a few days, when you realize that your recommendations have not been implemented, you might not feel as elated and might again feel the gates of AP closing in. What went wrong? You must have been the smartest person in the room and you were bold, man you were bold in there, and at times gusting to arrogant, what happened. Have you been labeled a pariah, a whistle blower? Should you start looking for a new job? Was your analysis debunked? Are their critics out there you don’t know about? Did you do something wrong in your analysis? Are you becoming paranoid and perhaps losing your mind? Do you see the PM walking behind you down the street? What happened? Chances are, you did your job right, exactly right. If you are not upsetting someone you are probably not doing your job. Unfortunately, what happened is leadership. Good or bad, we as analysts don’t make the decisions, they do – and it’s time to fully consider that truth as well.

Art

A Child's Artwork

Some children are happiest when they have a marker in their hand and a large blank sheet of paper on which to explore whatever is in their imagination. Why would we as analysts wish to abandon the creativity of our children? We should have the ability to finger paint, if we so desire. We should have the ability to use play dough to sculpt a fictional creature, or have the entire ceiling of the Sistine Chapel to paint our masterpiece. In addition to neglecting the science, the artistic side of analysis is often forgotten.

We have heard in our office many times that analysis is part science and part art. The science part is most often mistaken for the engineering. But the art side is also not what they are thinking and is a catchall phrase used to explain soft things that are not well understood. Art as it is commonly expressed is more like black art. Something mystical that can’t be defined that emerges as a result of a “not quite” understanding of the black boxes containing algorithms or procedures to which a problem was committed in order to obtain a calculated solution. Presto a result appears.

We are not talking about this black art, mysticism, or even the fog of war as expressed in analysis. We are talking about true artistic freedom of expression to explore a blank sheet of paper. But unlike science that is methodically seeking answers, the artist is trying to generate questions. And of course, these essays do not take the place of “Finger Painting 101” and are not intended to get you to think like a Thomas Edison. We can, however, establish a few rules to follow which you should consider adding to you tool box.

First dispense with any boundaries. This is the blank sheet of paper and should cut across parochial boundaries and fiefdoms you know already exist. Take their budget; take their programs, what would you do with a million dollars. There are many ways to skin this cat, you can call it brainstorming if you want, but there are rules to brainstorming. Use the ones that work for you and get it done.

Inspiration comes from many places. But unless you seek it, you will never find it. Find your source and cultivate it. Just like putting on your lab coat for your science experiment, you should find inspirational moments to be highly satisfying. For us they have never occurred at the office. They occur bolt right up in the middle of the night, during long runs or soccer games. They just might be the single most satisfying moment of our profession short of proving a Zealot wrong and having him removed from his job.

Along with inspiration be creative. Change the medium in which you work. Change the people whom which you work. If you have a brain you can be creative – that’s the funny thing about our brains – it’s up to you to unlock your creative side.

Now, unlike engineering, which could be executed in larger teams, and science, which should be done with small teams, art should be reduced to the smallest or even single individual. Michelangelo worked alone. You cannot have even two people painting on the same canvas. This will be the hardest thing to do. If it is your art it will be a part of you and you will want to show it off and be as proud as your children are to hang their painting on the refrigerator. However, unfortunately, just as a parent is proud of their child’s artwork, hanging in the refrigerator may be the right place for it. Just because it was your creation and you are attached to it does not make it the Mona Lisa. Some self-control is warranted here. But you will not necessarily know until you host an exhibit. Get your artwork out there. This could be dangerous to your pride – and perhaps why many of us lose the creative child in us. If our drawing of a lion, which our grandmother mistook for a raccoon, embarrassed us perhaps we stopped drawing. If we were embarrassed by our voice, perhaps we stopped singing. Nothing ventured nothing gained here though, so put on a thick skin and hang your ideas on the refrigerator at work. You don’t have to sign your name, but you should allow them to make comments.

But how do you really know if you have created something that can be described as art. How do you know you have created something that resembles the truth or something that will be appreciated by more than just your mom? Here to we believe we can help.

The truth will be self-evident. The Golden Gate Bridge is self-evident to most. Most great works of art indeed are self-evident to many. Why is this the case? Because during the creative and inspired times of an artist, based on all the experience and hard work they had done up until that point, a moment of clarity on the solution shines through. This is the 1% inspiration moment followed by the subsequent 99% perspiration that can be handled by the engineers who will build the project.

Another way you will know if your product has achieved the status of art will be if the senior decision maker to whom your analysis was aimed sees something in the solution that filled in a missing hole in their intuition. They might not have even recognized that there was something missing, perhaps just a feeling. But bang, if you nailed it, they will know it and let you know as well.

Free Soloing

The feature article in the May 2011, National Geographic is entitled, “Daring, Defiant, Free”. I believe these same three words characterize the essence of what is required to be on the front end of a scientific revolution. In his April 4th blog, Merf gives us a passage to read on scientific revolution taken from the book, “Chaos: Making a New Science”, by James Gleick. Gleick has applied his words to tip of the spear activities that were going on with the creation of chaos theory more than two decades ago. But his passage is timeless and applies to any worthy endeavor that must first overcome a mountain of resistance in order to succeed. It is within the words “mountain of resistance” that I see the parallels to the National Geographic article and begin this essay.

Seated in his office to my left is Alex Honnold. When he was 23 years old he climbed Half Dome in Califorina’s Yosmite Valley…without a rope. In this picture he is relaxing on what they call “Thank God Ledge” during an encore climb he executed, also without a rope. It’s important to understand that I am ghastly afraid of heights. As I stare at this picture I become physically ill. I experience vertigo while sitting in my padded  chair. I want to scream out in fear. I want to look away, turn the picture over, or otherwise remove the image from my mind. Alex is clearly a daring, defiant, and free individual. Some might also say…and here it is…that he is crazy. That he is stark raving mad, a lunatic, and belongs in a padded cell vice a comfortable chair. Sound familiar? But he has overcome gravity to climb a mountain as free and effortlessly as we ascend a flight of stairs…well maybe not as effortlessly…but certainly with no strings attached.

I cannot walk out onto a hotel porch above the fourth floor without backing myself up against the wall…even though I know the probability of me plummeting to my death is zero. These completely irrational fears must be deep seated in my psyche perhaps as an over manifestation of a survival instinct. Individuals without those instincts plummeted to their death and were removed from the gene pool of my linage. However, my direct ancestors survived, apparently by backing up against the wall when they walked out onto their cave balconies. Alex Honnold’s ancestors must have not only walked out on their balconies, but climbed higher in search of food. Those who stayed at home with their backs against the wall were the ones who perished. But it was a slower death, one via starvation rather than a fast decent to the rocks below. I am definitely in the slow starvation camp. Nevertheless some of my ancestors must have survived as well so living with your back up against the wall must have worked out in some cases. In fact, in the main, living life conservatively definitely increases ones chances of survival in most cases.

The same is true with most things we humans do. But if most of the scientific community is staying at home with their backs against the security of their cave walls, how then are we to discover the improbable? How do the scientists that start the revolutions become daring, defiant, and free individuals? I don’t think it’s magic. I think it’s through hard work.

As improbable as his feat may seem it didn’t just happen. Alex works unbelievably hard at his sport. All world class athletes do. It would be easy to dismiss Alex as a crackpot with a death wish. But that’s the furthest thing from the truth. In order for him to climb Half-dome without a rope he climbed Half-dome many times before. He has climbed the route so many times he has memorized the moves and holds necessary. In fact they are so ingrained in his brain; he can, as so many world class athletes do, visualize his movements ahead of time. He didn’t just show up at the granite wall of Half Dome and begin his ascent, free of ropes. So too is true with all great genius. The foundations for genius are set through hard work and discipline. Mandelbrot could visualize the equations in his head because his head was full of the math he needed to see a fractal in the clouds. That doesn’t mean that everyone with equations jammed in their head can be Mandelbrot…or with climbing acumen can be Honnold for that matter. It takes more…much more. And equations are not even the science. Equations form the discipline behind the science.

In rock climbing the engineering behind the sport is found in the discipline of the equipment. The devices that are used to secure climbers safely to a smooth granite surface, the knots and the rope which serpentine through high strength aluminum carabineers, and of course the tight fitting rubbery climbing shoes that when smeared cross a granite face stick like glue. And then practice...a lot of practice.  Mind numbing repetition of these same skills over and over again.  After the foundational basics become routine then more advanced studies can proceed.  The science is in the experimentation and discovery of new tools for the rock climber’s arsenal as well as the techniques of movement and conservation of movement. If you get tired lifting your 180 lbs carcass up a flight of stairs consider climbing a ladder that’s 1300 feet straight up. Years of climbing science came ahead of the current generation of modern climbers that now enable what comes next…the artistry.  Until Honnold first climbed Half Dome with all his equipment, and then free climbed Half Dome with minimal equipment, he was not ready to create his free solo masterpiece.

With the final package comes the inspiration and creativity to do something new with the foundation and legacy of what came before. Sometime that path leads to incremental changes in the status quo. Occasionally, a revolutionary leap can be made. Typically, when a revolution is at hand, there is no shortage of the old guard screaming about the impossibility of it all, telling the few that might see a better way that they are crazy, and laying before them a mountain of resistance that is the hallmark of a shallow thinker.

Now that is not to say there is no risk involved. There is huge risk involved. And the penalties for miscalculation, as Clausewitz has said, as are the miscalculations that lead to war, could very easily lead to death. Certainly a physical death for rock climbers ascending a stone face without safety gear, but they could also lead to the death of a publication, a reputation, or a career. But for those brave few, those that can be daring, defiant, and free from the gravity of the ties that bind a revolution awaits. Just make sure when you begin your free solo of your granite mountain you have nerves of steel and have prepared as well as Alex Honnold.