Friday, September 21, 2012
IBM Has Many Eyes
Glenn gave me permission to post his email along with many of the comments he received. Glenn wrote:
Last week I attended a series of talks sponsored by WINFORMS and IBM. The speakers were Brenda Deitrich -the leader of IBM's math department (now branded as Business Analytics) - and her lead visualization guy, Stephan Jou. It was exciting to see where IBM's math team has gone, considering that when I graduated I was impressed by the work they did but was unsure about moving there due to industry changes (cf. Bell Labs, Kodak).
But for this discussion, I'd like to think about the visualization ideas. Jou gave a tour through their web site IBM Many Eyes, as well as IBM Many Bills which focuses on depicting US legislation. Tim - what was the web site you were using with lots of visualization techniques? This looked very similar, where the public can upload their own data and try different and creative ways of visualizing their data.
Here are two of the IBM sites for a closer look:
Many Eyes
Many Bills
It got me thinking again about how Tim and Swish provided A8 with an interactive map, and how Brian developed interactive maps and sliders. These techniques are a major change in how we tell the story to decision makers and provide information to the rest of the community. We're used to building or using tools to make maps and other visualizations for ourselves, because we know how valuable they are to analysis. But each time we do, it usually takes a lot of effort, is hard to adapt on the fly (like FalconView), or is not presentable beyond the analyst (STORM w/ OpenMap).
I'm certain there is value in having a warm-base of graphics capability ready to support our analysts and AO's. But how can we get there cost-effectively? IBM sees a profitable market there - they bought the makers of SPSS not for statistics but for business analytics - data manipulation and visualization. Jou cited Leland Wilkinson's classic tome "Grammar of Graphics" about rule-based graphics language. Instead of building an Excel-like library of charts that often falls short (because "there will always be a need to tweak"), they aim to allow users to quickly build their own library with these rules.
Short of licensing IBM's new project when it's built and contracting their specialists, how can we do that? Do we need to keep reinventing the wheel (or settling for what Excel gives us) each time? Is any in-house effort doomed to follow our attempt to bypass Sharepoint with MIS? Is there any open source work that gets us most of the way there, and could we use it on the networks that matter? How much up-front and ongoing effort would be required to make a significant impact on A9, AF, DoD visualization?
I know some of you have pondered down these lines already. What do you think?
Regards,
Glenn
Subscribe to:
Post Comments (Atom)
@ Glenn,
ReplyDeleteSadly your question is akin to saying to Michelangelo, "Hey Mike, I really like what you've done there in the Chapel...I'd like to repeat something similar over here in the lobby of my hotel but I want the story to be based on a different book, one that hasn't been written yet. And I don't want to use the same paints, I want to use some of these new resins that have been coming out of France, and I need this thing done in a year...can't wait 4 years to have it complete...I'm running a business after all...and by the way, can you recommend someone to do this...I know your busy and I can't afford your rates?"
I remember when Dr. Ted Warnock showed up at AFSAA. He was so saddened by the lack of visualization software he went out and bought 20 licenses for a software call Axiom...I think he spent $20 grand...it was 1996. It was the best graph making software of its time. I think 18 copies remained shrink wrapped and on a shelf for 10 years.
The tools are one thing... but we have to build a culture of analysts who can think visually, tell stories visually, and present their ideas in a way that can captivate and then in turn grow the ideas and concepts we are selling. And alas, everyone will not be like Mike (the Italian Mike).
So what we've been doing, and should continue to do, is the share the best ideas we come across with each other. I've read your email...I get it...but I don't get it. Where are the links to view some of this work to spark my interest and my ideas. We should also continue to send our folks to the classes where they learn this stuff, and like you, are bringing it home. All the applications will be different...so we have to search for the right place, and the right time, to use it, but then we have to have it at the ready. PowerPoint and Excel are flexible, ubiquitous, and easy...they win...and we still have a long way to go to squeeze better stories and better visualizations out of them...instead of the standard staff, bulleted, text based briefings. The sliders on the Extreme Endurance Study were exactly that next step, but that application will not work with everything. The fact that all the graphics were done with tools other than Excel is just as exciting. A success story...but still a Power Point success story.
v/r
Mooch
Ken H wrote:
ReplyDeleteSo piggy backing on this idea of collaboration, I will bring in my latest
data visualization book purchase next week:
A new book, "Visual Strategies," is meant for scientists and engineers, but
it will be useful for anyone who wants to make clear presentations.
http://www.nytimes.com/2012/09/04/science/visual-strategies-transforms-data-into-art-that-speaks.html
Mike P wrote:
ReplyDelete@ Mooch et al,
And let us not forget Mark Grabau's simulation/visualization of Santa's Workshop! It was the key to our purchase of Arena, which we used for several products but has since all but disappeared.
I think you and Glenn are talking about two slightly different things: the analyst and the tool (no, that's not the title of a book about me and you, respectively). I agree with everything you've said, but still lament the difficulty of execution. I default to VBA, which facilitates production of both HCAT and Extreme Endurance, because it's the path of least resistance. We seriously need to look at ways to make it easier for analysts pick up and dispose of whatever tool best suits their need at any given time. I wanted to use the same d3.js stuff that Tim used for the A8 project, but couldn't get it to work with the browser I currently have and am forbidden to download and use one that is compatible. No problem at home, but I can't even view it here. Could Ted even get approval to load Axiom on a system now? A2AD. We work in our own A2AD environment that we've created for ourselves. That's a huge problem I'd like to see addressed.
Can you bring that up at the Cyber Summit?
@ everyone
ReplyDeleteSo who has access to the analysis blog site on Milsuite that Dr. Gallagher
set up? We are engaged in a perfect conversation for any one of a number of
collaborative analytic blog sites...if there are no objectives I will post this dialogue to the MilSuite discussion...
Tim B Wrote:
ReplyDeleteHere is the biggest problem.
There are tools out there. We all have browsers on our desktops, but our computer support is too far behind to support us.
Hence my bold idea: we kill the nipr enclave. Seriously, NIPR is not secure,we should not pretend that it is. Local organizations should manage theirnetwork to meet their needs. SIPR handles sensitive data and other networks
fill in as needed.
If you do a cost benefit analysis of a cookie-cutter approach to security -- are we really going to out-innovate industry? Are we more secure through central control?
I claim no on both counts. Let a local organization have their computers and
give them 3500 funds to manage their IT -- let them use google mail if they want to -- and yes, know, that it is unsecure. Much better than a false sense of security -- available to you for the small price of 5 billion through the FYDP.
Our centralized approach to IT is like our suicide or assault prevention training, or our recent BRAC outside of leased office space to several central locations. We believe in central control central execution in the US military. Except when it really matters and we go to war.
Training built for young airman doesn't reach Phd's at A9. One of the best
suicide prevention sessions was at AFIT. Our commander decided not to show the video and instead had a PhD student talk about their personal experiences with depression. It hit home and she said things like, "we are
older, we know the system, we know how to hide it" that wouldn't make it in a cookie cutter approach.
Angela G wrote:
ReplyDelete@ Tim, in spirit, I absolutely agree with your rant. However, I'm not quite sure how we take that huge step in divesting NIPR and accepting outright that we're operating in a highly-contested domain every day. Universities
and commercial entities also try to protect their unclassified networks. Do they take a fundamentally different approach than we do? Are they any more successful or less constraining? I don't know the answers, and I think we would want to do an in-depth comparative look before taking action.
Now that I've had some time to digest this thread and think about it, I'd also like to submit a rejoinder to Mooch's e-mail from the 19th. Mooch contends that "we have to build a culture of analysts who can think visually, tell stories visually, and present their ideas in a way that can captivate and then in turn grow the ideas and concepts we are selling."
I wonder whether that is really something we need to focus on building. Maybe we just need to stay out of the way. I wish I had a link for the short video they presented at my son's middle school back-to-school night.
The underlying message was that today's students are interactive learners with many dynamic, interactive, virtual and live options for their
educational process. I see that in how my sons' classroom experiences and homework assignments differ from the classroom experience and assignments I used to have. The work is much more holistic and varied in mode/media.
I pulled in several of the A9 "youngsters" to get their thoughts. Lieutenants/ladies, how do you think we could best practice visualization in quantitative analysis and decision support here at AF/A9 and in the broaderdefense analysis community?
Brian Rose wrote:
ReplyDelete@ All,
Though I certainly have a bias and ‘dog in the fight’, I generally support developing non-traditional visualizations in Excel/PPT/VBA. Granted there are tools and languages that can produce many types of charts faster and
easier, but Excel/PPT/VBA are flexible, use the same language as many of our models, and are always available on any computer for development or presentation. They are among the only tools that you are guaranteed to have
anywhere in DoD (at least until Tim's grand decentralization concept is
realized).
I agree with Mr. Muccio that there is still a lot of visualization capability we can squeeze out of those mediums. For one, it is possible to build Tufte approved charts in Excel and PPT without any VBA coding using various small tricks. A few simple examples are shown in the attached ‘Tufte in Excel’ file.
I additionally included an example that takes this line of thinking further. In the ‘ARFORGEN Production Model’ I developed for the Army, I wanted to be able to show a 3-D scatter plot with number of Active, Guard, and Reserve
units on the three axes that I could rotate around. Excel did not have anything like that chart by default, but by using basic perspective
projection
(http://en.wikipedia.org/wiki/3D_projection)
and form slider bars, I added a 3-D scatter plot with a controllable camera in Excel without
any VBA code (see the ‘D to S 3D Chart’ tab). With the newer versions of Excel, I could have also just used a surface chart if I did not care about including the camera controls or custom formatting options. The O&S charts
in Extreme Endurance were another example of this method. They included overlapping bars, scatter data, and sparklines in the same chart without any VBA. The majority of the visualizations in IBM Many Eyes
(http://www958.ibm.com/software/data/cognos/manyeyes/page/Visualization_Options.html)
could be replicated in Excel without VBA programming.
With VBA, essentially any type of visualization is possible including interactive and dynamic content as my data visualization idol, Hans
Roslings, advocates
(http://www.ted.com/talks/hans_rosling_at_state.html).
These types of visualizations can take considerable time to build, but highly customized visualizations always require significant time regardless of the medium used to produce them. If there is a recurring need for a certain type of non-standard visualization (e.g. Many Eyes style network diagrams, block histograms, or treemaps), we could potentially write generalized VBA modules to build them.
Personally though, it is more useful for me to have various quick references to different types of visualizations for inspiration. I often use my LSS primer, Tufte references, TED materials, and old models and briefings. For example, the O&S charts in Extreme Endurance came from an idea Tim suggested that used points and bar plots together combined with sparkline and
overlapping bar methods I used before in a STORM dashboard concept.
Though it sounds incredibly cliché, I think the main challenge in general is to make sure analysts are able to think outside the box (the box being default setting Excel charts) when the situation calls for it. If analysts do not see the potential applications for non-standard visualizations in their work, no toolset, repository, or software capability is likely to have much impact.
@ Angie
ReplyDeleteConcur with Angela...we've always had a few young folks show up with bright ideas and their iPod in hand equipped with the very latest in technology and their bright ideas only to have their exuberance choked out in the cradle
upon their arrival and then become disillusioned with the whole business. Although occasionally we will have someone rise above being bound and gagged technology wise and they have been able to make the best of this backward/far behind situation...
And it can only be even more depressing to teach them how to "think visually, tell stories visually, and present their ideas in a way that can captivate and then in turn grow the ideas and concepts we are selling". And then caveat their training with, "Oh by the way, you can't use any of the latest or greatest technology...you have to use this cave wall and a rock".
Yet to me, the most profound stories, told in the simplest ways, are always best...and I would disagree that everybody who walks through the door can tell the stories...if only they had their own teraflop servers and could
display data like they can at the NASA Scientific Visualization Studio...
http://svs.gsfc.nasa.gov/vis/a000000/a003900/a003991/Min_sea_ice_1979_2012_1080.mp4
Another TED talk on visual display of data is at the following link. http://www.ted.com/talks/lang/en/david_mccandless_the_beauty_of_data_visualization.html
ReplyDeleteThe blog below the TED talk video contains additional links to online resources for data analytics/visualization.
After reading through the emails and thinking about the current state of data visualization/decision support within the Air Force, I would argue that your discussion has only taken a superficial look at the issue. The lack of software/tools available is distressing, but I think its the symptom of a larger problem. You have to ask yourself the questions: 'why aren't their more data visualization tools available?' & 'why aren't more analysts pushing for these tools?' Although I believe that policy/bureaucracy is a factor, I think the main reason is that most analysts aren't concerned with 'telling the story.' In my limited experience, I have found that analysts are often so focused on the narrow scope of the problem at hand that they forget their audience and how their analysis fits into the 'bigger picture'.
ReplyDeleteI think you touched on the issue when you were discussing the need "to build a culture of analysts who can think visually, tell stories visually, and present their ideas in a way that can captivate and then in turn grow the ideas and concepts we are selling," but I think you are still putting too much emphasis on visualization. The key is to build analyst who are capable of articulating their ideas in a context that is relevant to the decision maker. Once analyst begin to focus on their audience and the decisions they are trying to illuminate, they will be better equipped to find new and innovate ways to visualize their ideas.
The analysts that truly understand the importance of 'telling the story' are also the analysts that go out of their way to find effective methods of visualization. As such, I don't think new tools are the solution to the problem. Instead I think we need to focus on building analysts who understand the need to 'tell the story.' Once we do this, the tools and software will follow.