The Gauntlet

There is a great scene from the Clint Eastwood movie, “The Gauntlet”. The Clint Eastwood character is asked, “Why do you think they gave you this job?” and Eastwood responds, “Because I get the job done”. His boss retorts, “No, it’s because you don’t get the job done, you were never supposed to get this job done”. And that, in a nutshell is what we call the Gauntlet syndrome.

When an analyst is brought in on a hopeless task for the sake of either showing progress, or for documenting that everything conceivable has been done and we are still failing. This is the nefarious twist. The same result occurs when the analyst is brought in on a project with a known outcome. You are there to prove that the right solution has already been discovered. Woe be to the analyst that thinks they can make a difference or perhaps worse yet, finds an actual solution to the problem. What’s left is nothing short of having to run a bureaucratic gauntlet that will leave you bloody and near dying if you have the guts to run through.

Most don’t have the guts to run through it, or having started will back down when their health begins to fail. Let’s say on some odd occasion the analyst makes it through. Perhaps, they might exclaim, “I stuck to it and I won, and the system works”. This is the height of nonsense. The system did not work if it takes months or years for the truth the emerge and for people to put everything on the line in it’s pursuit. Also, should you win, the ramifications could be the stigma of someone who was noble in their efforts but could be labeled as not a team player or someone who was a zealot in their own right.

We have seen this many times before. The best analyst is the one who can maintain a bit of independence from a problem. The best advocate is someone with a true interest in correcting the problem. If you are close to the issue you might be willing to run the gauntlet. If you are removed you will probably be unwilling to get bloody over the issue even if you know you have the truth in the palm of your hand.

So by definition the system does not work. And we, all of us, are perhaps no greater than the Eastwood character. We were brought in believing we can get the job done. But no one actually believed that we could, or we were really just supposed to go through the actions to support an a priori answer, or worse a decision that had already been made. This just makes us process bureaucrats and you don’t need a whole lot of education in our disciplines to be one of those.

So you have to decide each time – as you peer down through the gauntlet. Do you have enough protection to make it through? Is this one worth getting bloody over? This will probably be the most frustrating dilemma you will have to face. We can only offer that at times like these, if you haven’t done your homework, if you haven’t done the analysis to the best of your ability, if you haven’t followed the prescription for the truth that we hope we have captured as you move forward through these pages, this might not be the time to run down the rabbit hole. But if you have, if you are in possession of the facts, and have the answers to the majority of the questions that will come from both sides of the gauntlet, can there be anything short of life and death combat that could bring heighten your sense of awareness and make you feel alive. Don’t let it consume you; don’t let the sleepless nights ruin your health. But do occasionally run the gauntlet. That is what you are getting paid to do.

Rest in Peace Benoît Mandelbrot

Benoît Mandelbrot has passed (20 November 1924 – 14 October 2010). Let us pause to honor him briefly on our blog and reflect upon his work. 

Click here to link to his Wikipedia Bio

Click here for his obit in the NY Times

The video and song below contains strong language so it's not for the kids...but it honors him in certain ways.   It's more fun than anything else.

The Charlatan and Other Weasels

The professional analyst better known as the cost analyst, the systems analyst, the operations research analyst, grew it’s own profession during World War II and directly thereafter. Some would credit the work done to locate German U-Boats as the first attributable activities of our formal profession. Whereas there is plenty of documentation to suggest the mathematical formulations that were developed during this timeframe constitute what we now refer to as operations research, analysts have been around much longer than 1940.

Frederick Taylor published his work; "The Principles of Scientific" Management in 1911 giving him an auspicious start in the field but one, which is typically credited with the Modern MBA, vise operations research. There is no doubt that his time and motion studies constitute, for the analyst, a very well documented way to collect the first data.

But way before Taylor arrived on the seen anyone in business strived for efficiency although they might not have been certain how to achieve it. They learned by making trades and by making mistakes. Almost 100 years previous, however, Eli Whitney labored to produce machinery and interchangeability to improve efficiency in his factories. Whitney, however, has both supporters and critics. The individual typically credited with the invention of interchangeable parts is Samuel Colt who took before Congress 15 guns. After disassembling them, he mixed up the parts and reassembled them. The approving Congress immediately placed orders for 10,000 of the weapon never realizing that the demonstration was a shame and Colt had marked all the parts so he could reassemble them correctly.

Regardless of what took place on that day, Colt, enlisted the help of Eli Whitney and together they eventually got it right and the first assembly line took root – this well before Henry Ford apparently invented one. This paints a darker side of analysis, a side that allows the initiated to obscure the truth with promises of greater efficiency and cheaper products. Perhaps they will, perhaps they will not, but many well-documented cases of fraud occurred throughout the 1800s culminating in Congress passing a False Claims act in 1863.

The Charlatan is the guy who convinces his boss that by adding sawdust to the gunpowder they were selling the Government that the company could save money. Perhaps it wasn’t the analyst who told his boss to do this, but if the analyst was telling his boss not to do this he never blew the whistle. This creates a problem for our profession, and therefore becomes the heart and soul of our ethical dilemma. This is a typical problem, which is called the whistle blower scenario. Blowing the whistle is a no win scenario and cheating inside the no-win scenario makes matters worst. Even with protections in place blowing the whistle must be considered carefully with both facts and figures.

As with profit, the primary motivator of malfeasance in the commercial sector, crooks may actually desire the "wrong" answer. Let's hope you are not the one with the time managment study showing how to manipulate the machine to make an extra buck. Let's also hope you an not employed by a crook.

The Knee Jerk Reaction of an Anti-Fluids Bigot

The following entry is in response to a debate I have been having in a closed blogosphere at work. It is based on a comment I posted to this article.

http://www.popsci.com/cars/article/2010-09/giving-traffic-lights-mind-their-own-can-reduce-congestion-study-says

I will not replicate the blog comments here…you can pick up the gist of the commentary from my narrative below.

First just to be clear I entered into this debate with Bryan on my own free will although he started the argument by baiting me with his tactics. I tend to be an arrogant troll myself on occasion so I am not at all intimidated by his discussion tactics. The evidence of his particular bullying is his use of the term “knee-jerk reaction” which he directed at me for my very quick, and somewhat negative, response to the article posted above by Sphinx. In that comment I stated that modeling the traffic in a city is not like a fluid so they are doing it wrong (Bryan is clearly in “ Stage 3” as given by King and Logan in the book “Tribal Leadership”) It’s not clear that he was rising in defense of Sphinx and that somehow by posting a negative comment to her blog I was attacking her personally (which is certainly not the case) or he is defending the integrity of the authors of the study upon which the article was based (which was most definitely the case) and in both cases, his and mine, we are not in full possession of the facts within the actual study (the actual study can be found here [1]). Nevertheless the debate ensued and I can either stand down or continue to fire back. Obviously I have chosen to fire back.

As an additional note, I also made in my original comment the statement, “If they were saying they model the traffic as a fluid as an analogy” …for laymen to get the picture…that’s Ok. But if they are using fluid dynamics in their actual calculations, as I stated above, they are using a flawed technique.

I followed up my original comment with slightly more detail on my meaning which included the necessity of scaling various traffic modeling problems from intersections with single car/driver interactions through free flowing highways. I said that in the second case fluid dynamics might find some application but they have no place in the former. Although to be even clearer, classic fluid dynamics may seem like they apply in the second case, and are in fact the source of many studies over the past 50 years, in the main they are no longer used by the industry. Occasionally you will see a fluid dynamics PhD student write a paper on some new application of fluid dynamics to traffic flow. However the problem as defined in the original statement, the requirement is to take a systems wide view of a network of urban intersection in order to synchronize the signal lights…but synchronization is the system response…the idea is for each light to work autonomously while considering the bigger picture. This will not work if modeling traffic as a fluid (the algorithm to drive the timing of the lights needs traffic input and output, it will work as a surrogate for traffic but my argument is that it is not a good surrogate and most likely means the algorithm to drive the timing of the lights is also suspect) That of course is my opinion but if you check the core disciplines aligned under fluid dynamics you will not find traffic…because why? It is not a fluid.

Traffic flow on a road is nothing like fluid flow, either compressible or non-compressible fluid flow in a pipe. As you approach free flowing traffic on a highway with smooth curving bends and flow that is uninterrupted it is easy to believe that fluid equations might apply and simplify down to a basic case. This has been the silly mistake of many individuals who have tried. Yes with a lot of trouble you can show flow with the equations, but then you have modeled a system that is not grounded in reality and most certainly does not represent real traffic. One might read [2] to see how all the math might predict the motion of cars modeled as a compressible fluid in a closed circuit for instance, as a soliton (wave) …but it’s not reality and it’s far too complex to be practical.

Starting from first principles all of fluid dynamics arises from the Ideal Gas Law which states:

pV = nRT

Pressure x Volume = Molecular Mass x Gas Constant x Temperature

Pressure doesn’t apply to traffic. Volume doesn’t apply to traffic. Molecular Mass doesn’t apply. The Gas Constant doesn’t apply. Temperature doesn’t apply.

This is where Bryan tries to convert cars to molecules, traffic speed to temperature, and traffic density to pressure x volume. Go for it. Obviously I can’t deny that folks have done just that…just as folks have tried to drive a nail with a set of pliers as well. I contend there are better and much more accurate methods.

But that’s just the start with fluids…How about viscosity? How about friction? How about the length of the pipe and the inlet/outlet pressures? The list goes on. You can find all the equations for pipe flow “Q” here in this reference below [3].  But here is a look at it.

Now if you throw out everything but traffic density and speed you are reduced to this equation:

Q = D x V

And who would argue that traffic volume/flow in not a function of the number of cars on the highway and their speed…which is traffic but it doesn’t behave like a fluid in any way other than it moves from point A to point B, sometimes. Sometimes it turns, sometimes it crashes. But one “fluid like” equation, is far too simplistic to model or predict the behavior of cars on a road, highway, or at an intersection. A great deal of work has gone into describing traffic mathematically in its many forms. All of that work is better than fluids but none of it better than discrete event simulation which is the standard for the industry and what has been used in thousands of transportation and traffic studies worldwide. A survey of the tools available is given here [4].

But it doesn’t end there. As we use discrete event simulation tools to model the interactions of the individual cars in a network in order to effectively model what is happening more and more autonomy has to be given to the individual elements. Not only do the cars have to behave independently, speed, direction, etc., they have to think for themselves interact with what happening around them. They have to receive feedback as well (try that with a fluid). A better way to do it is by treating the traffic network as a complex adaptive system with the use of agent based modeling to simulate the traffic in the network. Here is a traffic application and references [5]. That is the full sum of what I was trying to say with my comment.

Before I started my simulation company in 1994, I first described agent based modeling in discrete event simulation for traffic flow research I was conducting for Dr. E.B White at George Mason University during my masters program. I have an obvious bias against fluids which I studied in several courses during my undergraduate work in Mechanical Engineering at Texas A&M University. My traffic research was entitled the Strategic Transportation Operational Planning Simulator (STOPS) [6]. Alas the computing power for complex adaptive simulations and agent based modeling was not sufficient in 1992. It is now. But now the field is abounding with these techniques. Couple these techniques with the power of cloud computing and the power of netted information coming from an urban traffic grid and the solution to decentralization of traffic control systems is upon us. Again, all I was saying in my original comment was that fluids are bad and complex adaptive systems are good.

As I have now read the original study it is obvious that the cars are indeed treated as discrete events and were modeled using a simulation called PTV Vissim [7]. My original thought that the use of the word “Fluid” was simply an analogy is true. However it would be interesting to treat this system with a complex adaptive simulation as I have suggested. If my reaction was “knee-jerk” I apologize to those I offended. I am an “Anti-Fluid Bigot” when it comes to traffic modeling.

For those of you who do not like to start with first principles and do the thinking for yourselves, a criticism of using fluids to model traffic, that is not my own, they can be found under Daganzo’s Requiem [8]. Coincidently I observed in several studies that Helbing [9], a coauthor of the subject study, has been a vocal critic of Daganzo’s criticism of fluid dynamics for traffic modeling. This is truly odd since Bryan and I have inadvertently entered into a microcosm of this same debate. Since Helbing choose a direction other than fluid modeling in his most recent work can we assume he is now on-board? Since the word “Fluid” does not appear a single time in the recent report, I have to wonder why it was used at all in the subject article.  To venture a guess would be to suggest the use of the word fluid is an echo from a once lively debate between Daganzo and Helbing.

In the end I must now thank Bryan for forcing me to expand upon my comment so everyone has a better understanding upon which I based my “knee-jerk” reaction.

References:

[1] Lammer & Helbing, Self-Stabilizing Decentralized Signal Control of Realistic Saturated Network Traffic, 2010

[2] Saavedra & Velasco, Solitons in a macroscopic traffic model, 2010

[3] Schroder, A Tutorial of Pipe Flow Equations, 2001

[4] Jones, et al. Traffic Simulation Software Comparison Study

[5] Khalesian & Delavar, A Multi-agent Based Traffic Network Micro-simulation using Spatio-temporal GIS.

[6] Muccio, Strategic Transportation Operational Planning Simulator, 1992

[7] http://www.ptvag.com/software/transportation-planning-traffic-engineering/software-system-solutions/vissim/technical-details/

[8] Daganzo, Requiem for second-order fluid approximations of traffic flow, 1995

[9] Helbing & Johansson, On the controversy around Daganzo’s Requiem for and Aw-Rascle’s resurrection of second-order traffic flow models, 2008

The Truth About Profit

As you drive over to the client site your mind begins to consider Empires again.  My company is an empire, or they are striving to be.  I have always considered my boss to be a little Napoleon, but he has always been straight with me.  He has never asked me to do anything, “unethical” but yet I know he has been consumed lately with the organization chart and our spot in the bigger “business unit”.  Whatever that means.  His annual bonus is tied directly to his position in the “business unit” and how we finish the year in revenue compared to the rest of the business units.  Fortunately for me I am on a fixed salary.  I can be free to say the things that need to be said.  If I can’t help this client, there will be another one, and although we didn’t land this contract, I’ll still get paid.  My boss will just have to wait until next year to take his wife to the Bahamas. 

At this point I should clarify that in any empire, ah hum, organization there are two classes of clients.  Those who wish to profit whom I will refer to as the commercial sector client or CSC, and those who do not wish to profit or that unfortunate class of entities that I will refer to as the not-for-profit client, the NFP for short.  The NFP’s can include non-profit organizations but I am primarily referring to the ubiquitous and all powerful government entities that dot our landscape and blot out the sun, particularly if you live in your nations capital. 

If you happen to be a CSC the truth is as follows – there are many of you competing for a share of the marketplace.  In any particular market sector, unless you are a monopoly (but keep that dirty little secret to yourself), there are many of you who can fulfill the same need in your sector.  What distinguishes you?  Is it fresher ingredients?  Is it an altruistic need to donate 5 cents of the purchase price of your product to a favorite charity?  Is it better marketing and packaging?  Or is it, hopefully, an actual product that is superior perhaps in both quality and service?  These would be rare, but if they could be found wouldn’t they dominate the marketplace?  Yes, in fact they would, or should, if it wasn’t for all the other charlatans running around claiming the same superior quality and service.  How can they do that?  Clearly someone has to be better.  Maybe we should hire a professional analyst to make that determination – or maybe we should just leave well enough alone.  To the CSC the truth might seem like the right and reasonable thing to discover – but it could also be very dangerous.  So for the analyst, employment to seek the truth with the CSC will have to be reserved for perhaps improving the efficiency of internal methods and procedures not so much in marketing a product – that’s where the real money is – but unfortunately, that money goes to the analysts that can help them sell a product.  And those analysts work on Madison Avenue.  We refer to them as marketing and advertising firms or MAFs.  And we all know they always tell the truth.  

Don’t worry we still have the NFP client so you are not out of a job yet – at least one in which you can tell the truth.  Since the NFPs are not for profit they must have room for the truth.  Not so fast.  Profit might be a measure of success for the CSC but it can also be a motivating force in the running of  a NFP, maybe not so much in the way of monetary gain, but typically in the form of prestige and power.  NFPs and those who run them, love to make the right decisions.  Right decisions are good and make you look good and feel good.  Wrong decisions are bad and help you loose your job or get voted out of office.  The problem with NFP is that they typically are heavily bureaucratic organizations that have been built up over many years on a combination of perhaps good decisions but also many bad decisions.  When bad decisions have been made they tend to hang around for a while.  Dirty laundry is everywhere and hey it smells like a locker room in here.  But I digress.  The trouble is, no matter how many good decisions are currently being made, if an analyst comes in and starts asking questions, sooner or later the bad decisions will surface.  Maybe something can be done, maybe something can’t be done, but the dirty secrets make everyone feel uncomfortable.

So even if there is a clear problem with several courses of action, the best course of action – or the course of action determined to be the best – might require airing out the decision that is causing the smell.  A recommendation of this nature, while satisfying to the analyst as the truth, might not be the path the NFP client wants to take.  Sometimes it’s simple if the particular client was not the client when the bad decision was made.  But many times, it was the client themselves who made the bad decision in the first place.  At these moments for the analyst, thinking about opening up that bar down in the Bahama’s doesn’t seem like such a wild idea after all.  Yes, organizations can profit from the truth, and they should.  However, the third principle for seeking the truth is being aware that the truth might not be sought.