If you’ve been watching the media in the last couple of weeks, you’ve probably seen the spat that has developed between John Broder of the New York Times and Elon Musk of Tesla Motors. Broder took a Tesla Model S sedan on a test drive from New Jersey to Connecticut to test out the theory that the new supercharger stations that have been installed along the way would help electric cars to take long road trips without fear of running out of electricity. Along the way, he ran into some difficulty and ultimately needed to have the car towed to a charging station. After the story came out, Elon Musk immediately defended his product with a promise of data to support that assertion. A couple of days later, he put up a long post on the Tesla blog with lots of charts, claiming that the Model S had lots of data to support longer driving distances, failure to fully charge at supercharger stations, and even that Broder was driving in circles in a parking lot. After this post, Broder responded with another post of his own clarifying the rebuttal made by Musk and reaffirming how the test was carried out. It’s certainly made for some interesting press releases and blog posts. There has also been a greater discussion about how we present facts and dat in a case to support our argument or prove the other party is wrong.
Data Doesn’t Lie
If nothing else, Elon Musk did the right thing by attaching all manner of charts and graphs to his blog post. He provided data (albeit collated and indexed) from the vehicle that gave a more precise picture of what went on than the recollection of a reporter that admittedly didn’t remember what he did or didn’t do during portions of the test drive. Data never lies. It’s a collection of facts and information that tells a single story. If x equals 7, there’s no other thing that x could be. However, the failing in data usually doesn’t come from the data itself. It comes from interpretation.
Data Doesn’t Lie. People Do.
The problem with the Elon Musk post is that he used the data to support his assertion that Broder did things like taking a long detour through Manhattan and driving in circles for half a mile in a parking lot in an attempt to force the car to completely discharge its battery. This is the part where the narrative starts to break down and where most critics are starting their analysis. Musk was right to include the data. However, the analysis he offers is a bit wild. Does rapid acceleration and deceleration over a short span of distance mean Broder was driving in circles attempting to drain the car? Or was he lost in the dark, trying to find the charging station in the middle of the night like he claims in his rebuttal? The data can only tell us what the car did. It can’t explain the intentions of someone that wasn’t being monitored by sensors.
Let The Data Do The Talking
How does this situation apply to us in the networking/virtualization/IT world? We find ourselves adrift in a sea of data. We have protocols providing us status information and feeding us statistics around the clock. We have systems that will correlate that data and provide a big picture. We have system to aggregate the correlated data and sort it into action items and critical alert levels. With all this data, it’s very easy for us to make assumptions about what we see. The human brain wants to make patterns out of what we see in front of us. The problem comes when the conclusion we reach is incorrect. We may have a preconceived notion of what we want the data to say. Sometimes its confirmation bias. Other times its reporting bias. We come to incorrect conclusions because we keep trying to make the data tell our story instead of listening to what the data tells us. Elon Musk wanted the data to tell him (and us) that his car worked just fine and that the driver must have had some ulterior motive. John Broder used the same data to support that while his recollection of some finer details wasn’t accurate in the original article, he harbored no malice during his test. The data didn’t lie in either case. We just have to decide who’s story is more accurate.
Tom’s Take
The smartest thing that you can do when providing network data or server statistics is leave your opinion out of it. I make it a habit to give all the data I can to the person requesting it before I ever open my mouth. Sure, people pay me to look at all that information and make sense of it. Yes, I’ve been biased in my conclusions before. I realize that I’m nowhere near neutral in many of my interpretations, whether it be defending the actions of myself or my team or using the data to support the correctness of a customer’s assumptions. The key to preventing a back-and-forth argument is to simply let the data do all the talking for you. If the data never lies, it can’t possibly lose the argument. Let the data help you. Don’t make the data do your dirty work for you.
The Networking Nerd 
Tom,
True. Data never lies.
The problem stems from the way the “test” was set up. There should of been a Tesla employee accompanying Mr. Broder. Would you allow someone with little or no experience in your network lab to independently conduct a test? Is it possible the data from their (inexperienced tester) results would not be the same as yours?
Mr Musk should of considered this, prior.
Don’t you find it disturbing that manufactorer had access to all your vehicle’s information and was willing to publish it? Okay this was probably sample one used for testing, but will model you buy as an end user feature the same and who will have access to all that data???
Great post Tom. I would however see the issue of your own interpretation differently. No one is paid to give just the facts. Our ability to give meaning to facts and data is how we add value to what protocols and sensors etc do and that’s what the suits pay us and respect us for (if they just wanted the data, then they don’t need a human being to tell them what can be more efficiently done by a slew of sensors and dashboards and dials). How effective we are ultimately lies in how accurate our subjective interpretation of the data turns out. And it goes back to being honest with ourselves – this means
(a) stating clearly what our theory is (the point we think we are trying to prove – e.g. gas ran out fast because Brodie drove in circles and off course)
(b) pointing out the data points that confirm our theory together with any assumptions made.
(c) pointing out any data points that do not support our theory.
Building credibility then would mean offering up alternative theories (analysis) of what those non-conforming data points mean.
Shalom