Absence of Evidence is not Evidence of Absence


Abraham Wald, in his youth

Image via Wikipedia

Abraham Wald, an Austrian-Hungarian born mathematician applied his statistical skills to improve the armor of aircraft returning from battle during WWII. His approach was insightful. He examined where the  bullet holes and damages were on returning aircraft and recommended that armor be added to all the places where bullet holes and damage did not exist. His reasoning was simple. The aircraft that returned could take the flack where the damage was. The aircraft that did not return must have been hit elsewhere.

In our own analytical work, we sometimes ignore the lack of evidence, which is sometimes more important than what we can quantify. In the context of product development or leading change or any other endeavor, what are the things you cannot see? This is an indicator. Are we fixated on counting and measuring damage, or are we also thinking about why we don’t see any? Just because we cannot see it does not mean it is not there. Absence of evidence is not evidence of absence.

The metrics game is one area where we are vulnerable to make the mistake of counting what we can at the expense of identifying what is  important. If you are coaching a team, are you looking for the absence of evidence in addition to it? Are you looking at evidence from both perspectives? The notion of False Dichotomy and Confirmation bias are errors in thinking triggered, among other things, by absence of evidence or ignorance of evidence.

One of the challenges of leading change is to look for things which are not part of the model, not part of the picture, to focus on outliers and anomalies. The damage to the aircraft is relevant only because the aircraft returned safely. It’s the dark side of the moon and the iceberg below the surface. We cannot see it, but it is there.

What is it that you do not see?

Advertisements