Bad Decision Making

I tried an experiment with a few colleagues over the past couple of months based on Karl Popper’s famous experiment for confirmation bias. The results have big implications on how we can easily head down the path to making bad decisions.

The experiment goes like this. The experimenter gives the subjects a 3-digit pattern, “2 4 6” and asks them to write the rule that makes any other pattern conform to the same rule. The subjects may ask questions in the form of another pattern. For example, a subject may ask if the pattern 1 2 3 conforms, and the experimenter will answer either yes or no depending on whether or not the pattern conforms. After several attempts, the subjects write their “rule” for the pattern.

For example, subjects may ask:
“Does 4 8 12 conform?” The answer is yes.
“Does 6 8 10 conform?” Yes.
“Does 3 6 9 conform?” Yes. And so on…

Confirmation bias guarantees that you will ask more questions that get answered with a yes than with a no. The reason is that you start to build a model in your head of what the pattern conforms to, and you go about proving that you are right.

The answer to this puzzle is that the numbers must be increasing in value from left to right. (a < b < c). All someone has to do is offer the question “Does 2 3 1 conform?”, and you quickly start to converge because the answer is NO. One of my colleagues supplied negative integers (-2 -4 -6) and the answer was no. This led to some debate among the pair and they were far faster at converging than when they got yes answers.

I did the experiment with a pair of programmers and another pair of people, a programmer and a tester. The programmer-tester pair won. Why? Testers have the mindset of disproving rather than proving. Programmers like to prove their design works. Testers like to break features. Ironically, two testers in a pair don’t always do as well as a designer-tester pair. The diversity of the pair is what helps create the conditions for rapid convergence. Try this simple experiment yourself and share the results. The experiment is described in detail at the Developmental Psychology web site here.

Confirmation bias implies you filter and interpret data to support a belief you already have. The debate on Gun Control is an example of how the same data is interpreted different ways by people with differing perspectives. To increase government spending or reduce taxes to stimulate the economy is another debate where confirmation bias plays a role. There are huge implications for decision making. Human beings try to fit data to support their beliefs. Forcing yourself to disprove a hypothesis is powerful.

If you are responsible for delivering high quality products, what are the implications of confirmation bias on how you work? What can you do to create healthy team diversity and conflict which leads to better collaboration, and ultimately, better products?


Retrospective & Happy Holidays

November marks the one-year anniversary of this blog.  Looking back, the most popular posts of 2010 were as follows.

  1. The Pomodoro Technique – Agile Personal Time Management
  2. Team Performance – Signal to Noise Ratio
  3. March 29 2010 – Dinner with Jeff Sutherland
  4. Martin Fowler’s Survey of Version Control Tools + Subversion Multisite
  5. Servant Leadership
  6. The Marshmallow Challenge
  7. Launching an Agile Pilot
  8. Stuck in Paris – I’m still alive – Day Five
  9. ICST Paris 2010 – Innovation in Test
  10. The Trajectory of Change

It’s been a great year of change and growth. I have many family members, friends, and colleagues to thank for making this a special year. Ironically, the most traffic to this blog was in April when I was stuck in Paris for 5 weeks during the Icelandic volcano. Friends and family kept track of what I was up to along with the latest news on when I expected to get home, and of course, how I amused myself with food and volcano protection devices. I’ve had a few memorable professional moments. I was invited to speak at AgileTour RTP. I made a presentation at ICST Paris. But the best of all from a professional perspective, was watching our Scrum teams deliver outstanding results throughout the year. I am so proud of them.

A special Thank You to the important people in my life…you know who you are! Wishing you and yours the very best of the season and looking forward to an even better 2011.