The illusion of validity

Early in Psychologist Daniel Kahneman’s career, he took a role with the Israeli army.

To study new recruits in training, analyse their interactions through a range of data models, then provide recommendations as to which officers would make it as leaders in the field.

When comparing his recommendations generated through the use of data models vs simply the opinions of experienced Commanders using observation and instinct – it turned out that only nominal improvements were produced from Kahneman’s approach.

Regardless, the army continued to run the tests, confidently believing in the importance of the data they were collecting even though the results continued to disagree.

This pre-conceived confidence in the best approach is what Kahneman labelled “The Illusion of Validity”.

Kahneman reflected that:

“Confidence is a feeling, one determined mostly by the coherence of the story and by the ease with which it comes to mind, even when the evidence for the story is sparse and unreliable…”

Some stories we’re still surprised to see marketers put confidence in include:

These stories that were true in the Industrial Economy are actually a handicap in our new digital Connection Economy, where the rules have been re-written.

If you’re transitioning from traditional broadcast marketing, rather than take your existing assumptions into the online space, better to start from first principles.

One Comment

jonathonsciola
July 8, 2015 3:14 pm

Google is the chief example of this. They only hire engineers in management positions,yet they proved themselves utter failtures when it came to Google Buzz, Google Plus (yet to be seen), Google Wave, and so on. While innovation and risk taking is to be encouraged, the note to take here is that even with ALL the knowledge and data in the world, (they own AI companies!) they can’t wing it.

Leave a Reply