What is a Theory For?

People use theories for two different purposes, one of which is much more helpful than  the other.

The recent death of Clayton Christensen of The Innovator’s Dilemma – “disruptive innovation” – fame reminded me of controversies related to the theory.  The book was hugely successful in the business world, and then came the backlash –  a New Yorker article in 2014 by Harvard historian Jill Lepore questioning its factual basis. Now Prof Lepore is a distinguished historian and what she said was very probably true, but the criticism misses the point. Or at least, it ought to miss the point.

Prof Lepore’s article was entitled “What The Gospel of Innovation Gets Wrong.” And that is the problem. These sorts of theories are not gospels – not big overarching explanations of how everything is supposed to work. The problem with Christensen’s theory, which is not his fault, is that it got stretched too far. In his book (1997) he in fact says that most innovations are not disruptive. By 2014, however, the idea had developed that all significant innovation was disruptive.

If you look to Christensen’s theory as a theory of how all innovation works, or as a way of predicting which innovations will be significant and which will not, or even more as a recipe for creating huge change in a given industry, then it’s not up to the task.

Alternatively, if you look at it as “here are some interesting descriptions of  things which operated through a particular mechanism and produced results that were very good news for some people and very bad news for others, which suggest that you, particularly if you are a long-established incumbent in a market, should keep a careful watch out for similar things in your world which might operate according to similar mechansims and cause you similar grief, ” then I would suggest that the Innovator’s Dilemma is a must-read.

So, what is a theory? Is it a position to defend, or a tool? If the first, then the Lepore-Christensen debate has a point. But if on the other hand a theory is a tool, then it is something we can keep in our mental toolbox and use when necessary. Disruptive innovations may be fundamental in the development of our industry, or they may be irrelevant. It doesn’t matter. We have the tool if we need it.

Using theories as tools not as positions to defend has two huge benefits.

We avoid sterile debates about whether our theory does in fact explain everything. (The fact is, it can’t. The Unified Theory of Everything is still tantalisingly out of reach of even the physicists, but here we are in the realm of social science where it is nowhere in sight).

Using theories as tools means that we can chooose between them. We have not a single view but a toolbox. And the thing about toolboxes is that the more tools they have, the better.

(Acknowledgement: the idea of theories as tools or positions to defend comes from a book, Already Free: Buddhism Meets Psychotherapy On The Path Of Liberation by Bruce Tift. It takes the idea in a different direction altogether, but if the title intrigues you it’s an excellent book).