Let’s take a difficult one: Should we require doctors to have licenses?
This isn’t difficult at all, you say? It’s simple. Yes, we should require doctors to have licenses, OBVIOUSLY! You don’t want your surgeon to be the taxi driver who just took you to the hospital. DUH!
That turns out to be the wrong answer.
The right answer is: “I don’t know.”
Oh. Well. What should I do with that? You can always say “I don’t know” to anything, and there’s no discussion. Actually, what I mean is that the right answer is “I don’t know. How can we find out?” What method would we use to understand this particular problem?
The one method we use far too often is to look into our limited store of experiential data, pull out a few anecdotes or clips from CNN, and declare that we have a pretty good way to answer the question. After all, this is what they do at the Harvard Business School, and we know it doesn’t work.
The right method is what we call the outside view: how would a Martian descending to earth answer any question. The Martian would have to start doing research, gathering data, asking questions, and trying to put a puzzle together. Here are example good questions:
Is there anywhere that actually doesn’t require licenses? How are their results?
Are there varying degrees of licensing? Can we compare their results in some smart way?
What is the track record of different kinds of doctors with licenses in different places?
Are there any specialties where a license is particularly helpful or harmful? What data do we have on this?
Is there any academic research on this?
Are there any metastudies?
Have there been any natural experiments?
Have there been any controlled experiments?
Now we’re getting somewhere. So let me ask you: Should we require doctors to be licensed? Take the outside view and SHOW me some evidence in the comments by pointing to research you think is valid. DON’T give me your unvarnished opinion - save that for Twitter and Facebook.
If you don’t get Leandro Herrero’s amazing daily thoughts, I highly recommend them. Start with this post:
Don’t forget to subscribe when you’re there!