I was listening to a podcast a few days ago, when I fell into a familiar trap. As soon as the podcast was over, I found myself looking for a particular supplement online. Why? Because according to a real and credential medical doctor in the podcast, one study found that mortality was close to zero for people taking this supplement. Wow, what a discovery! Who wouldn’t want to reduce their probability of dying?!?
Fortunately, as I was adding it to my online shopping cart, I stopped to think: Why haven’t I heard this before? How does this supplement actually work? Are there confounding variables that could explain the results in the study? Has this been replicated? What do other medical doctors think?
Anything that seems too good to be true needs to be scrutinized. Even experts need to be scrutinized. Here’s an example: If 4 out of 5 dentists recommend a toothbrush, it’s probably a very good toothbrush. So do I just go ahead and buy it? Yes… no… maybe… it depends! If I have 5 minutes to make a decision, then absolutely go with the consensus from experts. BUT if there is time to evaluate, then it might make sense to find out what the 5th dentist suggests and learn more about it to determine if it is a better option for you:
- Read other credentialed experts: They’ve been studying this for a while and most likely have a better grasp of the big picture than the average person.
- Evaluate what they did: Does their explanation of the science behind the suggestion make sense?
- Compare their sample to yourself: Do the people in the study have similar characteristics to me? For example, are they in the same age range? Are they in the same physical condition? Do I have specific factors like allergies to consider?
- Look for replications: This can be tricky if it’s a new study but always check for other research that confirms or rejects the findings of the study you just learned about? For example, are there studies showing that the supplement had no effect on the outcome you are interested in?
- Look for other points of view: Do other experts agree? Look for reputable sources and differing points of view – don’t fall into a confirmation bias trap, where you only value information that confirms what you already “know” to be true. This will also help you formulate your own counterarguments when evaluating the original claim.
- Recognize your limits: If you haven’t taken a course in human physiology, defer to those who have. If you want to build your knowledge in specific areas of science, take a free online course. There are plenty of resources out there: Coursera, EdEx and Udemy are some examples. It may not be as rigorous as enrolling in a program but it will be more comprehensive than a 20 minute Google dive into a topic you know little about.
- Make up your mind: Once you’ve evaluated different points of view and evidence go ahead and make a decision.
Listening to people with novel ideas is important and it is those innovative thinkers that drive science and technology forward. However, it is important to be careful consumers of information. Stop, evaluate, learn, evaluate again, and then decide.