I think we live in an unscientific age in which almost all the buffeting of communication and television–words, books, and so on–are unscientific. As a result, there is a considerable amount of intellectual tyranny in the name of science.
~ Richard Feynman, 1966
Refuting unscientific anti-e-cigarette commentaries takes a lot of work these days. These articles pull facts from thin air, make specious arguments, cherry-pick evidence, and manufacture conclusions to support the personal biases of the author. Analyzing one of these articles with an eye to separating fact from fiction takes time–time most people donât haveÂ or are unwilling to devote to the process. However, sometimes an author unknowingly makes it easy for you. Just a small amount of effort, perhaps just a few minutes, is all you need to determine whether the piece is credible. An infamous example of this is the Margaret Cuomo video. She has some fame, speaks with great confidence, and appears to know what sheâs talking about. Most critics focused on her outrageously unscientific claimsÂ but ignored a much more fundamental problem: She had no legitimacy to speak authoritatively about e-cigarettes in the first place. This lack of legitimacy should have been obviousÂ but was missed by many people because of a basic critical thinking error known as poor source evaluation.
The reality of poor source evaluation was demonstrated by a recent series of studies by Stanford researchers Sam Wineberg and Sarah McGrew. They assessed studentsâ ability to judge the credibility of online sources of information and were taken aback by the glaring mistakes the students made when coming to their conclusions. Students trusted the search engine to place the most reliable sources first. They readily accepted all sources as being equally reliable, failing to properly distinguish between established and respected scientific organizations and extremist political front groups with lofty sounding names. They were routinely fooled by sponsored links (essentially advertisements), mistaking them for legitimate news articles. All of this, despite the fact that these students were free to make a few extra clicks and learn some critical information that would have at least raised skepticism about the reliability of some sources. That is, they engaged in no source evaluation.
College students arenât especially more prone to making this error than anyone else. Everyone who prowls the internet looking for reliable and trustworthy information is prone to making this mistake. As a lesson in source evaluation, weâll take a look at a recent op-ed written by Sudip Bose, MD, published at the Huffington Post. I read widely in the field of nicotine and tobacco research, not just articles in the press, but the research reports they refer to. After decades of being in this field as a scientist and researcher, Iâm fairly confident I know the major players in the field. Even so, Iâd never heard of Sudip Bose. Because heâs writing on a topic in my field, the natural reaction was to think, who is this guy? Doing what most of the Stanford students failed to do, I did a Google search of his name, and it returned this bio. Looking it over, I learned Dr. Bose is a practicing emergency care physician, Iraq war veteran, leadership speaker, CEO, and a guest and consultant forÂ The Dr. Oz Show. He seems to enjoy a modicum of fame and respect by engaging in these and many other pursuits. However, Dr. Boseâs bio also indicates he has absolutely no experience, training, scholarship, or practical knowledge about e-cigarettes and related fields like tobacco use or addiction. Those extra few clicks raised a puzzling question: Just what, exactly, qualifies him to be offering his opinion about e-cigarettes?
I wonât spend much time on the specific content of Boseâs article; itâs unmitigated junk from start to finish. He commits striking errors of fact, demonstrates a fundamental lack of understanding of the basic concepts, makes unsupported claims, and engages in feeble logic to arrive at his conclusions. A reader, even without any deep understanding of the topics Dr. Bose pontificates about, should suspect his piece could hardly be anything other than unmitigated junk. This comes from a rough assessment (from his bio) of what knowledge base he likely does have expertise in, the nature of how genuine expertise is acquired, and how it is unrealistic to expect that having expertise in one field confers expertise in another.
To become truly expert in a specialized area of knowledge takes long, hard years of study, practical experience, synthesizing research findings, and basically eating, sleeping, living, in the complex world of a specialized area of knowledge. Along the way, one would naturally pick up a survey-course level of knowledge in closely related fields, but this doesnât qualify as expertise. For example, an electrical engineer is likely acquainted with the basics of civil engineering, but no one would hire them to develop a plan for building a flood wall. Likewise, thereâs nothing about the process of becoming an accomplished emergency care physician (or writing a book about cancer survivorship) that would confer a trustworthy level of expertise in the unrelated fields of tobacco use, nicotine, or addiction. This is not to say they couldnât have developed expertise in these areas. Itâs just highly unlikely because it takes a herculean amount of time and effort to become proficient in two different bodies of knowledge. In the case of Drs. Bose and Cuomo, there is zero evidence from their bios that theyâve made even the barest attempt to do so.
Just to be clear, source evaluation is not some royal road to truth. Itâs merely a first critical step one typically might take to arrive at a confident judgment of truth or fiction. Moreover, itâs a step too easily omitted when you agree with the main thrust of the commentary. Most of the time source evaluation doesnât tell you enough, and you need to analyze the content of the article. But sometimes doing a little source evaluation can sharpen your critical eye and save you a lot of work. When thereâs a complete mismatch between an authorâs background, training, education, and current field of expertise and the topic theyâre writing about, then you can dismiss them as inexpert and give their opinion no more weight than you would your drunk uncleâs view on international trade. Although it makes sense to consider what Bose has to say about emergency medicineÂ or tap Cuomoâs views on cancer survivorship, it simply makes no sense to trust they know what theyâre talking about when it comes to e-cigarettes.
Dr. Brian Carter
CASAA Director of Scientific Communications