A healthy dose of skepticism
The right tools can help you weigh inherent uncertainty in medical research.
The world is filled with uncertainty — even when it comes to the most carefully researched medical study. Learning to recognize that uncertainty is key to having a better understanding of those studies and how they can benefit your patients.
During Friday’s new session, “U025 – Uncertainties Masquerading as Certainties: Can You Trust the Study You Just Read?” Stephen Malachowski, MD, MS, dermatology resident at Virginia Commonwealth University’s Department of Dermatology in Richmond, said using statistical tools and keeping certain ethical and logical considerations in mind can aid in the correct interpretation of medical literature.
“When the goal is to positively impact patient care, clinicians and researchers alike should strive to step outside of their comfort zone to reason with data, and question statistical assumptions,” he said.
Assessing the variables
One common mistake people make when developing studies is dichotomizing — or categorizing — quantitative variables.
“An example would be to take age as it is measured but deciding to classify people as below some age limit or not,” Dr. Malachowski said. “This is problematic in many ways. First, information is lost, which mathematically reduces the statistical power of a study. In the case of a randomized clinical trial, the study must now recruit more patients to the trial to reach the same amount of information. If we create information loss that requires us to recruit more people than necessary, this raises the ethical consideration of exposing more people than needed to the risks of the trial.”
Dr. Malachowski said dermatologists should review study statistics such as sensitivity (how often a positive test is rendered to patients who have the disease in question) and specificity (how often a negative test is rendered to a healthy patient) with caution.
“The problem with this is inherent in the interpretation,” he said. “For a diagnosis, we don’t yet know if a patient has a disease, so something like sensitivity or specificity alone cannot tell us much about diagnosis. In fact, sensitivity and specificity are ‘backward’ for diagnosis because we don’t know who has the disease at the time of completing a test. We only know the test result and try to use the result, along with other information, to make a diagnosis.”
Limitations at play
Lee Wheless, MD, PhD, FAAD, assistant professor of dermatology and epidemiology at Vanderbilt University Medical Center in Nashville, Tennessee, said when it comes to reviewing a study, it is important to understand the limitations of its p-value (the likelihood that the study will find a particular set of observations to be associated with the outcome if there is no difference among the groups being studied) and its confidence interval (the range of values that are plausible, based on the data).
“It’s important to keep in mind that both of these values are really measures of how well a model explains the data in the study,” Dr. Wheless said. “A p-value does not reflect the clinical importance of the finding but rather can speak to how likely or not it is the variables in the model describe the relationship with the outcome. Similarly, a confidence interval does not tell us that ‘truth’ is found 95% of the time within its bounds. Rather, confidence intervals give us a sense of how certain we are about the estimates made by the model. Neither a p-value or a confidence interval tells us about the probability or truth of an error, which is a common misconception.”
The ripple effect
Drs. Wheless and Malachowski said it is important to recognize uncertainty in medical research because a clinical mistake based on a dermatologist’s uncertainty will affect a single patient, but an uncertain measure in a study that becomes accepted as truth has the potential to impact many patients.
Dr. Malachowski echoed that sentiment with an old adage. “There is an old saying: ‘The trouble with people is not that they don’t know but that they know so much that ain’t so,’” he said. “Recognizing uncertainty in medical research is important because the world we live in is filled with uncertainty, even about things we think we understand. Without recognizing uncertainty, patients may receive treatments that appear to work but don’t, or we may dismiss treatments that might work but are written off without proper evidence.”
Visit AAD DermWorld Meeting News Central for more articles.
