Do online reviews convey physician quality? Research says no: A discussion with Dr. Timothy Daskivich

By John Murphy, MDLinx
Published September 22, 2017

Key Takeaways

Americans love to use online review sites to check the ratings for restaurants and hotels—and even plumbers and babysitters. But how well do rating sites measure the attributes of physicians?

Fifty-nine percent of patients report that online physician rating sites—such as Healthgrades, RateMDs, Vital, and Yelp—are important for choosing a physician. Of these patients, nearly three-quarters (72%) use review sites as their first step to find a new doctor, 19% use them to validate the choice of a doctor they’ve tentatively selected, and 9% use them to evaluate the doctor they already have, according to a 2015 survey.

In short, physician rating sites are important to patients. However, they’re important to doctors, too—85% of physicians and health care providers monitor online reviews about themselves, and 36% check out their competitors’ reviews, a 2013 survey reported.

But what are these review sites really rating? Consumers appear to believe that these reviews generally correlate with measures of physician quality, such as board certification, rating of medical education, and physician volume. But what little evidence is available shows that this is not the case, according to a recent article in the Journal of the American Medical Informatics Association.

Timothy Daskivich, MD, MSHPM

“Health care consumers might be less enthusiastic about consumer ratings if scores were definitively shown to be poor predictors of actual clinical performance,” wrote the study’s lead author Timothy Daskivich, MD, MSHPM, and colleagues from Cedars-Sinai Medical Center, in Los Angeles, CA.

In this interview, Dr. Daskivich explains the results of this study, describes how online reviews may translate to patient satisfaction, and what could be done to make online ratings a more valid indicator of physician quality.

MDLinx: What prompted you to conduct this study?

Dr. Daskivich: Patients frequently consult online consumer ratings to help select physicians—in fact, previous studies have shown that over 75% of patients are willing to make provider choices based on consumer ratings alone. We wanted to raise awareness among patients that online consumer ratings may not be telling the entire story about how “good or bad” a physician really is.

What we hypothesized—and eventually concluded from the results—is that ratings are not correlated with critical determinants of quality of care, such as Choosing Wisely measures. While we believe that consumer physician ratings are a very important part of how consumers should evaluate prospective doctors—since they are a direct measure of the patient experience—they should not be used as the sole source of information in the absence of other data on quality or value of care.

MDLinx: Was the study large enough to be generalizable for specialists across the country?

Dr. Daskivich: Patients should not rely solely upon one study for most any health decision. Patients and other stakeholders should rely on the preponderance of evidence, of which our study is one piece of evidence in this case. There are a few articles now showing that online ratings do not correlate with technical skill or adherence to best practice guidelines.

That said, while our sample size is relatively small, we were struck by the unequivocal consistency of the data. As stated in our paper: “Despite many comparisons across diverse performance metrics and multiple platforms, we found no signs of any statistically or clinically significant relationships, other than among the consumer scores themselves. Increasing sample size is therefore unlikely to capture any meaningful association.”

MDLinx: One association that you did find was that ratings of individual physicians were generally consistent across multiple review sites. Why do you think that is?

Dr. Daskivich: We addressed this in the article: “Despite the lack of understanding of what consumer ratings of physicians are intended to measure, we found that ratings were relatively consistent across platforms, suggesting that they may jointly measure a latent construct. Nonetheless, it remains unclear what the latent construct being measured is, though in part it may be patient satisfaction. A recent publication found that online reviews were modestly associated with National Committee for Quality Assurance consumer satisfaction scores for insurance plans (Pearson correlation = 0.376). Defining this latent construct is critical for appropriate use of consumer ratings scores, since it will at least offer patients a sense of what these scores mean, which is especially important given their strong influence on provider selection for the majority of patients.”

MDLinx: Did any of the findings in your study surprise you?

Dr. Daskivich: We were surprised to find that physicians who were rated poorly most often provided high quality care. In fact, as mentioned in the paper, “among physicians in the lowest quartile of specialty-specific performance scores, only 5% to 32% had consumer ratings in the lowest quartile across platforms.” This would suggest that patients seeking high quality care should not decide against seeing a physician based on consumer ratings data alone.

MDLinx: What advice do you have for physicians? Can they do anything to align their performance measures with their online reviews?

Dr. Daskivich: As service providers, we physicians should be held accountable for online ratings since they are a reflection of the service-related aspects of the care we provide. However, right now it appears that online ratings hold too much sway over how patients make decisions, out of proportion to what they offer in terms of comprehensive assessment of a physician’s performance.

MDLinx: What advice do you have for patients/consumers? How can they find out about performance measures rather than reading online reviews?

Dr. Daskivich: Based on the results of our study, we believe that patients should consider metrics evaluating quality and value of care in addition to consumer ratings when choosing physicians. Health care consumers can access information on quality of care of physicians across the country by referring to websites such as Medicare’s Physician Compare website (www.medicare.gov/physiciancompare/).

About Dr. Daskivich: Timothy Daskivich, MD, MSHPM, is Assistant Professor in the Division of Urology, Director of Health Services Research in the Department of Surgery, and a Urologic Oncologist in the Samuel Oschin Comprehensive Cancer Institute at Cedars-Sinai Medical Center, in Los Angeles, CA.

Share with emailShare to FacebookShare to LinkedInShare to Twitter
ADVERTISEMENT