AI disagreement may shake patients' trust in doctors
After the session, participants answered questions about the doctor’s recommendation, their perceptions and trust in the doctor.
The research team found that agreement between Dr. Alex and the AI chatbot boosted patient confidence in the doctor’s recommendation. Disagreement had the opposite effect, increasing perceptions of medical uncertainty and doctor laziness. Anthropomorphism, or how human-like the AI-powered doctor seemed, also played a role in the observed outcomes. The positive effects of AI agreement and the negative effects of AI disagreement occur only when the doctor was perceived as more human-like.
The findings have important implications for services like telehealth and app-based and other online medical consultations, the researchers said.
“More than half of the participants in our study perceived the AI-simulated doctor as being human-like, demonstrating the capabilities of AI to replicate the professional behaviors of human doctors, at least from a conversational aspect,” said first author Cheng “Chris” Chen, assistant professor of emerging media and technology at Oregon State University who received her doctoral degree from Penn State. “This can bring challenges to doctors who provide online services where their true identity is not visible or clearly communicated. Patients may perceive AI as human or human as AI.”
The effect of AI disagreement on medical uncertainty was especially strong for individuals who believe that machines like AI are more accurate, objective and precise than humans, the researchers noted. These findings suggest that AI can have tremendous authority in planting seeds of doubt in patients’ minds, Sundar said.
The team also suggested strategies for communicating AI disagreement in ways that do not undermine patient trust in their physicians.
“A doctor can still communicate AI disagreement in implicit ways, like saying one of the tools they’re using has highlighted a few points that may be worth exploring a bit more closely, and offering to explore them together,” Chen said.
Explaining potential reasons for AI disagreement can help to reduce medical uncertainty and perceived laziness, Sundar added.
“Calling extra attention to the fact that there’s disagreement and showing that they’re on top of it can reduce the perception of laziness,” he said. “Then if the doctor can lend more nuance into why the medical uncertainty happened — like if the AI system is using data coming from a mostly white, Western sample, and the patient is non-Western, so the results might not apply to them — will communicate the doctor’s activeness.”
Other study co-authors include Yuan Sun, University of Florida, Gainesville; and Mengqi Liao, University of Georgia, who both received their doctoral degrees at Penn State.
This article was originally published on Pennsylvania State University Health and Medicine News.