ChatGPT is apparently taking the medical world—or at least the media—by storm, and I’ve been wondering just how smart this robot is. So, I decided to put it to the test.
I'm a family physician, and I asked the AI chatbot to describe a typical day in the life of someone in my profession. Here’s what I found, in a nutshell.
ChatGPT's strengths and weaknesses
ChatGPT was able to correctly identify the major components of my working day, from patient consultations and note-taking to administrative tasks, but its knowledge was lacking in some key areas, such as dealing with health insurance billing codes. (It also believes there is plenty of time for physicians to take breaks—and that these frequent breathers can be spaced out evenly throughout the day. Laugh out loud.)
A series of tasks, according to AI
When I asked the bot the question, “What do family physicians do all day?” ChatGPT was particularly verbose. For example, the bot listed “patient consultations” as a task separate from medical history, physical exams, and treatment planning, even though these all fall under the umbrella of a patient consultation. According to ChatGPT, a family physician’s day also includes the following:
Chronic disease management
Vaccinations and immunizations
Health promotion and education
Reviewing test results
Continuity of care
Caring for emergencies
An impractical number of breaks
ChatGPT correctly noted that physicians take appointments with patients during the average work day.
It broke up appointments into time periods of morning, mid-day, and afternoon. It also indicated there was plenty of time for physicians to catch their breath, including a mid-morning break and a lunch break.
But as too many physicians know, this isn’t quite correct.
Breaks really depend on how busy the practice is and can vary from day to day. It is quite possible that some days there can be time for a proper lunch, whereas other days, the physician may be lucky enough to grab a quick bite while going over patient notes.
Don't let ChatGPT do your books ... yet, anyway
I also asked ChatGPT, “How much time does a family physician spend with patients each day?” Quite frankly, this is a difficult question to answer, as it depends on the level of care the physician determines is necessary for each patient.
The amount of time a physician spends with a patient often correlates to the type of visit (acute problem vs complicated chronic diseases, new patient vs returning patient, and etc.), and the amount of time and billing codes will determine how much the physician is paid for each of these patient visits.
In general, a patient visit is 15 to 45 minutes. Timing also depends on personal preference, as some physicians opt to spend more time with each patient to better develop the doctor-patient relationship (which could negatively impact the practice financially).
In contrast, other physicians prefer shorter appointments to be able to tend to a higher patient volume (which could be positive, financially).
ChatGPT didn't demonstrate an understanding of this part of the industry (aka, the "business side" of medicine).
Its knowledge was also lacking regarding any type of billing codes for visit types. I believe this shows that ChatGPT answers these prompts without adding any additional layers, such as billing and finances, which are often truly the most time-consuming parts of a physician’s day.
The most challenging part of my day, per ChatGPT
ChatGPT listed 10 items as the “most challenging” part of my day (which indicates a lack of understanding of the question, I think). ChatGPT listed the following as challenges:
Balancing patient needs
Continuity of care
Keeping up with medical advancements
Managing patient expectations
Healthcare system challenges
I thought ChatGPT was especially articulate in describing administrative burdens: “Family physicians often have to handle significant administrative tasks, such as updating medical records, completing paperwork, obtaining prior authorizations for tests or medications, and dealing with insurance-related matters. These tasks can consume valuable time that could otherwise be spent with patients,” the bot said.
"Missing from this list, in my opinion, is the need to balance medical ethics with patient requests."
— Kristen Fuller, MD
I found it interesting that it toggled between descriptive and vague—while it was able to detail “administrative burdens," it didn’t explain what it meant by “healthcare system challenges.”
All in all, a fine effort
Despite some shortcomings, ChatGPT is well-aware of what a family physician experiences during a typical day in the office. However, its knowledge wasn't comprehensive.
I believe it is limited in how it draws connections and in how it decides what is the most relevant information to impart to the user. Its especially vague understanding of the intricacies of insurance billing and medical ethics, in particular, leads me to believe that only humans are capable of fully grasping the nuances of patient care.Read Next: ChatGPT: The clinician's new tech assistant?