Will the new pass-fail system for USMLE Step 1 help relieve the toxic pressure of board exams?
Key Takeaways
The USMLE Step 1 exam has changed from an assessment with a three-digit score to a pass-fail exam.
Critics of the previous version of the exam said it had a detrimental effect on medical student well-being, while critics of the new format say that the change has unintended consequences.
While pass-fail may alleviate some pressure on students, it also may increase the importance of medical school prestige and/or the Step 2 exam, critics said.
It’s an ordeal many physicians can recall, but for the wrong reasons. One 3rd-year medical student, writing in Academic Medicine, called the experience dissociative, surreal, and chaotic. It left her feeling “a sense of profound emptiness.”[]
This doctor wasn’t writing about the challenges of working during the height of the COVID-19 pandemic, the pain of losing a patient, or the grind of burnout. She was describing the US Medical Licensing Exam (USMLE) Step 1, a grueling gauntlet through which every aspiring attending must run. Step 1 has changed, but some wonder whether the revisions will have their intended effects.
From scored to pass-fail
What was once a numerical score became pass-fail as of January 2022, according to the AMA, which wrote that the Step 1 exam “created a parallel curriculum” directed to exam preparation and had adverse effects on medical student well-being.[]
In 2019, two medical students writing in BMJ Opinion described Step 1’s toll in detail.[]
"We’ve directly seen the untold depression, anxiety, and toxic stress that Step 1 causes."
— Reddy, et al., The BMJ Opinion
They added that the knowledge that “a poor score could wipe out your career dreams” means that medical students across the country “routinely study for up to 12–15 hours a day during a 4–8 week preparation period.”
According to a 2020 AMA article, Step 1 was always intended to be a pass-fail exam, with the primary purpose of determining medical licensure eligibility.[]
"Concerns were less about the examination itself, and more about how the score has been misapplied in the residency selection process."
— Brendan Murphy, AMA
Placing so much emphasis on the three-digit scores also distracted students from developing soft skills, such as communication abilities and teamwork, that the AMA considered to be critical competencies. And the old scoring system may have had another unintended effect.
Step 1 and diversity
A 2020 Health Science Reports research article sought to determine whether Step 1 exam scores affected the diversity in the medical school interview process.[]
The study reviewed score data from 10,541 applications at the Zucker School of Medicine at Hofstra/Northwell Health from May 2014 to May 2015. Researchers compared mean test scores of White applicants with those of Black and Hispanic applicants, who were referred to as underrepresented minorities in medicine (URiM).
The analysis showed that mean Step 1 scores were significantly higher among White applicants compared with URiM applicants. White applicants also tended to be younger, and their percentage of applicants who were male and members of the Alpha Omega Alpha Honor Medical Society was higher as well.
The researchers concluded that using Step 1 cutoff scores as an application filter could have undesirable effects on diversity.
"The use of USMLE step 1 cutoff scores as a recruitment and selection criteria contributes to the ‘leakiness of the pipeline,’ that is, the departure of students, particularly minorities, from a medical career path."
— Williams, et al., Health Science Reports
Academia weighs in
Will the change to pass-fail ultimately be a net positive? Some in academic medicine aren’t so sure.
A group of researchers publishing in the Journal of Surgical Education sought to assess how surgical program directors feel about the change and what it might mean for medical students in the long term.[] The researchers queried all program directors of ACGME-accredited surgical programs, and 55.5% responded.
Among the findings:
78.1% did not support the scoring change
19.6% believed it would boost student well-being
63.5% believed it would heighten the importance of medical school pedigree
52.7% thought it would create a disadvantage for international medical graduates
88.7% felt it would only make the Step 2 exam more important
Respondents were also concerned that pass/fail scoring may actually worsen socioeconomic disparities, noting that these disparities are linked with race in the US.
The researchers acknowledged “that those from lower [socioeconomic status] backgrounds may opt to attend a less prestigious medical school for financial reasons.”
"Without objective measures of achievement like Step 1 scores, these students lose an opportunity to distinguish themselves from students graduating from more prestigious schools."
— Pontell, et al., Journal of Surgical Education
‘Pressure to perform’ for Step 2
Similar findings emerged from a survey of 502 IM program directors (with a 41% response rate), as published in a Journal of General Internal Medicine report.[]
11.7% thought pass-fail scoring would help student well-being
74.8% felt it would actually make the screening process more difficult for applicants
88.4% thought the change would place greater emphasis on Step 2 scores
57.3% said it would increase the importance of medical school reputation
The researchers noted that, unlike other specialties, IM tends to emphasize Step 2 clinical knowledge (CK) performance, with 90% using it as an interview checkpoint. Most program directors consider it similarly to Step 1, they found.
“Previously, strong performances on Step 2 CK could help remediate low Step 1 scores,” the researchers wrote. “However, without a numeric Step 1 score, there will be even more pressure to perform well on Step 2 CK.”
Looking ahead
How this change truly affects medical students and medical education is anyone’s guess. According to an AMA April 2022 article, time (and research) will tell.[]
For many students, preparing for the exam likely won’t change. Though the format is now pass-fail, the content is roughly the same.
In 2020, 98% of test-takers passed on their first attempt. However, those who fail will still get four more tries, along with score reports to help them prepare for their next attempt.
Ultimately, it’s up to residency programs as to how they want to proceed. Some are using numerical scores and others have pass-fail results as filters, according to the AMA article. But will this prioritize the Step 2 exam in their eyes?
This is what David Marzano, MD, an associate professor and ob-gyn residency director at Michigan Medicine, is afraid of. He told the AMA he predicts Step 2 will eventually become the new Step 1.
"Our worry is we are trading one type of apple for another."
— David Marzano, MD
What this means for you
Physicians-in-training may breathe a sigh of relief that Step 1 is now a pass-fail assessment—but be careful what you wish for. Residency program directors fear that this will only make Step 2 scores more important, along with emphasizing medical school prestige. And medical students will still need to pass Step 1—something 10% of test-takers failed to do in 2020.