Publish date: February 1, 2024 – MDedge
By Alicia Gallegos
The United States Medical Licensing Examination (USMLE) program is invalidating scores attained by some examinees after an investigation revealed a pattern of anomalous exam performance associated with test-takers from Nepal.
In a January 31 announcement, the USMLE program said that officials are in the process of notifying examinees with results in question and that the examinees will be required to take validation exams. The program did not offer further details about its investigation or how the questionable performance was identified.
“The USMLE program regularly monitors and analyzes examinees’ test performances for unusual score patterns or variations, and other information that could raise questions about the validity of an examinee’s results,” the program said in a statement. “Highly irregular patterns can be indicative of prior unauthorized access to secure exam content.”
Some medical graduates say the action against students cheating on the USMLE is long overdue.
The selling and buying of USMLE questions online have become rampant in recent years, particularly by groups within the international medical graduate (IMG) community, according to multiple IMGs who shared their concerns with this news organization. Sellers operate under pseudonyms across social media platforms and charge anywhere from $300 to $2000 for questions, Medscape research shows.
Facebook posts often advertise questions for sale, said Saqib Gul, MD, an IMG from Pakistan who has voiced concerns about the practice on social media.
“People make up fake profiles and tell others to [direct message] them for recalls,” he told this news organization. “There was a dedicated Facebook page that was doing this. In other cases, a couple of friends that took the exam remember a certain number of questions and write them down after the test.”
Ahmad Ozair, MD, an IMG from Lucknow, Uttar Pradesh, India, said that he has come across many groups online sharing or selling USMLE recalls. He first became suspicious when he saw several students, all from a few medical schools in Nepal, posting on social media about scoring in the 270 and 280-plus range.
“The statistical probability that you would have three or more candidates in the same year, scoring in the 99th percentile worldwide, belonging to a small geographical area is extremely low.”
Dr. Ozair, who now is studying public health at Johns Hopkins University in Baltimore, said that the issue is important for “all stakeholders” who care about patient safety: “Would you want a doctor who has cheated on the medical licensing exam to take care of you?”
In an interview, USMLE program spokesman Joe Knickrehm said that the program relies on multiple processes to detect and respond to claims that exam integrity is being compromised. The process includes monitoring performance data, an anonymous tip line for reporting suspicious behavior, and a thorough investigative process.
“The USMLE program regularly monitors social media channels for comments relating to exam security and irregular behavior and will initiate an investigation if warranted,” Mr. Knickrehm told this news organization. “The covert nature of this activity does not lend itself to a definitive statement regarding whether the problem has increased or decreased in recent years.”
Mr. Knickrehm said that the program’s STOPit app allows people to report suspicious behavior electronically to the USMLE program. Since its launch in 2021, the program has received more than 80 tips per year through the app, according to Mr. Knickrehm. Security violations are investigated by USMLE staff and reviewed by the USMLE Committee for Individualized Review (CIR). Anyone found to have engaged in irregular behavior by the CIR for activities undermining exam integrity are typically barred from access to the USMLE for multiple years.
How Easy Is It to Buy Recalls?
Two years ago, Dr B was approached by a former study partner who had just completed Step 2 of the USMLE. She asked whether Dr B wanted to buy recalled questions to help her pass.
“She paid this guy almost $2000 for recalls and told me if I pay this money, he’ll give me the recalls,” said Dr B, who asked to remain anonymous for fear of being associated with students cheating on the USMLE. “I told her I was not interested, and she said the guy would lower the price. I broke contact with her.”
Dr B, an IMG from Pakistan, was appalled. But she said that the episode was not the first time she has come across groups selling USMLE recalls or heard peers brag about having access to exam content.
“I am baffled at how many [groups] post on social media and brazenly advertise their ‘services,’” she told this news organization. “No one arrests them, their customers go on to score abnormally high on the boards, making it unachievable for people who take the honest route, plus giving IMGs a bad rep.”
Groups offering recalls are easily findable on sites such as Telegram and Signal. Telegram is a cloud-based messaging app that focuses on security, and Signal is an encrypted messaging service.
The website recallmastery.com purports to offer a range of USMLE recall packages, from a free, unsorted version to Step 1 and Step 2 packages that include “fresh updates,” and sections with “mostly repeated topics. Prices range from the free version to the $799 VIP package.
Another site called MedPox.com boasts 2024 Step 2 recalls, advertising “actual exam questions to get HIGH scores.” The website’s owner states that the recalls were collected “by my friends,” and to message them to be added to the “recalls group.”
A reporter was able to easily download a free version of alleged USMLE questions and answers from recallmastery.com. The document was a combination of typed and handwritten notes about medical questions, with red circles around recalled answers.
J. Bryan Carmody, MD, who blogs about medical education, reviewed a copy of the document. He said that the content appeared “credible” and was in fact recalled USMLE questions. However, the extent of which the question stem was recalled was incomplete at best, and there was little production value to the document, said Dr. Carmody, a nephrologist and associate professor of pediatrics at the Eastern Virgina Medical School in Norfolk.
The person selling the recall packages states on the website that the free version is not organized or sorted, but it allows viewers to “see how this works before paying for premium recalls.”
Mr. Knickrehm said that the program could not comment on the document, but that “whenever the USMLE program receives or locates information about a potential security violation, we investigate and take necessary action.”
When asked about the specific websites noted above, Knickrehm said that the program routinely monitors a wide array of websites, message boards, and chat rooms for USMLE-related materials. Though many sites advertise having USMLE recalls for sale, it’s more likely they are selling non-USMLE content, he said.
Using past content to cheat on medical exams is an old problem. In 2010, for example, the American Board of Internal Medicine suspended 139 physicians after they were caught cheating on the board exams. The scandal involved a vast cheating ring that included physicians memorizing questions and reproducing them after the tests. The board later sued a gastroenterologist for her part in the scandal.
In 2012, a CNN investigation exposed doctors who were memorizing test questions and creating sophisticated recall banks to cheat on radiology boards. The Association of American Medical Colleges sued a medical student in 2017 for attempting to secretly record content on the MCAT using spyglasses.
In recent years, Dr. Carmody said that he has received multiple messages and screenshots from concerned students and residents who were offered or encountered recalls.
“One thing that’s unclear is how legitimate the claims are,” he said. “Many of these recalls may be faulty or outdated. It could be someone who took the exam yesterday and has a photographic memory or it could be some sparsely recalled or mis-recalled information. Unless you’re willing to pay these people, you can’t inspect the quality, or even if you did, you wouldn’t know if the information was current or not.”
‘As an IMG, There Is So Much at Stake’
Whether recall sellers — and those buying them — are more frequently IMGs has fostered heated debate on social media.
On a Reddit thread devoted to IMG issues, posters expressed frustration about being bombarded with recall advertisements and unwanted messages about buying USMLE questions while trying to find study materials. One poster called the practices a “huge slap to all those IMGs who are struggling day and night, just to get a good score.”
In an X thread about the same subject, however, some self-described IMGs took offense to claims that IMGs might score higher because they have access to recalls. The allegations are “incendiary” and “malign hardworking IMGs,” posters wrote.
When Dr. Gul spoke out online about the “biopsy” culture, he received multiple private messages from fellow IMGs telling him to remove his comments, he said.
“I received a lot of backlash on social media,” he told this news organization. “Some IMGs asked me to take down my posts because they thought I was making IMGs look bad, and it might prompt authorities to take action or shut down international examination centers for IMGs.”
Most of the IMGs who spoke to this news organization were afraid to be publicly identified. Several IMG advocates and IMG associations contacted for the story did not respond. One medical education expert said that his institution advised him to “steer clear” of commenting because the issue was “controversial.”
“As an IMG, there is so much at stake,” Dr B said. “Any association with shady operations like these is an absolute suicide. I’m personally afraid of any repercussions of the sort.”
USMLE officials declined to comment on whether the buying or selling of recalls appears to be more prevalent among the IMG community, saying it is “difficult to generalize this behavior as ‘prevalent’ simply due to the clandestine nature of this activity.”
Cheat-Proofing the USMLE
The USMLE program has taken several steps intended to prevent cheating, but more needs to be done, medical education advocates say.
For example, Dr. Carmody called the recent change in the attempt limit for taking USMLE exams from six to four times a good move.
“The reality is, if you’re taking a USMLE exam five-plus times, you’re far more likely to be memorizing questions and selling them for shady test prep operations than you are to be legitimately pursuing U.S. residency training or licensure,” he wrote on X.
The 2022 move to make USMLE Step 1 pass or fail is another positive change, said Dr. Gul, who added that US programs should also put less weight on test scores and focus more on clinical experience.
“Many programs in the US prioritize scores rather than clinical experiences in home countries,” he said. “If program directors would remove these criteria, probably the cheating practices would stop. Clinical practice matters. When a doctor gets matched, they have to be good at seeing and treating patients, not just good at sitting in front of a screen and taking an exam.”
Turning over questions more rapidly would help curb the practices, Dr. Carmody said. Another strategy is using math techniques to identify unusual deviations that suggest cheating, he said.
A blueprint for the strategy was created after a cheating scandal involving Canada’s Medical Council of Canada Qualifying Examination (MCCQE) in 2004. After learning which questions were circulated, MCCQE administrators evaluated exams by comparing answers of compromised questions with the answers of noncompromised questions.
“For a person who was not cheating, the error of performance should be pretty similar on those two groups of questions,” Dr. Carmody said. “But if you were given the questions in advance, you might have very poor performance on questions that had not been compromised, and very high performance on those that had been compromised. That disparity is very unlikely to occur just by chance alone.”
Based on his research, Dr. Ozair is working on an academic review paper about cheating on the USMLE and on the Medical Council of Canada Qualification Examination. He said that he hopes the paper will raise more awareness about the problem and drive more action.
He and others interviewed for this story shared that the websites they’ve reported to the USMLE program are still active and offering recalls to buyers.
“Even if they are not actually offering something tangible or true, appearance matters,” Dr. Ozair said. “I think it’s worth the USMLE sending cease and desist letters and getting these websites taken down. This would restore faith in the process and underscore that this issue is being taken seriously.”
A version of this article appeared on Medscape.com.