CFA Institute addresses complaints about 25% Level I pass rate

As we reported yesterday, it's been a terrible week for anyone who took the CFA® Level I exam in May 2021. Just 25% of people passed, lower than at any point in the exam's 58-year history.

The low pass rate has been welcomed by some existing CFA Charterholders. "Some rebalancing was required," one told us, speaking on condition of anonymity. "Plenty of Charterholders have been really unimpressed at how much easier it's become to pass the CFA exams over the years." 

For the most part, though, schadenfreude has been outweighed by derision and despair among this year's test sitters. One junior told us his job is on the line after failing the Level I qualification: "It doesn't feel fair. Anyone with a basic understanding of statistics understands that such extreme outliers are extremely, extremely,  unlikely." Another pointed out that the average pass rate since 1963 has been 42.6% and the standard deviation is ~4.4%. A 25% pass rate is therefore -4 standard deviations, or an event with a less than .01% probability. 

"I suspect that the computer-based results were statistically different from what they expected, and the CFA has been scrambling to react," said one candidate. "It was probably the perfect storm of events (pandemic, move to computer-based testing, decreasing test length, and cheating) that produced a range of scores the CFA never expected.  I think the CFA realized there were factors in play that they didn't control for, and instead of letting 40-45% of people pass by appropriately adjusting the MPS, they decreased the pass percentage." 

We put some of these complaints to CFA Institute. This is what they said...

We understand that this year's results were delayed and that guidance was revised from 7-10 weeks after the exams to 10 weeks. Was this a factor in the pass rate change?

CFAI:  "The May administration results were not unduly delayed. While we advise that exam results are expected to be available approximately 8-10 weeks after the close of an exam window, our priority is to allow for the right amount of time for the thorough analysis of performance that takes place after every examination sitting.

After each administration, we conduct a thorough analysis of performance for individual test questions and the overall exam, including factors such as difficulty level. This process derives an MPS (Minimum Passing Score) that ensures fairness and comparability across administrations, as variations in the difficulty of each test is taken into account. The MPS draws on several principles which include consistency of the "bar” setting methodology over time, ensuring fairness and a consistent passing standard to candidates, and the integrity of the exam. Once determined, the Board of Governors convenes to consider and approve."

If the lower pass rate was due to the stop-start nature of testing, why didn't this impact the December results?

CFAI: There were some Level I candidates who were deferred from June 2020 to December 2020, however there were thousands of May 2021 Level I candidates who were deferred more than once for a total of a 12-month delay. 

Do you have any more of a breakdown on the results provided on your site?

CFAI "Our data policies permit us to release candidate pass rate information at the global level only.  Exam candidates pass or fail relative to their performance against the minimum passing score (MPS), which is the pass or fail “bar” reviewed and approved by the CFA Institute Board of Governors. The MPS we used to assess candidate performance in the May Level I exam administration is comparable with prior administrations, including the first Level I computer-based testing exam in February 2021."

Some candidates have suggested there may have been more cheating due to the computer-based exams (with no one followed to bathrooms and the opportunity to share content from similar exams taken on different days) and that this prompted the low pass rate as a response. Can you comment on what was done to prevent cheating?

CFAI: "CFA Institute goes to great lengths to protect the security and integrity of the CFA exam and the testing process worldwide. When addressing the security and integrity of the exam, we have three specific goals: (1) to maintain the security of exam content; (2) to ensure no candidate has an unfair advantage over another; and (3) to ensure only those individuals truly qualified to pass the exam do so.   

We have numerous measures in place to meet these goals. We are not at liberty to disclose all the security measures we have in place, including the number of exams administered throughout the window.  However, we can share that CBT exams are delivered in proctored physical test centers with robust security measures in place.

Candidates go through a significant security review upon check-in at the test center.  This review includes:  identity and passport confirmation, enforcement of the personal belongings policy, and physical inspection of the candidate and their belongings. Test centers closely monitor candidates for cheating in any form. All unusual candidate behavior is documented by on-site proctors and shared with CFA Institute for investigation.  Should any candidate come across information that threatens the security of the exam, or any unethical candidate behavior, we encourage them to share their knowledge with us so that appropriate action can be taken. 

The transition to computer-based-testing aligns our testing experience with global testing and credentialing best practices. This technological progression addresses how global testing and credentialing practices have changed and recognizes that the investment profession operates in a digital world. Further, this change addresses the learning and exam preferences of today’s candidates. Offering the exams and learning experience in a digital environment allows CFA Institute to better model the tasks and work experience of today’s investment professionals."

While none of these responses are entirely illuminating, the suggestion that the May influx of repeat candidates was a one-off should hopefully mean that the low pass rate isn't repeated in the coming round of exams at the end of July. The disgruntled CFA Charterholder said today's candidates have it easy anyway. "My L1 pass rate was below 40%, which was pretty tough. Plus I failed L3 which meant I had to wait 12 months to retake. That's much more brutal that a L1 candidate who has to wait 3 months to resit (material still super fresh in their mind). Don't you think?"

CFA Institute does not endorse, promote or warrant the accuracy or quality of eFinancialCareers. CFA® and Chartered Financial Analyst® are registered trademarks owned by CFA Institute.

Have a confidential story, tip, or comment you’d like to share? Contact: sbutcher@efinancialcareers.com  in the first instance. Whatsapp/Signal/Telegram also available. Bear with us if you leave a comment at the bottom of this article: all our comments are moderated by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libelous (in which case it won’t.)

Photo by Ivan Aleksic on Unsplash