What the CFA Level III AI study actually measured
Researchers evaluated models on exam-style questions; those scores did not award a professional qualification.
Originally posted · Published here · 1 source · Resume reading
A CFA Level III benchmark study evaluated 23 language models on multiple-choice and essay-style responses. Its revised 2025-09-22 paper reported composite scores of 79.1 percent for o4-mini and 77.3 percent for Gemini 2.5 Flash. Those were research results, not professional qualifications awarded to software. [1]
Keep the grading method with the result
The researchers compared prompting strategies and used a revised essay-grading method. A score therefore depends on the question set, prompt and grading procedure, as well as the model. Read those conditions before comparing the number with a person's examination result. [1]
The social caption compressed the study into a claim that AI had passed the exam. The useful finding is narrower: models produced responses that scored well under the researchers' evaluation.
For a financial workflow, an exam benchmark can suggest tasks to evaluate. It does not establish reliability for an unfamiliar client's circumstances or authorize an assistant to make decisions with real money.
Keep this guide
Filed under AI agents
Sources
- CFA Level III benchmark paperarxiv.org
Next in this stack
Keyword LOOP
Five rules for building an AI agent that works
Keyword STARTER
Write skill metadata an agent can recognize
Keyword SIMPLEFIRST
Choose an AI workflow before building an agent