# What the CFA Level III AI study actually measured

> Researchers evaluated models on exam-style questions; those scores did not award a professional qualification.

Source: https://fullbytedeveloper.com/blog/cfa-level-three-model-benchmark
Originally posted: 2025-09-27
Published here: 2026-11-06

A CFA Level III benchmark study evaluated 23 language models on multiple-choice and essay-style responses. Its revised 2025-09-22 paper reported composite scores of 79.1 percent for o4-mini and 77.3 percent for Gemini 2.5 Flash. Those were research results, not professional qualifications awarded to software. [1]

## Keep the grading method with the result

The researchers compared prompting strategies and used a revised essay-grading method. A score therefore depends on the question set, prompt and grading procedure, as well as the model. Read those conditions before comparing the number with a person's examination result. [1]

The social caption compressed the study into a claim that AI had passed the exam. The useful finding is narrower: models produced responses that scored well under the researchers' evaluation.

For a financial workflow, an exam benchmark can suggest tasks to evaluate. It does not establish reliability for an unfamiliar client's circumstances or authorize an assistant to make decisions with real money.

## Sources

1. [CFA Level III benchmark paper](https://arxiv.org/abs/2507.02954)

## Original posts

- [Instagram post](https://www.instagram.com/fullbytedeveloper/p/DPHRY9VDNJT/)