# Check the paper behind a RAG failure claim

> A real limitation in single-vector retrieval does not establish that every document assistant is broken.

Source: https://fullbytedeveloper.com/blog/rag-retrieval-claim-evidence-check
Originally posted: 2026-01-13
Published here: 2026-11-06

The original post attributed a critical flaw in document AI to Stanford without naming a paper. That caption is not enough to identify or verify the claimed result.

A relevant, identifiable study is On the Theoretical Limitations of Embedding-Based Retrieval. Its LIMIT benchmark examines constraints of single-vector retrieval. The associated research repository is maintained by Google DeepMind; this is not evidence for the unspecified Stanford attribution. [1][2]

## Test retrieval separately from the answer

Retrieval-augmented generation, or RAG, finds source material before asking a model to answer. If the relevant passage never reaches the model, a fluent answer can still be wrong.

Create a small set of questions with known supporting passages. Check whether retrieval returns those passages, then check whether the answer uses them correctly. Include questions with similar wording but different constraints, and questions the collection cannot answer.

A failed retrieval test tells you where to investigate: document parsing, chunk boundaries, search, filtering or ranking. It does not automatically prove that every RAG system shares one fatal flaw. Read the experiment's scope before applying its conclusion to your product.

## Sources

1. [LIMIT retrieval paper](https://arxiv.org/abs/2508.21038)
2. [Google DeepMind LIMIT research repository](https://github.com/google-deepmind/limit)

## Original posts

- [Instagram post](https://www.instagram.com/fullbytedeveloper/p/DTc_sHJjOiA/)