Codos: #1 on EnterpriseRAG-Bench at 512k documents

TL;DR

Codos is now first on EnterpriseRAG-Bench at the full corpus: 92.64 across 511,962 documents and 500 questions, re-scored and published by the benchmark maintainers at Onyx under the entry name codos.ai. The next entries are Flexor.ai at 91.48 and Causal Dynamics Lab at 90.91. Cloud search products sit between 42 and 49 on the same questions.

In September we published a preliminary 94% at 42,587 files under our own judge. This post is the official number at twelve times that scale. It is the result we were building toward.

The benchmark at full scale

EnterpriseRAG-Bench simulates a company called Redwood Inference: 285,605 Slack messages, 121,390 emails, 35,308 Linear issues, 25,108 Drive documents, 15,017 CRM records, 10,173 meeting transcripts, 8,052 GitHub items, and 11,309 Jira and Confluence pages. The 500 questions are the kind employees ask: what was decided, who owns it, which number applies, what changed since.

Three things make the full corpus hard in ways the smaller tiers are not.

• Look-alikes. The corpus is seeded with near-duplicate documents that describe a similar situation for a different customer, date, region or version, with different values. A question's qualifiers decide which document is the source. Reading a plausible document instead of the right one produces a confident, fluent, wrong answer.
• Distance. For many questions the source document is not lexically close to the question. A paraphrased situation has to be matched against half a million candidates, most of which share its vocabulary.
• Strict scoring. The overall score is correctness times completeness, judged by GPT-5.4 against a gold answer and a list of gold facts. An answer that states one wrong value, or offers two values for the same quantity, scores zero for that question regardless of everything else it got right. Twenty of the questions are designed to have no answer in the corpus at all.

Results

Published leaderboard values, October 19, 2026. Invalid extra docs counts cited documents the judge deems irrelevant; lower is better. The lower rows are reproduced from the September post's snapshot; a dash means we did not re-check the sub-metrics.

Codos leads on all three quality metrics, and the margin is widest on document recall: the answers cite the right sources 96.45% of the time. That matters beyond the score. A company brain whose answers can be traced to the correct record is one people can act on.

By question type

Scores by question type, from our scoring of the submitted answers with the official scorer:

Semantic questions, where the question describes a situation in words the document never uses, are where the lead is built: 92.5 on a quarter of the benchmark, four points clear of the next system. Constrained questions, whose qualifiers rule out the look-alikes, and completeness questions, which ask for every item, follow. Every "not found" question and every high-level question was answered correctly.

How the result was verified

We submitted answers; Onyx re-ran its own scorer and judge, checked a held-out set of unseen questions against the main result, and published the number. The main set is one end-to-end run of the system, not the best of several, and no model was trained on the benchmark.

What changes at half a million documents

Our September post reported what a graph-based memory does at 42k files. Between 42k and 512k the problem changes, and most of the work behind this result went into three things.

1. Rank globally, read locally

At full scale no agent can browse its way to the answer; the Astra file agent that reached 86 at 42k would, by our estimate, fall into the sixties at 512k, because grep over half a million files buries the right document under thousands of plausible ones. Every strong system on the board now ranks candidates across the whole corpus first and lets a model read only the top of that ranking. The quality of that ranking, and especially how it handles a question whose wording is far from the document's, is where the points are.

2. Decide which record governs

The look-alike documents are the benchmark's sharpest trap, and they mirror real companies, where the same incident, deal or policy exists in several versions across Slack, tickets and wikis. Finding the right document is not enough; the system has to commit to it. Aethos commits to one governing record per answer rather than hedging between versions. An answer that offers two values for the same quantity scores zero on the benchmark, and it should not ship from a company brain either.

3. Say "not found" and mean it

Twenty of the 500 questions have no answer in the corpus. A system that always produces something scores zero on all of them and, worse, misleads the person asking. Aethos answered every one of those questions correctly, and the discipline behind that is the same one that keeps the other answers honest: an answer is only as good as the record it can point to.

What is still hard

Project-related questions are the frontier for the whole field, and the clearest headroom in our own numbers. They ask for the state of a project whose facts are spread across a dozen loosely written documents with no single governing record, so the answer has to be assembled rather than found. Every system on the board scores far lower here than on any other type. Getting these right is a synthesis problem, not a retrieval one, and it is the next thing we are working on.

At 92 and above, a point of score is a handful of questions, and the judge itself disagrees with itself on a few of them per run. We say so rather than claim a precision the benchmark cannot support.

What this means for the Codos Company Brain

Aethos is the memory behind the Codos Company Brain: the system that turns a company's Slack, email, tickets, documents, CRM and meetings into knowledge that people and agents can query. EnterpriseRAG-Bench is the closest public measurement of that job at the scale of a real mid-size company, and on it Codos now answers more questions correctly, more completely, and from the right sources than any other published system, including the search products of the three largest cloud providers, which score roughly half.

For a company deploying a brain, the metrics translate directly. Correctness is whether the answer can be trusted. Completeness is whether the person has to ask again. Recall is whether they can click through to the source and verify. "Not found" is whether the system knows what it does not know. Those are the properties that decide whether employees and agents use a company brain or route around it, and they are the properties this benchmark scores.

The September post forecast that enterprise memory would be a solved problem by the first half of 2027. At the full corpus the gap to the ceiling is now a few points, concentrated in one question type, and the cost per question is under a dollar. We will keep publishing what we measure, including the parts that are still hard.

Thanks to Gleb Sidora, Dima Khanarin, Rahul Sethuram, Kevin Simback and Kate Bainova for their work on Aethos and on this research, and to the Onyx team for running the benchmark and its verification process.