Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses 📄 https://arxiv.org/abs/2610.41172 This is the paper behind the upcoming RAGFlow 1.0. Rather than another RAG pipeline, RAGFlow 1.0 is built around a different idea: an agent-native database. The paper is where that idea gets argued out, so I'd like to share the thesis and the conclusion rather than the benchmarks. The shift we keep underestimating. RAG's center of gravity has moved. Retrieval is no longer a query→embedding→similarity call; it's a loop in which an agent writes queries, reads what it finds, reformulates, and decides when it has enough. Once that loop exists, the classical objectives of the retrieval layer — better chunking, better embeddings, higher Recall@k — stop being the binding constraint. The agent can already find things. What it can't do is avoid paying for the reading. Two conclusions from the paper. First, retrieval is a harness problem before it is an index problem: the index decides what can be found, the harness decides whether finding becomes an answer. Second, compilation is not a better retriever and doesn't need to be. We tested the usual structural artifacts — summary trees, outlines, wikis, graphs — under a competent harness, and none of them produced a detectable accuracy gain. Structure is not additional evidence. What compilation can do is supply better context: smaller, provenance-bound, sufficient. The reframing. So we stopped treating compilation as retrieval and started treating it as a supply layer. Knowledge compilation is an ingestion-time pipeline that enriches source documents into typed artifacts — claims with verbatim evidence, entity relations, timelines, ontologies, wikis, memory — and the harness decides which artifact enters the window and when the loop stops. Compilation manufactures context; the harness governs it; tools are the contract between them. Context becomes infrastructure when it is compiled once, typed, versioned, and governed. Why "agent-native database". Once context is compiled upstream, an agent no longer queries a ranked list of chunks. It's supplied with provenance-bearing context that is compiled once and reused across queries, artifacts, and sessions. The compiled artifacts are the data; the harness is what decides what enters the window; the tools are the only path from stored context to model. It's not a relational database — no query language, no schema enforcement — and it's not a new retrieval paradigm. It's a different answer to what a retrieval system owes an agent: not passages to read, but the smallest sufficient set of verified facts, supplied in advance and governed like data. What we're honest about. The cost saving is real but it is bought with retrieval rounds, and we say so in the abstract, the discussion, and the conclusion rather than burying it. Deployment outcomes for claims, graph, wiki are published as an empty ledger — we list the metrics and the decision rules, and we mark every row "no measured effect" because none has met its pre-registered target yet. An unused artifact is cost without context, and we'd rather say that than fill the column with demo-collection claims. RAGFlow 1.0 is the open-source realization of that contract. Happy to discuss the economics, where we think this breaks on enterprise corpora, or why we think the dense leg is the wrong default inside a harness.   submitted by   /u/newpeak [link]   [comments]