I'm the author of the proxy tested here (mnemonic-proxy), so read this with that in mind. Translated with AI. What it is, in one line. An OpenAI-compatible proxy between an agent and a local server. When the conversation outgrows the window, it moves old tool outputs to an archive and gives the model a recall tool to read them back. What I measured Model: Qwen3.8 Flash Next, IQ3_XXS. Engine: Strata 0.1.40, KV int8, MTP speculative decoding, 10 experts. One RTX 4070 Ti 12 GB, 64 GB RAM. Seed 1. T 0.6, top_p 0.95, top_k 20. max_tokens 4,096, reasoning budget 3,072. Benchmark: AgentLongBench, kf-c, tool_response. Two question types: Count Frequency and Find Duplicates. 25 questions per type at 64k (~74K-token history) and 128k (~150K-token history). 100 questions. Four methods, official scorer (an answer counts only if fully right): full window: 81,920 / 163,840 tokens; the proxy (0.3.5), window 32,768; summary: the same model compacts the history into at most 2,048 tokens when the window fills, window 32,768; truncation, window 32,768. Window vs history: the proxy, summary and truncation never see more than 32,768 tokens at once; the conversations are ~2.3× (64k) and ~4.6× (128k) longer than that. Primary test, preregistered: proxy vs summary, paired, exact McNemar, 64k + 128k pooled. First run: everything near chance correct / 100 full window proxy summary truncation main run 51 52 56 44 Proxy vs summary: 13 vs 17 discordant, p = 0.58. Answering "0 / no" every time scores 50/100. Why: the question was ambiguous The benchmark means "the tool results for round N" as the tool output before the "Round N: Guess…" message. I checked: the official answers come out exactly that way on 100 of 100 questions. The model, even with the full window, read it as the output after "Round N". On 51 of 100 questions the two readings give different answers. Methods that had the evidence were systematically wrong there. Methods without it were guessing, and guesses aren't systematically wrong. Second run, preregistered before launch Same 100 questions, same everything. Only change: one sentence added to every question saying which tool output "round N" means. Official answers and scorer unchanged. No re-scoring. Validity check, fixed in advance: do answers still follow the "after" reading? full window: 41/51 in the first run, 15/51 now; proxy: 41/51, now 8/51. So the note worked for the methods that have the evidence. The summary stays at 29/51, about what answering "0 / no" every time would give (31/51 match the "after" reading). It isn't misreading, it is guessing: the full evidence is in its final prompt in only 15 of 100 questions. correct full window proxy summary truncation 64k (/50) 38 41 27 23 128k (/50) 31 45 20 26 both (/100) 69 86 47 49 https://preview.redd.it/yd2rm3r0ifwh1.png?width=2000&format=png&auto=webp&s=95a2027b416d5313f2e4e117c60e628c4a1cece2 Proxy vs summary, pooled: 44 questions right only with the proxy, 5 only with the summary. p = 7.6e-9. 64k: 18 vs 4, p = 0.004. 128k: 26 vs 1, p = 4e-7. Proxy vs truncation: 42 vs 5, p = 2.5e-8. Proxy vs full window: +17 points, 90% CI [+7, +26]. At 64k alone it's +6, CI [−7, +19]: not shown either way. At 128k the full window's reasoning hits the 3,072-token budget in 32 of 50 answers (the proxy's in 8; summary 45, truncation 44). That may explain part of the gap. I haven't tested it. Summary vs full window: −22 points, 90% CI [−33, −10]. Dropping truncated answers, or the one question I used while developing: no verdict involving the proxy changes. https://preview.redd.it/kx2ntdfaifwh1.png?width=2000&format=png&auto=webp&s=3dd1e10d14cc0ef032f08c495ff5c3601464a6c8 Time (median per question) Median time per question (full window / proxy / summary / truncation): 64k, end to end: 70 s / 136 s / 127 s / 58 s 128k, end to end: 102 s / 286 s / 251 s / 65 s 128k, time to first token of the answer: 61 s / 1.9 s / 8.6 s / 11 s https://preview.redd.it/4oou5aycifwh1.png?width=2000&format=png&auto=webp&s=8b5e0a1ced3749737b4281392fbc92be938e4c47 Most of the proxy's time is replaying the whole history through it before the question (229 s median at 128k). In new prompt tokens per question it is the most expensive: 328K median at 128k vs 150K for the full window. At question time alone it reads 19K new tokens on average. In a real agent that replay doesn't happen in one block; the work is spread over the turns. I didn't measure a long session, so I won't claim a saving there. VRAM was the same for every method (~11.6 GB; the engine reserves the GPU). Limits One model, one quant, one engine, one benchmark, two question types, 100 questions, one seed. One window size: 32K. An earlier 16K pilot (15 questions, ~37K and ~74K histories, three question types, the original ambiguous question, an earlier 0.3.5 build) did not favour the proxy: full window 9, truncation 7, summary 6, proxy 6. At 16K the masked prompt still takes 9–11K tokens and 4K are reserved for the answer, so there is almost no room left to search the archive. So far I have no evidence that the proxy helps below 32K. In daily use I run it with a 128K window, but I haven't measured answer quality there. The clarifying note was added after the first run. The second run was preregistered, but it exists because of the first one. Run-to-run noise is large: on the 49 questions where the note doesn't change the right answer, the full window went from 42 to 35 correct between runs, the proxy from 46 to 44. Find Target Offsets is excluded: in a pilot even the full window scored 0/5 (floor effect plus a scorer issue with "id1 and id2" answers). 3 of the proxy's answers at 64k came back empty: the model kept calling recall after the last allowed round. It's fixed in 0.3.6, but I did not rerun them. They count as wrong. Repo: github.com/lastreload/mnemonic-proxy   submitted by   /u/Any_Flounder8768 [link]   [comments]