Hey r/LocalLLaMA, a while ago I posted BeeNara, a 332 MB CPU model that puts a document into one of your folders or says "none fits". Your questions in that thread turned into most of the work since. v0.4 is out (Apache-2.0): https://huggingface.co/collections/Kwokou/beenara-local-document-sorting What changed since v0.2 -Folders it has never seen: on 1,500 tasks with unseen folder trees, names and documents (written by a different LLM than the training data), accuracy went 70 % (v0.2) → 85.5 % (v0.3) → 91.5 % (v0.4). The old v0.2 benchmark stays around 95 %. -It files on its own only when it's safe: a Learn-then-Test threshold promises that, with 90 % confidence, at most 1 % of auto-filed documents land in a wrong folder. v0.4 auto-files 59 % of those tasks, 0.3 % of them wrong. v0.3 managed 21.5 % with 2.5 % wrong, because its threshold didn't survive new folder names. -New: BeeNara Big. Qwen3.5-2B-Base + LoRA + a decision head that reads the whole folder tree (up to ~600 folders) and the page image. 92–94 % up to 24 folders, 89 % at 25–49, 82–84 % at 50–199, 66 % at 200+. Needs a GPU (~4.3 GB in 8-bit). -Biggest lesson: label smoothing killed "ask when unsure". With it, no threshold could be certified at all; without it, the same accuracy and it works. -Data: still only synthetic, no user documents. Two local teachers (Qwen3.8-27B, gpt-oss-120b) write documents and check each other's output, across 32 document types, with real OCR noise. Where it still struggles Big is certified to file on its own only for 3–9 folders. In larger trees it is sometimes confidently wrong, so it only suggests there. Big spots "none fits" in 74 % of cases (small model: 92 %). Short notes, receipts and badly degraded scans: roughly 60–80 %. With the right folder missing, it filed insurance policies into "Contracts". Arguable, but wrong. With folder descriptions it didn't happen. Everything is tested on synthetic data so far. Next for v0.5: decide in large trees step by step (branch first), more varied trees, and learning from its own training errors.   submitted by   /u/razer_psycho [link]   [comments]