I've been working on Agent Brain Hub, an open-source "brain" that several agents share. What one agent learns about a user, the others can recall, with permissions so private things stay private. GIF above: the repair agent hears "my car is in the shop for 3 days", and later the travel agent offers a rental car at the destination without being told again. Why it works with small local models: the brain does the heavy lifting itself (fact extraction, retrieval, ranking, permissions), so the LLM mostly turns a prepared context into a reply. I tested it end to end with Qwen3-4B-Instruct-2507-FP8 on vLLM. With no LLM at all it still runs, with rules and templates. Local setup: git clone https://github.com/leluong141996-dev/Agent-Brain-Hub cd Agent-Brain-Hub docker compose --profile vllm up -d # hub + Qwen3-4B on your GPU Ollama and LM Studio work too: pick them in Settings, fetch the model list, test, apply. No restart. A few things I learned along the way: - vLLM 0.10.2 crashed with "CUDA illegal memory access" when a greedy (temperature 0) JSON-mode request was batched together with sampled requests. Using temperature 0.1 for the JSON calls made it go away. - Servers disagree on parameters (max_tokens vs max_completion_tokens, temperature, json mode, chat_template_kwargs). Instead of a config matrix, the client reads the 400/422 error, drops or renames the parameter and retries, and remembers it for that server. - Embeddings are local feature hashing (256 dims, with character bigrams for Japanese) so nothing leaves the machine. It's crude but fine for a demo; a real embedding model is the obvious upgrade. Storage is SQLite. With 20k episodes it reloads in about 0.2 s, and a turn's write is about 40 ms. UI in English, Vietnamese and Japanese. Repo: https://github.com/leluong141996-dev/Agent-Brain-Hub Question for you: which small model do you use for structured fact extraction? I'd like to move more of the extraction from rules to the LLM without losing reliability on 4B-class models.   submitted by   /u/Inner-Ad-41 [link]   [comments]