I’ve been spending the last few weeks experimenting with running local agent loops (using Ollama / vLLM on a single GPU) tasked with analyzing structured tabular data, risk management, and quantitative decision-making. One recurring trend in popular agent tutorials and frameworks is having the model calculate formulas, deduce risk-reward ratios, or "reason through" quantitative boundaries directly in-context. After benchmarking Qwen 2.5 (7B, 14B, 32B) and Llama 3.1 (8B, 70B quantized), here are two massive failure modes we consistently hit, and I’m curious how the community is architecting around them: --- ### 1. The "Plausible Math" Illusion (Even with strict JSON Schema) Even when enforcing strict schemas with Outlines / Pydantic tool-calling, 8B and 14B models frequently generate numbers that look statistically reasonable at a glance, but are mathematically inverted. * A model would correctly extract `Price: 150`, `Target: 180`, `Stop: 140`, but then output `Reward-to-Risk: 1:0.3` instead of `1:3.0`, or hallucinate that a stock trading below net asset value has a PBR of 4.2. * When forced into multi-step Chain-of-Thought (CoT), 32B models improve slightly, but latency triples and they still occasionally drop negative signs or mess up compound percentage calculations. * **Our current conclusion**: LLMs should be treated strictly as semantic processors, narrative synthesizers, and natural-language routers. Any and all arithmetic, filtering, and mathematical bounds should be calculated in native Python and injected as immutable read-only context. --- ### 2. Multi-Agent Groupthink (Sycophancy Cascades) When running multi-persona debates locally (e.g., bull vs. bear vs. risk manager): * If Agent 1 produces a flawed premise with high linguistic confidence ("This asset shows strong momentum breakout"), Agent 2 and Agent 3 almost always collapse into sycophancy. * Prompting Agent 2 with *"You are an extremely skeptical auditor looking for fatal flaws"* rarely works reliably on 8B/14B models—they will politely concede to Agent 1 after 1–2 turns. * The only reliable fix we found was a deterministic hard-code veto: having a non-LLM Python gate that kills the workflow if mathematical thresholds are violated, completely ignoring what the LLMs "agreed" on. --- ### Questions for people building local agents: **Have you completely stripped arithmetic/math out of your local agent prompts, or has anyone found a reliable way to make small models (8B–14B) accurately compute and verify their own numeric logic?** **For those running multi-agent consensus locally, how are you preventing sycophancy without relying on hard-coded Python boundary rules?** **At what parameter size (14B vs 32B vs 70B) did you see a genuine phase shift in reliable structured tool-calling when latency is a constraint?** Would love to hear what architecture patterns have actually survived production/stress-testing on your local setups.   submitted by   /u/OpenComplaint524 [link]   [comments]