Help me post this on the most relevant reddits please: Title: I tested 48 local models on an M1 Max 64GB as agents (16 multi-step tool tasks + 10 rebuilt from real agent failures, x3 runs), plus speed and context for 54 Body: tok/s doesn't tell you whether a model can run an agent, so I built an unattended harness and ran my whole LM Studio library through it on an M1 Max 64 GB: speed and memory at the same 64k context, max context, a quality set, 16 multi-step tool tasks in a simulated workspace, and 10 tasks rebuilt from mistakes my own assistants actually made (stamping invoices without checking the order, moving data while 'tidying' a chart, deleting when the backup check timed out, acting before the plan was approved). Every agent task ran 3 times and is graded on what happened in the workspace, not on how the answer sounds. What to run Best overall: qwen3.8-27b Q4_K_M (GGUF) (99% quality, 100% agent, 16.9 tok/s) Best everyday pick (40+ tok/s): ornith-1.5-35b-a3b (GGUF) (94% quality, 98% agent, 94% real cases, 52.4 tok/s) Spanish writing: ternary-bonsai-2-27b (MLX) (writing 95%) Coding: lfm2.5-24b-a2b (GGUF) (coding 100%) Agents (Hermes, OpenCode, n8n style): qwen3.8-27b Q4_K_M (GGUF) (agent score 100%, 51 s per task, 100% of tasks solved in all 3 runs) Snappy agent (30 s or less per task): sage-a1.5 Q4_K_M (GGUF) (agent score 99%, 15 s per task, 100% of tasks solved in all 3 runs) Unattended agent (real-world cases): qwen3.8-27b Q4_K_M (GGUF) (98% on the real cases, 0 serious slips in 30 runs, 101 s per task) Coding agent: lfm2.5-24b-a2b (GGUF) (fixed the repos in 100% of runs, 12 s per task) Top 12 by the real-world tier (memory at 64k context; scores out of 100, under ~3 points apart is noise; bold model = a pick above, bold number = best in column or that model's standout metric) Model Mem @64k GB Real cases % Agent % Quality % tok/s qwen3.8-27b Q4_K_M (GGUF) 15.7 98 100 99 17 qwen3.8-27b Q8_0 (GGUF) 46.3 97 100 99 18 ornith-1.5-35b-a3b (GGUF) 25.6 94 98 94 52 qwen3.8-27b (MLX) 22.5 92 100 98 18 mimo-v2.6-distill-qwen-9b Q8_0 (GGUF) 16.6 91 95 84 34 mimo-v2.6-distill-qwen-9b Q4_K_S (GGUF) 10.6 90 92 86 39 ternary-bonsai-2-27b (MLX) 12.0 90 100 95 18 underdog-saluki-27b-1.0 (GGUF) 14.7 88 100 94 14 holo4-27b (GGUF) 28.0 87 100 92 13 kolibri-1 (GGUF) 43.1 85 93 98 43 flint Q4_K_XL (GGUF) 29.4 85 95 96 51 muse-glimmer (GGUF) 23.4 84 96 99 13 What stood out Hardest agent tasks: clarify (56% of runs solved); plan_meeting (65% of runs solved); long_order (69% of runs solved). 5 of 48 models obeyed a prompt injection hidden in an email at least once (kolibri-1 (GGUF), agents-a1-4b (GGUF), gpt-oss-20b (GGUF), underdog-woof-4b-1.1 (MLX), k2-horizon-3.7b (GGUF)). Best agent score: qwen3.8-27b Q4_K_M (GGUF) (100%, 51 s per task). Best under 30 s per task: sage-a1.5 Q4_K_M (GGUF) (99%, 15 s). Real-world failure tier: best qwen3.8-27b Q4_K_M (GGUF) (98%); 46 of 48 models made at least one serious slip (something hard to undo, or a false claim); hardest case: invoice_crosscheck (13% of runs solved). Biggest drops from the main agent score: gemma-4-26b-a4b (GGUF) (98 to 71), sage-a1.5 Q4_K_M (GGUF) (99 to 74), spark-x2.6-4b (GGUF) (97 to 72). Prompt cache: on GGUF models a follow-up turn on the same long agent prefix started 152x faster than the cold start (median of 41 models). Prompt cache: on MLX models a follow-up turn on the same long agent prefix started 41x faster than the cold start (median of 7 models). Caveats One person's tasks on one machine. Half the tasks are in Spanish because that's what I use. Writing quality is scored by local judge models from another family; everything else is deterministic code checking the final state. The base quality set is nearly saturated for good models; the real-world tier is where they separate. Full tables (54 models: speed, memory, context, quant comparisons, prompt cache, every agent task), the harness and every trace: https://github.com/DesignwithHugo/LocalLlmBench If you have a different Mac, run it and open a PR with your results folder; numbers from other machines are the most useful part. Model requests welcome if they fit in 64 GB.   submitted by   /u/HAVT_ [link]   [comments]