I've been building small models that do one job: answer the yes/no / pick-one decisions inside agents ("is this tool call safe?", "is this a refund request?", "which team gets this ticket?") and return a probability for every option instead of generated text. Numbers (the six benchmarks Jev 1.13 is evaluated on; Jev = their published figures): | Model | Size | Score | Laptop speed (1k-token doc, 5 questions, M4 Pro) | |---|---|---|---| | Decido-1 | 26B MoE, 4B active | 85.4 | ~4 s CPU, ~1.1 s Metal | | Decido-1-Fast | 1.7B | 73.0 | ~0.6 s CPU, ~0.3 s Metal | | Decido-1-Ultra | 0.6B | 62.2 | ~0.4 s CPU, ~0.13 s Metal | | Decido-1-Max (reasons first) | 26B MoE | 89.2 | slow on a laptop | | Jev 1.13 | API | 83.9 | — | - Q8_0 GGUFs on llama.cpp; repeat calls are bit-identical on CPU - Answers flip when options are reordered 4.8% of the time (Jev reports 14.6%) Try it - `pip install decido-ai` (prebuilt for Mac / Linux / Windows; uses Metal on a Mac) - No-signup playground (runs the 1.7B): https://getdecido.ai/playground - Weights: https://huggingface.co/decido-ai Happy to answer questions. I'm most interested in decisions where it fails.   submitted by   /u/gauravapiscean [link]   [comments]