If I ask an AI agent to help with an application, I might want it to research the role, fill in a draft, or show me what it plans to submit. That does not mean it can press Submit. The same thing happens with email. “Help me ask them about this” might mean “write a draft for me.” It does not mean “send the first thing you write before I see it.” I wanted a small local detector for this exact gap: did the user ask the agent to prepare an action, or did the user actually approve doing it? I tested six small language models. I kept each model frozen and looked at the patterns inside it as it read the request, the conversation, and the action it was about to take. These internal patterns are usually called activations. I then trained a tiny logistic probe to separate approved actions from actions that still needed permission. MiniCPM5-1B worked best in my test. The useful signal came from middle outputs 13 and 14, not the last layer. The learned probe and its normalization data take 28,708 bytes. On a separate test of 3,000 examples, the activation probe reached 0.8137 AUROC. A normal text classifier trained on the same 1,600 examples reached 0.7897. Higher is better. The paired difference was +0.0240, with a 95% bootstrap interval of +0.0056 to +0.0429 and p=0.0150. The probe, local scorer, hashes, pinned model version, and benchmark results are here: https://huggingface.co/mcpware/intentprobe-minicpm5-1b-authorization The wider IntentProbe repo is here: https://github.com/mcpware/IntentProbe The design is to use this as an extra signal beside exact checks: what is being sent, who will receive it, whether the user saw the final version, and whether anything changed after approval. The other finding I found interesting was that different models worked best at different layers. There is no single “always read the last layer” rule. If you have a clean authorization dataset, or a strange example where an agent prepared something and acted without clear approval, I would love to test it.   submitted by   /u/Think-Investment-557 [link]   [comments]