I wanted my local model to answer small questions without generating a paragraph or JSON I’d just have to parse. So I built semif-go: give it some state and a few options, and it returns typed choices, yes/no probabilities, or scores using the model’s next-token probabilities. It's mainly a server not just a touch and go script so you can use it for other applications and harnesses you have locally. Oh and btw, it works with text, and forwards images, audio, and video to llama.cpp when the loaded model supports them. I’ve tested text and images so far with Gemma 4 12B. Audio and video still need live testing. Huge credit to TheoLeeCJ and the SemIf/OpenJev contributors for the scoring approach, TypeSafe for the System One idea/API schema. https://github.com/wnzn/semif-go   submitted by   /u/wenyani [link]   [comments]