I set up a dedicated workstation for local AI experimentation with various models and tools. But every single one of them wants all the available VRAM. To fix this, I built a request dispatching layer called vramble. It automatically loads and unloads models and manages the queue using a priority system. Now I can run different workflows without manually managing models (or wasting Claude tokens to orchestrate the process). Any contribution is welcome, it works for me but is a new build so can't guarantee that will work directly on any machine Side note: Qwen 27B at 200+ t/s is performing way better than I expected. opeMIT license, available on GitHub: github.com/ValerioDolci/vramble   submitted by   /u/FeydRowan [link]   [comments]