Sup guys, I'm releasing a piece of internal tooling for one of my own projects. I threw it into a publicly facing repository with setup. I'll discuss briefly why it's useful and why I build it, and why I'm sharing it. I've kinda fallen in love with the Cursor harness. There's nothing I like using more than the IDE with agents mode within cursor. However, unfortunately, the included usage for cost-effective models is dissapearing and so is my interest. This is a proxy layer representing itself as an openai compatible endpoint. I originally built this for my own goals but I have a feeling others would benefit. Rather than explain the entire project in one go, here were my pain points and here's how I resolved each one. KV Cache Invalidation The biggest pain point for me. Cursor constantly changes and modifies its own conversation history, which repeatedly invalidates cached prefixes and forces my local models to recompute / redo prefill. Insanely annoying and time wasting. The proxy mutates cursor's responses behind the scene to keep the prefix cache valid. I have a 97%+ cache hit-rate across thousands of requests through the proxy from both local and openrouter models. Works cleanly with images as well - vision won't invalidate the cache. Replica pools as a single model I have three GPUS. I wanted to run three separate instances of qwen 3.8 27b, one to use as an orchestrator and two as subagents, and I quickly realized cursor doesn't allow you to define custom models as subagents - which is crap. I solved this with a replica pool - it appears as just a single model to cursor, so it can spawn subagents via inherit, but on the backend it's multiple openai compatible endpoints of the same model. The proxy intelligently routes requests to the correct backend worker based on the prefix, so you can run multiple subagents under the same local model identifier without compromising speed - cursor thinks it's just one model, but it's really multiple instances being intelligently load balanced. No settable context lengths for custom models, and issues with compaction Cursor defaults custom models it doesn't recognize to 200k tokens context, and it can't be changed. This means for smaller context lengths the model won't compact its conversation in time, and eventually you'll hit an error that you're over the conversation length limit. PrefixRelay tracks the length of your conversations through cursor according to pre-defined context lengths you set per model - it runs its own compaction system in the background when you approach the model's true context length limit based on the models endpoint - so you'll never see any issues within cursor no matter what context length your model supports. Having to repeatedly switch my openai endpoint configuration I gotta switch it for local models, I gotta switch it to openrouter, etc etc. Huge pain in the butt. PrefixRelay lets you connect multiple models from multiple different endpoints in a way that you can use them all with cursor at the same time with custom model identifiers. Not allowed to use local endpoints Cursor does not let you point your endpoint at localhost, which is crap. I personally use a named cloudflare tunnel and prefixrelay as system processes, but this repo includes a quick start that'll spin up a cloudflare quick tunnel to the proxy endpoint, and tell you the exact URL to type into your cursor settings. Enjoy! I hope someone finds this useful. https://github.com/jackjusko/PrefixRelay   submitted by   /u/jjusko20 [link]   [comments]