I get this question often from teams trying to decide whether to keep paying per token or bring the GPUs in-house. So here’s the practical read based on what we’ve seen, with numbers. API pricing for capable open-weights models is typically low single digits or less per million tokens. Self-hosting the same model has a fixed monthly cost, regardless of how much you push through it. I used an 8-GPU H200 node at ~$329k, spread over 48 months (monthly cost about $6.9K), plus ~$800/month for power and about 25% overhead. On the API side, I assume ~$2/million tokens. So break-even comes out about: $9.6k ÷ $2 x 1M = ~4.8B tokens/month (about 160M/day) Under 1B tokens/month: API is clearly cheaper, and you have zero operational burden. Between ~1B-5B a month (around 100M a day), it depends on whether your load is steady or spiky. Steady favours owning. Spiky favours the API, because you are paying for idle silicon the rest of the time. Above 5B tokens a month with a steady load, owning gets hard to argue against on pure cost. Ofc, these numbers assume the machine stays busy. If your traffic comes in bursts, you’d need more volume than this to break even. And some things can outweigh the math altogether. - Your data cannot leave your control. - Latency floors matter to you. Local inference removes a network hop and the shared provider queue. - You want fixed cost for budgeting. API prices and plans keep getting reshuffled, so a flat monthly number is easier to plan around. If you’re well under the crossover, you should just use the API. Anything above that, it's worth pricing out owning. Disclosure: I work at B3IQ, which builds and hosts GPU systems for teams. So i’ve tried to keep the assumptions neutral, but tell me if anything looks off. For those who’ve made the switch to self-hosting, what made you do it, and roughly where was your crossover?   submitted by   /u/recentheartbroken [link]   [comments]