I benchmarked over 130 LLM configs with multiple stacked 16GB "e-waste" GPUs (V100, P100, K80)

Wait 5 sec.

The benchmarking journey continues! Continuing the project of seeing how these very old, very cheap, very available enterprise GPUs can be used today. The complete results are too long for reddit. The complete set of graphs, power analysis and commentary are on my blog. I've expanded the testing suite to focus heavily on LLM workloads (over 130 new tests!) and integrated power consumption monitoring into the gpu_box_benchmark tool. The goal remains: find out if decommissioned enterprise GPUs (Tesla K80, P100, V100) paired with X99 era hardware can still serve as a viable daily driver for self-hosted AI compared to modern commercial setups. TL;DR of the new data: The 3x V100-16GB (~$1k build): If you are considering building a dedicated homelab box today, this is the sweet spot. Assembling a full setup with two Xeon CPUs, motherboard, PSU, and cooling costs around $1,042 before RAM 💀. This setup comfortably clears my threshold for "real-time" chat (70+ tok/s) with impressive models: Qwen 3.6 35B: Ran at ~72 tok/s (at 128k context depth, Q8 quant). For comparison, a brand new Mac Studio M5 ($2,500) hits only ~44 tok/s on the same model. Gemma 4 26B: Ran at ~75 tok/s (at 64k context depth, Q8 quant). Qwen 2.5 3B: Ran at ~79 tok/s. This proves that for a single-user daily driver, the legacy silicon is still competitively priced against modern hardware. Scaling is for Context, Not Speed: Stacking GPUs linearly scales your VRAM (allowing you to fit larger models), but it doesn't magically double your generation speed. Two 16GB cards in Q4 quant allowed fitting ~32B parameter models, a desirable jump for homelabbers. Power Efficiency: I also measured power consumption as well and it's worth noting the efficiency gains across generations. The V100 wins hands down on performance-per-watt compared to the P100s. The K80 Strategy: The Tesla K80 isn't dead, but its singular remaining use case is welding a bunch of them together just to squeeze big models in. You essentially have to stack a lot of cheap K80s to hit a 72B+ parameter model. Expect speeds of 1–5 tok/s. This is a party trick, but it proves the VRAM scaling is real even on ancient silicon. If you decide to build your own GPU appliance, the biggest challenge is cooling these hot cards without fans. I've been working on a custom active air cooler to solve exactly this. If you're interested in helping out by stress-testing a sample, you can get involved here. Any models/tests missing? Any other combinations you'd like to see in a follow up?   submitted by   /u/eso_logic [link]   [comments]