POV 8GB VRAM + 16GB RAM folks checking locallama everyday:

Wait 5 sec.

One can only hope for Qwen4 35B A3B or similar (maybe with extra ngrams for 35B -> 70B+... until then, Gemma 26B QAT my beloved (solid 26 Tg/s)...   submitted by   /u/RobustLokiX [link]   [comments]