I built an engine that runs Qwen-Flash-Next at nearly 400t/s on cheap(ish) hardware. 3x V100 32GB in an old R740. Originally forked from strata but the engine guts all built for the old Volta generation.

Read post on reddit.com

  submitted by   /u/jfinkpottery [link]   [comments]