I think it's all working. I asked various models to make it all work for me, with the bulk being GLM 5.3 flash. Its not as fast as some of the benchmarks I've found but getting them working apparently requires even more surgery. I wanted a simple "Qwen 3.8 Flash Next FP8 MTP, 128GB VRAM in 4x Intel B70, 128GB RAM, vLLM in a container. Fully reproducable." According to the LLM tests: 450 t/s prefill, 20 t/s generation. The LLM found and made or backported fixes for several 232 overflows causing random weird crashes and it all seems stable now.... maybe. https://github.com/wormley/B70_Quad/tree/master/qfn-38-fp8-4xb70-mtp Before this all my manual vLLM testing ended in crashes for almost every model trying to run on multiple B70 cards.   submitted by   /u/38andstillgoing [link]   [comments]