🌋 Pushing PCIe x1 to the limit: lvgale concurrently serving 27B, 9B, and 2B LLMs on a hardware abomination 📈 Everyone loves a clean, unified server rack, but my standby node (lvgale) is an absolute physical abomination. It is a Frankenstein collection of mismatched parts and bolted-together VRAM that has no business running efficiently. The Hardware Setup (lvgale) CPU: Intel Core i7-6700K. GPUs: AMD RX 580 8GB VRAM (PCIe x16) AMD RX 570 8GB VRAM (PCIe x1) NVIDIA GTX 1660 Super 6GB VRAM (PCIe x2) NVIDIA GTX 1070 8GB VRAM (PCIe x1) The Bottleneck: Activations must travel across severely constrained x1 and x2 lanes between entirely different AMD and NVIDIA architectures. The Software Engine The cluster is orchestrated by Slipstream, a custom Nuitka-compiled Python orchestration engine. Here, it utilizes a mixed-vendor Vulkan backend with Flash Attention enabled to maintain stability across the fragmented topology. Concurrent Inference Performance In all cases seamlessly handling OCR and complex vision parsing. Qwen3.8:27b-vision: 3.82 t/s. Qwen3.5:9b-vision: 4.75 t/s. Qwen3.5:2b-vision: 12.47 t/s. While the 2B model runs slightly faster on a modern laptop iGPU due to unified memory, keeping a 27B model stable across this mixed-architecture, low-bandwidth topology without crashing the Vulkan driver is a massive win. For deeper technical architecture details, the engine is documented at slipstream-llm.com.   submitted by   /u/FieldMouse-AI [link]   [comments]