Qwen3.8-Flash-Next (IQ3_XXS, 76 GB) at the full 262K context on 31 GB of RAM: 1,800 tok/s prompt, 130 tok/s decode, on a 5090 + a 4070 Ti SUPER on a PCIe x1 slot Strata (github.com/Niko1221/Strata) streams MoE experts from an mmap'd GGUF, and its own sizing rule says RAM >= expert shard + 10 GB, so 57 GB for the ISTA GSQ-RCO IQ3_XXS. I run it on 31 GB and it is fast now. Hardware: RTX 5090 32 GB + RTX 4070 Ti SUPER 16 GB (the small card sits on a chipset x1 slot, 0.8 GB/s), i7-14700K, one NVMe. Stock 0.1.33 with a layer split at 36: 80K-token prompt 890 tok/s with the 4070 at 100% and the 5090 idle, decode ~110 tok/s, and switching between two chats re-reads the other one (30K tokens = 48 s). Three patches on v0.1.33 (repo below, they apply to a pristine checkout): Prompts run entirely on the big card (port of Strata PR #269), the small card gets its layers' state copied afterwards, only the cells in use. Conversation parking works with the split, so alternating between a phone and a desktop session takes 0.6-1.2 s instead of 20-48 s. The real bottleneck on a box with less RAM than the expert file: the expert pool copied each 2 MB expert out of the mmap with memcpy after a MADV_WILLNEED hint. Under memory pressure the kernel drops that readahead and you pay one major page fault per 4 KB, hundreds per expert, while both GPUs wait. A pread per slice instead: major faults per benchmark run went from 24 million to 14 thousand. Bigger prompt chunks (--prefill auto:32768). Every chunk re-streams every layer's experts, so four times fewer chunks matters a lot when the working set does not fit in the page cache. Numbers on the final config (int8 KV, 32K cells resident per layer, MTP spec 4, vision on, 262,144 context): - 80K fresh prompt: 1,796 tok/s (45 s); 16K: 985; 2K: ~350 - follow-up turn on an 80K conversation: 6 s - decode: 128-134 tok/s median on real sampling (0.6 / 0.95 / 20) with --spec-min-p 0.8, 95-108 greedy - 40K needle + follow-ups and a two-conversation parking test all correct - VRAM: 5090 at 32.0 GB, 4070 at 15.7 GB; RAM: the engine ~5 GB, the rest page cache Repo with the patches, install script, launcher, benchmark tools and all measurements: https://github.com/ExTV/strata-5090-4070   submitted by   /u/Fz1zz [link]   [comments]