Basalt: Qwen 3.8 FN NVFP4, single 5090 Hi guys, so I finally got a RAM upgrade and have been working on the top two pain points from Basalt: NVFP4 support and faster prefill. NVFP4: Basalt runs NVFP4 natively now, a new .basalt pack keeps stock NVIDIA experts and stores the dense layers as Q6_K. Numbers for a 5090 (400w) + 128 GB of DDR5 RAM, single stream: - struct: 304 tok/s - prose: 183 tok/s - w/ peer gpu (5060 Ti): struct 479 / prose 239 Quality vs Q8_0 (100 chunks): PPL 3.308 (Q8_0: 3.286), KLD 0.068, 91.5% same top-1. Prefill: 8,582 tok/s on a 64K prompt (7.6s to first token). Against Infernix on the same NVFP4 weights and same box it's up to 1.3x on decode and 1.6x on prefill. Also merged a bunch of stability and performance fixes over the last week, coming up next Unsloth Q4 + Q8 (near lossless) tuning. Give it a try: Repo: github.com/jesdga95/basalt HF: huggingface.co/jesdga/Qwen3.8-Flash-Next-Basalt   submitted by   /u/jesdga95 [link]   [comments]