Followed up on the post about running Qwen 3.8 Flash Next with 8 GB VRAM and 32 GB RAM. I wanted to see how it does on my Steam Machine. Setup Valve Steam Machine, stock SteamOS 3.9.5, AMD Navi 33 with 8 GB, 32 GB RAM, NVMe Stock Strata terminal 0.2.9 (engine 0.1.55, HIP package), Gyro S TQ1_0 128K context, q4_0 KV, no vision, MTP off, served on the LAN Results (Strata's own timing line from the engine log, about 8K-token prompts, 3 runs each) Config Prefill t/s Decode t/s Default (--tune skip) 219-240 15.1-15.9 Quick tuner 237-239 14.5-17.2 (tuner said "already fastest") pcie-frac 0.35 239 14.8-16.0 kv-resident 20480 + vram-reserve-mib 900 227-229 14.8-15.5 resident-budget-gib 64 232-242 17.3-19.8 With the final config at 128K context, 58K and 61K-token prompts gave 234 and 236 t/s prefill (about 4 minutes each) and 17.6-17.9 t/s decode. No OOM, swap went from 0.2 to 0.3 GB, and available RAM bottomed out around 3.4 GB. I did not test beyond 61K tokens or with a desktop session busy. What mattered --set resident-budget-gib=64 was the only change that helped (about +20% decode). It makes the engine keep the experts in its own RAM tier. Without it they sat in the OS page cache, so my RAM monitor showed under 5 GB "used" even though the machine was using about 28 GB of cache. Prefill stayed at about 235-242 t/s in every config I tried, so none of the settings below moved it. pcie-frac 0.35, a larger kv-resident, and a bigger VRAM reserve did nothing measurable here. The OP's pcie-frac was calibrated for their own machine. The quick tuner changed nothing. MTP: --mtp off is what I ran, and I tested real MTP to check. Careful: on my install --mtp on silently did nothing. The saved config said mtp: true, but the draft layer was never downloaded or passed to the engine ("+0 MiB for the draft head" in the log). I had to build it by hand (mtp_fetch.py, mtp_pack.py, mtp_rt.py from the Strata source) and pass --set mtp=. With the draft layer really loaded (820 MiB VRAM, expert cache 2225 to 1179 slots): decode 15.6 / 18.4 / 16.9 t/s (no gain vs 17.2), prefill 210 vs 232 t/s (about 10% lower). So on 8 GB VRAM it doesn't pay off. 3 runs, 8K context only. I did not try --adapt-*, --spec-min-p, or the thorough tuner. SteamOS tips Anything started from an SSH session dies when the session ends. Use systemd-run --user --property=KillMode=process --property=RemainAfterExit=yes strata launch .... --no-tui shows no download progress, so check the file size instead. Check the generated config under ~/.local/state/strata/launch/ to see which --set values were really applied. Unknown names only warn. Command for the final setup (I saved it as a preset with --save-preset): strata launch --model gyro-s --family gyro --context 128k --kv q4_0 --vision off --mtp off --set resident-budget-gib=64 --access lan --port 8080 --no-tui --yes --restart Caveats Decode varies a lot between runs with speculative draft acceptance, so treat the numbers as ranges. With resident-budget-gib=64 only about 3.5-4 GB of RAM stays available. My machine rebooted once during testing with a desktop session logged in (cause not confirmed), so close other apps. Single user, one request at a time, 3 runs per config. I could not find any ready-made Strata config for 8 GB VRAM with 16-32 GB RAM, so if you have one, please share it. All scripts are on the Github repo.   submitted by   /u/Accurate-Door3692 [link]   [comments]