I wanted to share my setup thats working really well for my fellow 16GB desktop suffrage crew. I am able to get 110k context with ~1.5GB to spare and around 55 t/s on my 5070 TI. First off I found this model https://huggingface.co/hitsfmdj/Qwen3.8-27B-GGUF-4.2BPW-16GB with custom weights that seems to be pretty good and only 11.8GB. However even with this model, I ran into issues with context windows greater than ~60k. Sometimes it would fit on the GPU and other times it didnt. It was a good size, but not enough for larger coding tasks and network automation stuff. So I setup my computer to utilize my iGPU on my 9800X3D as my main display driver instead of the 5070 TI 16GB. Xorg alone was eating up 600-800MB of VRAM. The more applications I opened, the more VRAM I used the more problems/slowness I had. With my motherboard MSI MAG x870e Tomahawk, I can enable Hybrid Graphics which allows me to set the primary display driver as the iGPU instead of the 5070 TI but still utilize the 5070, reserving the full 16GB of VRAM for llama and gaming. I tested playing my main game Overwatch on my main display at 1440P 144hz and the game ran just fine at my FPS cap of 300. Ive been very happy so far with this setup, the extra context size has been a real game changer for longer and more complex tasks, its fast enough to persuade me away from ChatGPT. I do also have a pretty decent overclock on my GPU. +350 Core, +4000 Mem, +350 Xbar and 350 watt bios, I have 32GB DDR5. =========================================================================================| | 0 N/A N/A 1638 G /usr/lib/xorg/Xorg 55MiB | | 0 N/A N/A 44165 C ...ma.cpp/build/bin/llama-server 14734MiB | +-----------------------------------------------------------------------------------------+ ExecStart=/home/jayecin/AI/llama.cpp/build/bin/llama-server \ --model /home/jayecin/AI/Models/Qwen3.8-27B-4.2BPW-16GB.gguf \ --alias Qwen3.8-27B \ --host 0.0.0.0 \ --port 8080 \ --ctx-size 110000 \ --fit on \ --flash-attn on \ --cache-type-k q8_0 \ --cache-type-v q4_0 \ --batch-size 256 \ --ubatch-size 256 \ --parallel 1 \ --reasoning-effort medium \ --reasoning-budget 4096 \ --no-reasoning-preserve \ --metrics Oct 30 18:25:11 9800X3D llama-server[44165]: 26.55.877.747 I slot print_timing: id 0 | task 3302 | n_gen = 3460, tg = 54.69 t/s, tg_3s = 54.09 t/s Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.317 I slot print_timing: id 0 | task 3302 | prompt eval time = 70.53 ms / 4 tokens ( 17.63 ms per token, 56.71 tokens per second) Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.319 I slot print_timing: id 0 | task 3302 | eval time = 63510.16 ms / 3474 tokens ( 18.29 ms per token, 54.68 tokens per second) Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.320 I slot print_timing: id 0 | task 3302 | total time = 63580.69 ms / 3478 tokens Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.320 I slot print_timing: id 0 | task 3302 | graphs reused = 6840 Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.321 I slot print_timing: id 0 | task 3302 | graph reallocs = 0 / 0 (same size) / (changed size) Oct 30 18:25:11 9800X3D llama-server[44165]: 26.56.136.582 I slot release: id 0 | task 3302 | stop processing: n_tokens = 16449, truncated = 0 Oct 30 17:58:52 9800X3D llama-server[44165]: 0.36.587.381 I slot print_timing: id 0 | task 3 | prompt processing, n_tokens = 6912, progress = 0.53, t = 3.10 s / 2227.71 tokens per second Oct 30 17:58:52 9800X3D llama-server[44165]: 0.36.707.961 I slot print_timing: id 0 | task 3 | prompt processing, n_tokens = 7168, progress = 0.55, t = 3.22 s / 2223.80 tokens per second Oct 30 17:58:52 9800X3D llama-server[44165]: 0.36.828.660 I slot print_timing: id 0 | task 3 | prompt processing, n_tokens = 7424, progress = 0.57, t = 3.34 s / 2220.08 tokens per second Oct 30 17:58:52 9800X3D llama-server[44165]: 0.36.949.495 I slot print_timing: id 0 | task 3 | prompt processing, n_tokens = 7680, progress = 0.59, t = 3.46 s / 2216.54 tokens per second   submitted by   /u/jayecin [link]   [comments]