Hey all, I just picked up an EVGA RTX 3090 FTW3 (24GB) for $1,100 from a friend who is switching to an RTX 5070 Ti for pure gaming. It has low hours on it: no mining, no AI compute, just casual desktop use. He bought it for the prestige and never really pushed it. Right now, everything local runs directly on my daily gaming rig with a single RTX 4070 (12GB): LLM / assistant: Ling-3.0-tiny Voice (STT/TTS): Omnivoice (Design + Cloning) plus Parakeet 12GB does not stretch far across all of that once I want a usable context window. On top of that, killing background inference pipelines every time I launch a game got old fast. So the 3090 is going into my server box instead: CPU: Ryzen 5 5600X (6c/12t) Board: ASRock X570 Phantom Gaming 4 RAM: 64GB DDR4-3200 OS: Ubuntu Server 26.04 LTS (headless) Role: API backend for my assistant pipeline and coding inference over LAN, running alongside existing home lab services (Jellyfin, AdGuard, etc.) Three things I would really appreciate input on: 1. Coding models on 24GB (retro RE and systems work) What are you running right now for local coding, disassembly analysis, and refactoring at 32k to 128k+ context? Most of my work involves reverse engineering older DOS and 16-bit Windows titles, which means a lot of real-mode x86 assembly, C/C++, Ghidra/IDA pseudo-code, and legacy API mappings. I am looking for weights that sit comfortably in roughly 20 to 22GB with room left for the KV cache, and that do not hallucinate modern calling conventions when working with legacy code. 2. Headless serving and model swapping The box needs to serve endpoints to my desktop over LAN while its existing home lab services keep running. Which backend or runner are you using to handle on-demand model swaps without starving everything else on the host? Or is it cleaner to handle this manually via SSH scripts or systemd units to toggle services as needed? 3. FTW3 thermals and power management on Linux The FTW3 has a known reputation for hot rear GDDR6X modules during long tensor loads, and this card will live in a server tower without EVGA Precision X1 around. What nvidia-smi power limits (-pl), headless undervolts, or fan curve setups (coolbits, nvidia-settings, custom systemd scripts, or daemon tools like LACT) have worked best for you to keep memory junction temps in check without tanking tokens per second? Any advice, config snippets, or gotchas from anyone running a headless 3090 would be awesome.   submitted by   /u/NighthawkXL [link]   [comments]