Hello again! I'm back with another report. This time it's on temperatures. Also, [CE Robot] = Claude Opus 5.5 Copy Editor where applicable (per line.) Otherwise, I wrote it. TLDR While running DeepSeek V4.1-Flash, the GX10s can run as hot as 82°C without any throttling behavior. The most you will see is a max clock reduction of 3%, which is mostly to manage power leakage. There is no meaningful performance difference between running the GX10s at their coolest or hottest, about 3%, effectively noise. Intro I came across a post or two on DGX_Spark (friendly people I swear /s) and was curious about two posts with different hodgepodge cooling scenarios to keep their GX10s/Sparks from overheating. I have experience with NVIDIA's gold 4TB Spark and the ASUS 1TB GX10s and besides a few launch issues, haven't really noticed any throttling. I still had the scripts that ran benchmarks on DeepSeek V4.1-Flash. I decided to modify the scripts and query GPU temperatures. To keep some semblance of sanity (still unemployed, was ousted, etc.) and having a 3D printer I bought that was collecting dust, I decided to work on an enclosure for my GX10s to improve airflow. Below are the different test cases. How to read the tables: Min is the idle temperature at the start of each run. Under load averages only count samples where the GPU was drawing more than 30 W. GX10-0 is the head node. [CE Robot] Original Setup, IKEA Kallax https://preview.redd.it/fij2uhn88yxh1.jpeg?width=3725&format=pjpg&auto=webp&s=541692407f5bd91ff92cb1131c28c7bc2c16db03 Temperatures GPU temp GX10-0 GX10-1 GX10-2 GX10-3 Min 46°C 47°C 47°C 45°C Peak 75°C 82°C 78°C 74°C Under load Avg temp Avg GPU power Avg clock 69.4°C 50.9 W 2466 MHz Throughput Prefill 8K Prefill 250K Decode (median) 4,255 t/s 4,268 t/s 62.2 t/s IKEA Kallax, Fanned Out + Fans Had to add separators because of the strong magnets that keep them nicely together. Temperatures GX10-0 GX10-1 GX10-2 GX10-3 Min 47°C 50°C 50°C Peak 71°C 77°C 76°C Under load Avg temp Avg GPU power Avg clock 69.0°C 51.9 W 2466 MHz Throughput Prefill 8K Prefill 250K Decode (median) 5,067 t/s 4,353 t/s 75.8 t/s V1 Structure https://preview.redd.it/fkfnihry9yxh1.jpeg?width=5712&format=pjpg&auto=webp&s=103667df8f0d8b4bd7bcec5d1725679cfe080115 There were issues with this design is the bottom of the frame formed a lip that restricted airflow to the bottom GX10s. The hole on the bottom just past the lip also trapped air that warmed up before being sucked up by the GX10. The numbers reported for the two GX10s on the bottom were worse than the original Kallax setup and at best, with fans, just equal to the Kallax setup. I kept these numbers in to show that even with elevated temperatures, there was no sign of throttling. The 3% reduction in clock was to control power leakage. V1 without Fans Temperatures GX10-0 GX10-1 GX10-2 GX10-3 Min 51°C 44°C 51°C Peak 80°C 74°C 80°C Under load Avg temp Avg GPU power Avg clock 69.5°C 51.4 W 2466 MHz Throughput Prefill 8K Prefill 250K Decode (median) 4,888 t/s 4,264 t/s 71.7 t/s V1 with Fans Temperatures GX10-0 GX10-1 GX10-2 GX10-3 Min 43°C 43°C 45°C Peak 75°C 73°C 79°C Under load Avg temp Avg GPU power Avg clock 68.0°C 51.7 W 2469 MHz Throughput Prefill 8K Prefill 250K Decode (median) 5,128 t/s 4,340 t/s 70.0 t/s V3 Structure I had an idea of making an air duct. Looking at ASUS's website, I found the 3D model of the GX10 and this allowed me to build a better enclosure to fit the GX10. Looking at the GX10 underside and grill for a while, I got the idea of an "open-faced air duct." I looked at some examples of air ducts I've seen over the years (Pentium 4, I'm old) and gimmicky air duct cards that plug into PCI slots with fans to "distribute air"; these helped me not make the same mistakes. Asus Engineering Model If you look at the underside of the GX10, it's very flat and the grills go from front to just past the middle threshold. For this open-faced air duct, I would make a half-duct with a curve at the end that meets the GX10's backside with just a small edge of space. When the GX10 is seated on top, that would close our air duct. One thing to note is that the GX10 fan can only do so much. Completely enclosing the space underneath would make it hard for air to go through with no fan. That's why the sides are open when not using an extra fan: the open-faced duct makes the GX10's internal fan more efficient at pulling air. V3 with Fans https://preview.redd.it/lmox4ef7ayxh1.jpeg?width=4284&format=pjpg&auto=webp&s=f2d4e9dbfbbff99baa2226da091fb2f5b90fc7e4 Temperatures GX10-0 GX10-1 GX10-2 GX10-3 Min 44°C 43°C 43°C Peak 73°C 72°C 72°C Under load Avg temp Avg GPU power Avg clock 65.1°C 50.0 W 2473 MHz Throughput Prefill 8K Prefill 250K Decode (median) 4,881 t/s 4,225 t/s 71.5 t/s V3 without Fans https://preview.redd.it/636dkthcayxh1.jpeg?width=4284&format=pjpg&auto=webp&s=81dd556d74e38fc1181be344ce463aaa1be996a6 Temperatures GX10-0 GX10-1 GX10-2 GX10-3 Min 45°C 45°C 43°C Peak 73°C 73°C 74°C Under load Avg temp Avg GPU power Avg clock 65.3°C 49.9 W 2473 MHz Throughput Prefill 8K Prefill 250K Decode (median) 4,821 t/s 4,244 t/s 64.4 t/s Putting It All Together Temperatures across all setups Setup Avg temp under load Hottest GX10 avg Spread between GX10s Hottest peak Original (Kallax) 69.4°C 72.2°C 5.3°C 82°C Kallax, Fanned Out + Fans 69.0°C 70.8°C 5.1°C 77°C V1 without Fans 69.5°C 72.5°C 6.4°C 80°C V1 with Fans 68.0°C 70.4°C 4.4°C 79°C V3 with Fans 65.1°C 65.3°C 0.3°C 73°C V3 without Fans 65.3°C 65.8°C 1.3°C 74°C [CE Robot] suggested the "Spread between GX10s" column. Spread is the difference between the hottest and coolest GX10's average under load. [CE Robot] Does it throttle? Clock vs temperature on the hottest GX10 (GX10-1, Original Kallax) GPU temp Avg clock 53 to 60°C 2515 to 2525 MHz 62 to 68°C 2473 to 2502 MHz 70 to 79°C 2456 to 2493 MHz 80 to 82°C 2450 to 2455 MHz Does cooling change performance? Setup Avg clock Prefill 250K Original (Kallax) 2466 MHz 4,268 t/s Kallax, Fanned Out + Fans 2466 MHz 4,353 t/s V1 without Fans 2466 MHz 4,264 t/s V1 with Fans 2469 MHz 4,340 t/s V3 with Fans 2473 MHz 4,225 t/s V3 without Fans 2473 MHz 4,244 t/s Conclusion No throttling, even at 82°C. The clock steps down gradually as the GPU heats up, about 6.5 MHz at a time, from roughly 2525 MHz at 53°C to 2450 MHz at 82°C. That's about 3%, and it's the boost managing leakage power, not the GPU overheating. There's no sudden drop anywhere. V3 is the coolest setup. Compared with the original Kallax setup, the hottest GX10 dropped from 72.2°C to about 65.5°C on average, and the hottest peak from 82°C to 73 to 74°C. All four GX10s now run within about a degree of each other, where the Kallax had a 5°C spread. [CE Robot] Fans don't matter with V3. With and without fans, V3 is within 0.2°C on average and 1°C at peak, which is noise (about ±1.5°C). The open-faced duct lets each GX10's own fan do the work with no help from the external fan setup. Fans helped the older setups a little. Adding fans to the Kallax or V1 took 1 to 3°C off the hottest GX10, but neither came close to V3. [CE Robot] Cooler doesn't mean faster. Prefill at 250K stayed between 4,225 and 4,353 t/s across every setup (about 3%), and the coolest setup was actually at the low end. Average clocks moved by less than 0.3%. These GX10s ran the same speed hot or cool. [CE Robot] Overall I am very happy with the final product of my 3D printing project. V2 was never printed. It was going to be an adapter to add to the bottom of V1 to bring the GX10 higher and make a flat surface for better airflow. I mocked it up by propping the bottom GX10s up and forward so the grills cleared the lip (see "V2 mock-up" in the appendix). [CE Robot] The idea was scrapped because there were other issues with V1 that couldn't be fixed like the vertical spacing distribution. In addition, V1 was bulky and just had so much wasted space. I wanted a stackable design that can fit 4 GX10s stacked in the Kallax shelf without taking too much space. This whole 3D printer venture started when I tried some of the models available online specifically for the GX10. Most were awful and had fitting issues, and from the comments on each model, I wasn't alone. The other designs that added a fan attachment underneath felt so bulky. I just knew there was a better design without all the bulk. Furthermore, the open-faced air duct design makes the internal fan work better, making the extra external fans unnecessary. I believe there are some improvements that can still be made to V3. Currently it takes about 4 hours to print and it's very dense. I believe there is room for reducing the total mass while maintaining a strong foundation which would mean less filament and print time. For the temperature numbers, I can say I am quite impressed with the cooling on the GX10s. I'm really happy with the improvements in temperatures alone from the open-faced air duct design. I did not expect that outcome. Caveats These are GPU temperatures only. Room temperature stayed between 72 and 74°F (22 to 23°C). Ran at night with windows open and generally below 68°F outside. Decode uses speculative decoding, which swings about ±10 t/s between runs, so I left decode out of the comparison. [CE Robot] Each run is a single 8-minute sweep from 1K to 250K tokens with the model already loaded. Sustained multi-hour loads could settle a bit higher. Appendix: Every Run This includes a few extra runs that didn't get their own section above. Runs are in the order they were tested. [CE Robot] Dates provided for San Jose, CA weather lookup if anyone wants those data points. Temperatures (°C, Min / Peak per GX10) Setup Date GX10-0 GX10-1 GX10-2 GX10-3 Avg under load Original (Kallax) Oct 15 46 / 75 47 / 82 47 / 78 45 / 74 69.4 Kallax, re-run Oct 22 47 / 79 48 / 80 48 / 81 46 / 75 70.2 Kallax, Fanned Out + Fans Oct 22 47 / 71 50 / 77 50 / 76 48 / 75 69.0 Kallax, Spaced + 3 Fans Oct 22 46 / 75 46 / 75 45 / 76 45 / 74 67.7 V1 without Fans Oct 23 51 / 80 44 / 74 51 / 80 45 / 73 69.5 V1, tweaked Oct 24 48 / 79 45 / 76 51 / 79 46 / 74 69.7 V1 with Fans Oct 24 43 / 75 43 / 73 45 / 79 45 / 76 68.0 V1 with Fans, bottom raised (V2 mock-up) Oct 24 44 / 74 44 / 75 45 / 76 45 / 75 67.4 V3 with Fans Oct 26 44 / 73 43 / 72 43 / 72 44 / 71 65.1 V3 without Fans Oct 26 45 / 73 45 / 73 43 / 74 44 / 71 65.3 Power, clocks and throughput Setup Avg GPU power Avg clock Prefill 8K Prefill 250K Decode (median) Original (Kallax) 50.9 W 2466 MHz 4,255 t/s 4,268 t/s 62.2 t/s Kallax, re-run 51.3 W 2464 MHz 4,472 t/s 4,260 t/s 63.3 t/s Kallax, Fanned Out + Fans 51.9 W 2466 MHz 5,067 t/s 4,353 t/s 75.8 t/s Kallax, Spaced + 3 Fans 51.0 W 2468 MHz 4,905 t/s 4,280 t/s 69.8 t/s V1 without Fans 51.4 W 2466 MHz 4,888 t/s 4,264 t/s 71.7 t/s V1, tweaked 51.9 W 2465 MHz 5,089 t/s 4,344 t/s 64.5 t/s V1 with Fans 51.7 W 2469 MHz 5,128 t/s 4,340 t/s 70.0 t/s V1 with Fans, bottom raised (V2 mock-up) 51.0 W 2470 MHz 5,103 t/s 4,346 t/s 68.4 t/s V3 with Fans 50.0 W 2473 MHz 4,881 t/s 4,225 t/s 71.5 t/s V3 without Fans 49.9 W 2473 MHz 4,821 t/s 4,244 t/s 64.4 t/s P.S. How the heck did my whole post get the "spoiler" thing? P.S.S. Still fixing typos and bad word choices.   submitted by   /u/Aroochacha [link]   [comments]