A 120B LLM Without At Least 60GB RAM Is Impossible To Run, But One Individual Defied All Odds By Combining A Phone, Three Macs And Two Windows PCs To Make it Work

Wait 5 sec.

https://wccftech.com/120b-llm-phone-macs-windows-pcs-distributed-setup/ Equipped with a Galaxy S24+, an RTX 3060 mini PC with 12GB VRAM, a Windows notebook with a paltry 12GB memory, and a combination of an Intel-powered MacBook Pro, Mac mini, and an M3 MacBook Pro, Redditor “Medicine_Blogscanner” successfully managed to run the 4-bit quantized version of gpt-oss-120b, which requires 60GB of memory to run at a minimum.   submitted by   /u/ANR2ME [link]   [comments]