Hello everyone! I don’t follow the market super closely, but I can see how quickly everything is getting more expensive—and apparently this is going to last a few more years. RAM, GPUs, and even SSDs and HDDs have skyrocketed in price :D I’m looking for something to play around with in LLM, mainly for inferencing larger models (in the 30B–70B range, maybe even bigger sometimes) and building agents (for my own use). I want to utilize agents, do RAG and sometimes, rarely maybe also fine-tune. My budget is small ($4k), and where I live in eco-Europe, prices are much higher! (we care about ecology LOL). I was thinking about a mini PC or a laptop with an AMD 395+ and 128 GB of RAM. For that price I still can purchase something, but the choice is getting smaller every week :) Is a AMD 395+ AI with 128 GB RAM still good choice for local LLMs? Is it good for my purposes? I want to learn & use LLM locally (for many reasons). I have heard about RTX Spark N1X laptops, but they are now about 6.5-7k dollars here, so I dont even consider them. Can I say that this platform (AMD 395+ AI with 128 GB of RAM), where the model has 96 GB of RAM at its disposal, is something that could replace the RTX 6000 Blackwell, which also has 96 GB of RAM? I realize that, due to the difference in memory bandwidth alone, it will be at least 6–7 times slower. I’d like to have Hermes-type agents or similar ones locally, as well as my own agents for various tasks. For example, I’m interested in extracting data from documents (e.g., PDF files) and recognizing objects in photos, but I’m also interested in general-purpose agents. Is the platform I’m asking about a good choice, or is there something more interesting? I was thinking about an ASUS PX13 laptop or one of the mini PCs with a similar configuration. Thanks for your suggestions!   submitted by   /u/Repsol_Honda_PL [link]   [comments]