Hey everyone! I recently got a new Mac Mini M6 with 16GB unified memory and wanted to get into running LLMs locally. I'm relatively new to the local LLM ecosystem, so I'm trying to figure out the best way to get started and make the most out of this machine. I have a few questions: Models What LLMs can I realistically run on 16GB unified memory? Which models would you recommend for general-purpose use, coding, reasoning and agentic workflows? Would 7B/8B models be the sweet spot, or can I comfortably experiment with anything larger using quantization? Inference engines / Tools I've come across a few options: llama.cpp LM Studio Ollama MLX / MLX-LM Which would you recommend for a Mac? Is there a significant performance difference between them? Should I stick with one or use different tools for different purposes? Fine-tuning I'm also interested in experimenting with fine-tuning models locally. Is fine-tuning even practical with 16GB unified memory? Can I experiment with LoRA or QLoRA on this machine? Would MLX-LM be the best starting point for fine-tuning on Apple Silicon? Are there any smaller models or specific workflows you'd recommend learning with? Practical use cases My primary interests are: Running local coding assistants. Building AI agents and experimenting with agentic workflows. Cybersecurity-related tasks and automation. Experimenting with RAG and custom datasets. Fine-tuning smaller models for specific tasks. I'm not expecting to run massive models on this machine. I just want to understand its limits and get the best possible performance out of the hardware. For anyone running local LLMs on Apple Silicon, particularly with 16GB RAM: What does your setup look like, which models are you running, and what would you recommend to someone starting out? Any advice, benchmarks, resources or recommendations would be appreciated! Thanks!   submitted by   /u/Clean_Initial_9618 [link]   [comments]