Meta just released the weights for Muse Spark Open, a multimodal large language model with a mixture-of-experts architecture. It has over 800 billion parameters in total, with 40 billion active per token, and it can run on the latest Mac Studios.Meta worked closely with Apple to make that happen specifically on the M5 Ultra Mac Studio with the support of MLX, the Apple-led, open-source machine-learning framework. The result is a frontier-class model that can be run locally on desktop hardware.Apple’s Machine Learning Research blog put the M5 Ultra through its paces standalone and connected with other M5 Ultra Studios, and the results are impressive. If you’re at all interested in running LLMs locally, it’s worth checking out all the details, but a few things jumped out at me:The M5 Ultra’s time to first token exceeds the M3 Ultra by 4.6x thanks to the M5 Ultra’s Neural Accelerators.Tokens generated by the M5 Ultra beat the M4 Ultra by 1.6x.However, thanks to MLX’s support of RDMA over Thunderbolt 5, big models like Muse Spark Open can be split across multiple M5 Ultra Mac Studios accelerating tasks further. For very large contexts that means a time to first token that’s up to 1.8x faster, but it’s also what lets the model fit onto desktop hardware because even the 4-bit version of Muse Spark Open requires 493GB of memory.Some of Apple’s testing was done with the 512GB model of the M5 Ultra Mac Studio, which is slated for release at the end of the year. Apple hasn’t said how much the new model will cost, but for context, the 256GB model with 2TB of storage will set you back over $11,000. So the 512GB model isn’t your typical consumer purchase, even more so since running Muse Spark Open requires at least two of them.However, that doesn’t make this any less of an accomplishment. Muse Spark Open is at the bleeding edge of models that can be run locally on computers that can sit on your desk and don’t need more power than can be delivered from a wall outlet. That’s made possible by the unique combination of Apple silicon and updates to MLX. Plus, with the pace of advancements in AI and Apple silicon hardware, what may only meet enterprise budgets today could become accessible to more consumers faster than most of us might expect.