An excerpt from their blog post: 1-bit Bonsai 8B implements a proprietary 1-bit model design across the entire network: embeddings, attention layers, MLP layers, and the LM head are all 1-bit. There are no higher-precision escape hatches. It is a true 1-bit model, end to end, across 8.2 billion parameters. Despite being 14x smaller than the 8B (16-bit) full-precision models in its parameter-count class, it performs competitively on standard benchmarks while operating at radically higher efficiency. Read the full blog post here: PrismML — Announcing 1-bit Bonsai: The First Commercially Viable 1-bit LLMs   submitted by   /u/elemental-mind [link]   [comments]