Hi! I wanted to share that we just got NVIDIA's latest Nemotron Bolt model running at 27.7 tok/s (compared to 15 tok/s on the official release with Llama CPP) on Jetson Orin Nano Super (1.8x faster)! It can program games on a model running completely on a cheap edge device! We have our own Jetson devices serving that model on our website, if you want to get a feel for how the model runs on the device here or run it locally as a TensorRT engine with our own variant of TensorRT edge LLM supporting int4 quantization: https://embedl.com/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16/orin-nano-super?tab=run   submitted by   /u/Any_Frame9721 [link]   [comments]