Hi! I'm Shubham, creator of SmolChat. I would like to share some exciting updates on my project SmolChat, an Android app that let's you run GGUF models on-device, locally. The updates are present in the latest v17 release: SmolChat now supports an OpenAI-compatible HTTP server running over a foreground service that keeps the port alive even when the phone is locked. Check the video to see how it works. Support for global chat settings, which override the default chat settings (min-p, top-k, temperature etc.) for all newly created chats. This reduces the hassle of configuring each new chat with the options of your choice. Adding a copy button for code snippets rendered within the chat. Sync with upstream llama.cpp (till this commit) Fix the logic that measures tokens/sec for a LLM generated message. The time shown previously was also including prompt processing (pp) time. The new logic now only considered the token generation rate (tg). Models are downloaded directly by the app, instead of using the Android system's download service. GitHub: https://github.com/shubham0204/SmolChat-Android Play Store (the latest version is still in review): https://play.google.com/store/apps/details?id=io.shubham0204.smollmandroid&hl=en_IN Previous threads on SmolChat on r/LocalLLaMa: Available on Play Store announcement Initial Announcement   submitted by   /u/shubham0204_dev [link]   [comments]