Hi LocalLlama! In the last months - as a hobby - I build Poldi - a personal, offline, fully local AI assistant/"companion" with speech as user interface. A hands-free (via wakeword) and a browser-based client (e.g. usable with your smartphone) are available to talk to the server via HTTP. Poldi is especially targetting GPU poor hardware. I made a video, you can go to the parts you are interested in via the links below the video (click the "..."): https://www.youtube.com/watch?v=8AQWnoBXt4U Features (links go to the relevant part of the video) Automatic chat resume. Internet search. Wikipedia search. Interrupting the LLM. "Infinite" chat. Chat history search. Extraction and injection of memories (right now, facts about the user, only). Support for English and German (other languages should be easy to add). Console mode. ... Technical details Poldi is mostly written in C#, JavaScript and C/C++. The server can run on Windows or Linux, the hands-free client is Linux-only (you can use a Raspberry Pi or something as a satellite). I am using a system with an Nvidia 5060 Ti and 16 GB VRAM, but you can get good results with 8 GB of VRAM, too. CPU-only usage is also possible, but some features may have to be deactivated, then (more details at GitHub). Stack STT: Whisper.cpp, via mt_stt. LLM inference: llama.cpp, via mt_llm. TTS: Piper, via mt_tts. It is released under GPLv3. Get all the details at GitHub: https://github.com/RhinoDevel/poldi An installation and configuration tutorial is also included in the read-me. Feel free to open an issue at GitHub, if problems arise! Have fun! Marcel   submitted by   /u/rhinodevil [link]   [comments]