I've been building Swaraloom, an open-source, privacy-focused AI voice agent designed to run locally on Apple Silicon Macs. The idea was to build a voice assistant without relying on cloud AI APIs for its main pipeline. I also wanted to understand the engineering behind the complete system, from speech recognition to speech generation. What it currently includes: Local speech-to-text with Moonshine and Whisper options A local Qwen language model with tool calling Streaming text-to-speech using Kokoro Voice cloning with an explicit consent flow Encrypted conversation memory Sandboxed tools for files and approved commands Hindi speech generation and a web interface A few measured results on an M3 Pro with 18 GB RAM: The main voice pipeline uses approximately 5.5 GiB of resident memory under the measured configuration. The first sentence can begin speaking in about 280 ms in the isolated TTS streaming benchmark. The complete conversational loop is still slower than I'd like, so reducing response latency remains a work in progress. It's not perfect. Hindi speech quality needs more evaluation, the real-time loop has latency limitations, and some desktop packaging work remains. I'm sharing the code because I'd appreciate feedback from people who build local AI systems, speech models, and voice assistants. Repository: https://github.com/atulsinghhhh/Swaraloom I'd especially appreciate feedback on the architecture, the model choices, and what you'd improve first.   submitted by   /u/Original_Clock_9643 [link]   [comments]