TL;DR: Why are we trusting local LLMs if we can't observe what they are doing? With recent events, something has been made more clear: For some reason, LLMs are not as observable as they should be. This, notably, is not a failure with LLMs themselves. If you take a look at https://www.boozallen.com/expertise/cybersecurity/whats-in-americas-code.html that was published as far back as May, and of course discussion on this has been happening longer. But how do you observe a LLM? How do you know what it is doing? This isn't a new question, but I think it deserves to be raised with higher priority now, considering some of the latest compromises brought to light. I also see people jumping on deeply insecure harnesses that raises even more fundamental insecurities and problems with understanding what the LLM is doing, and there's no good reason for it. Worse, I see new publishers of LLMs popping up out of nowhere, promising the world. I guess that one isn't new, but what are you doing to confirm what a LLM is doing? https://rakuensoftware.com/blog/everything-crosses-one-transport is a deeper dive into how we approached it, and this is universally applicable to any local LLM or harness that uses an Anthropic or OpenAI API. This approach is usable by any harness, and the code is open source. Take the core harness and run with it, or build your own. But being able to observe your own LLMs and what they do shouldn't be an afterthought at this point. Anything that is used should be building these core concepts by now. Key to any current harness design should have observability as a first-class citizen, not an afterthought.   submitted by   /u/KitchenAmoeba4438 [link]   [comments]