I’ve recently just started tinkering with using local large language models, focusing on simple, low-dependency CLI setups. I ended up going down a bit of a rabbit hole: I wanted to see if I could build a functional model interaction REPL using exclusively standard command-line building blocks. I might be reinventing a very weird wheel here, but it turns out you can get surprisingly far using just standard text streams (stdin/stdout), pipes, and append-only logs. I tried to abide by the Unix philosophy, breaking the REPL into the composition of a few small, single-purpose program. Because the logical flow is just text streams fed through pipes, at any step you can inject tools to inspect or modify the data—like using grep to filter out strings before they hit the model, or pv to benchmark model throughput (not really a standard tool but was pretty useful in my experiments). A few architectural details I thought this crowd might appreciate: Zero heavy dependencies: No pip, npm, package managers, virtual environments, etc. . It just requires bash, jq, and curl to talk to the local model server. These should be available in most modern CLI environments. Transparent, file-based state: The agent's memory is just an append-only .jsonl file (like .bash_history). If you want to rewind the agent's memory, you just run head on the log to drop the last few lines. Standard exit codes for control flow: Tool execution is handled by checking standard Unix exit codes within a basic bash while loop. I'm sure there are scaling limits to doing this all in shell, and I'm still figuring out the most elegant way to handle some of the edge cases, particularly around tool calling - but those appear to mostly be limitations of the underlying models. Nevertheless it's been a really fun experiment in stripping out bloat. I put the code up here if anyone wants to poke around: https://github.com/cloudkj/llayer Would love to hear if anyone else has tried orchestrating things this way, or if you spot any glaring anti-patterns in how I've structured the pipes!   submitted by   /u/cloud_kj [link]   [comments]