Disclosure first: I'm the author (Mahmoud, ghraibeh on GitHub). I'm not affiliated with TypeSafe AI. I know this sub is tired of Jev hype, so I'll lead with the limits. What it is: semantic caching plus nearest-neighbour voting. It isn't understanding and it isn't new. It's an MIT Python library and runs on CPU only. Inputs are embedded locally with bge-small-en-v1.5. If the nearest stored input is at least 0.90 similar and the 5 neighbours agree, it answers locally. Otherwise it asks Jev and remembers the answer. Benchmark caveat: these numbers are on BANKING77, with the dataset's gold labels standing in for Jev. That makes them a best case. No live Jev benchmark has been run yet. Warm (pre-filled): 87% of calls saved, 97.6% of local answers correct Cold (empty): 53% saved, 97.4% correct Why use it if Jev is cheap? Not for money. Local answers take about 30–50 ms vs 250–550 ms for Jev, you hit the rate limit less, and repeated inputs stay on your machine. Repo: https://github.com/ghraibeh/jev-saver Demo: https://g-connect.space/jev-saver/ Criticism welcome.   submitted by   /u/Aggressive-East-2815 [link]   [comments]