From about February 2026 to July 2026, I spent an immense amount of time building a legal harness to make Claude more predictable. It worked well. I tried to build the same thing that ran local models. That did not work. And then Opus improved so much that it would often skip my harness without me noticing. It still has a harness with hooks, but it's looser now. I bought lots of hardware to run local models during that time, repeatedly trying to vibecode a local model legal harness. Many failures, many lessons. Anyway, fast forward to the last three weeks and several things have happened: DeepSeek Flash 1017 was released, and it's the first model that is remotely similar to Opus 4.5/4.6. Opus 5.5 and Fable 5.2 are able to properly launch and configure DeepSeek to run at fast speeds, large context and high precision on my hardware. Optimizations, custom kernels, requants. Opus 5.5 (and my prompting skill) are now good enough to make my Claude harness into a DeepSeek harness. It's not as good as frontier, but it all works, right now. 100% local. Actual productivity gains, actual frontier tasks being replaced. Chat interface with DeepSeek works like Claude's did. And if I can brag on my stacks and hardware: I have 2 DGX Sparks that can run 3-4 concurrent DeepSeek sessions as agents (1M context). I have DeepSeek at nearly NVFP4 quality running on 8 V100s, 500K context, at 100+ tok/s decode, and about 160 tok/s concurrent. I have 8 more V100s and a second Rome server with 512 GB of 3200 MHz ECC RAM, so I'll be able to run another high precision DeepSeek as an additional agent or for parallel tasks. What I'm really shocked by is how well Fable builds custom Volta kernels. And how it can requant my experts to full NVFP4 precision based on the actual experts that activate when my legal harness fires. It's a real, optimized, production grade, workflow replacing local setup. Not a 3D pagoda or a bad video game replica. An actual enterprise solution. One that exists nowhere on the commercial market. Legal vibecoding is real folks. I'm AI native. The new V100 DeepSeek repo (optimized for quality and then usable speed) with real full context measures and all the optimizations (and failures) is public here: https://github.com/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-NVFP4-8xV100-SXM2-PP2xTP4 Twitter: nicothegreco Claude Code ran the benchmarks and crunched the numbers, which are real and in depth, below. What this build is My last V100 build (Volta sPLIt, part 4) ran every expert of the model from a 2-bit record. Fast, but 2-bit. This one is called Volta Max uPLIft. It keeps the 2-bit record of every routed expert on the cards, then swaps the experts that get used the most up to NVIDIA's NVFP4 weights, one at a time, until the cards are full. "Used the most" was counted on a mix of four kinds of text. One is my own legal harness traffic, which is private and not published. The other three are public prompt sets in the repo. 5,949 of the 15,360 routed experts sit at exactly NVFP4 precision on the default I serve about 78 to 85 % of decode routes land on those NVFP4 experts (predicted for the two-slot default, 80 to 86 % measured with one slot) same next token as NVIDIA's full NVFP4 pack on 95.1 % of compared tokens. The 2-bit build gets 92.2 % KL divergence from the NVFP4 reference is 0.079 nats against 0.196 for the 2-bit build multiple choice answers that change against the reference: 1.6 % against 15.4 % in decode every routed expert is in VRAM. Nothing is read over PCIe Speed (measured 2026-11-12 on eight V100-SXM2-32GB, graphics clock capped at 1,522 MHz) concurrency test, one request at a time: 118.9 tok/s two requests at once: 159.0 tok/s total (1.34x) three at once on the three-slot option: 192.3 tok/s total (1.62x). I run two slots short sets, one at a time: 123.2 tok/s general, 97.3 tok/s legal-style code: 150 to 180 tok/s from a 1K prompt up to a 256K prompt prompt reading at 256K: 2,329 to 2,430 tok/s, first token in 108.4 s context: 524,288 tokens per request What did not work, or is not good yet It is slower than my own 2-bit build, in decode and in prompt reading. That is the trade for the precision, and the repo says so in its first paragraph. The faster prompt reading path reads the less-used experts from their 2-bit records. Its effect on quality is not measured yet. A custom decode kernel for the hot path got closed with no change. The headroom I thought was there turned out to be smaller than an eight-card run can even see. A prompt-reading kernel that loads 64 rows at a time gave the same bits and ran slower. Not shipped. Send a short request while a 209K prompt is being read and it starts in about 7 s, then crawls at about 9.6 tok/s until the read ends. The hot set is fixed. It is picked once. Text that looks nothing like what I counted will hit the 2-bit experts more often. Fast prompt reading stopped the host or a card three times out of three. I blamed PCIe. After I changed the power delivery (I think it was one bad PSU) it read nine of nine long chats clean. The harness The harness itself is not public. My practice is in it. What I can say: it is the same idea as my Claude setup. Skills, hooks, my drafting engines, my RAG stores, tool calls, and a chat window that behaves the way Claude's did. A small router sends each request to the V100 box or to the Sparks. Everything the local model drafts is marked for my review before it goes anywhere. Nothing leaves the building. Credit where it is due The hard Volta work underneath is 1Cat-vLLM's DeepSeek-V4.1 package. The 2-bit records are ISTA-DASLab's. The NVFP4 weights are NVIDIA's pack. The Sparks run on somebody else's engine entirely. What is mine is the split across two NVLink boards, the hot set, the kernels on top, and every change measured A/B on this machine with the losers written down. Disclosure: this is my own project and my own repo. Nothing is for sale. The code, the kernels, the measurements and the failures are all in there under Apache-2.0, and the weights are one download on Hugging Face: https://huggingface.co/ProprietaryLegal/Volta-Max-uPLIft-FP4-DeepSeek-V4.1-Flash-1017-hot_iq-8xV100-SXM2-PP2xTP4 Earlier build, for the people who want speed over precision: https://github.com/ProprietaryLegal/Volta-sPLIt-DeepSeek-V4.1-Flash-1017-8xV100-SXM2-PP2xTP4 Ask me anything. Roasting is also fine. Photos: four of the eight V100s on their NVLink board, and my extremely professional setup (PEX switch cards, SlimSAS spaghetti, and the 3090 that just moved in).   submitted by   /u/TumbleweedNew6515 [link]   [comments]