Today I put my first two models up on Hugging Face I used Qwen4 to do direct weight editing to fix reasoning looping by up to 97% reduction on 1.7b and a bit less on the full 4b. The problem: the "but wait…" spiral. Spark-X2.6 has a nasty failure mode where it reaches the correct answer… and then refuses to commit. It spirals into re-decision — "but wait, should I…", re-deriving the same conclusion in fresh wording, re-enumerating its tools, never acting or stopping. It's a semantic loop, so normal repetition metrics barely see it. In an agent setting it's fatal: the model knows the answer and just won't pull the trigger.The fix: surgery, not training. Instead of fine-tuning or messing with sampling, the fix is a no-training surgical weight edit — literally one row of the output head 4B: 2,560 weights changed ≈ 6×10⁻⁷ of the model. 1.7B: 2,048 weights (it has an untied head, so the input embeddings are untouched). The diagnosis came from directly watching activations at the exact moment the model reaches a conclusion but before it spirals: closure () was essentially absent (rank ~54, P≈7×10⁻⁶) while restart vocabulary dominated. Because the head reads out as logit(t) = emb[t] · h, that closure deficit is a head-readout gap. So we solved for the minimum-norm row delta that raises P(close) by ~14 nats at the conclusion states — dequantize that one row, add the delta, requantize, splice it back. Every other byte is bit-identical. The edit is conclusion-selective: it doesn't truncate legitimate reasoning.Results.Loop rate (controlled branch-point benchmark, 12 seeds/prompt) ModelBeforeAfterReduction4B0.7500.31258.4%1.7B0.8750.06292.9%And it doesn't cost general capability — same suites, before vs after Suite4B Δ1.7B ΔARC-Challenge−0.040+0.020WinoGrande+0.030+0.020PIQA−0.010+0.010HellaSwag0.0000.000MMLU+0.0220.000GSM8K−0.017+0.233Mean−0.002+0.047The 4B is basically neutral overall; the 1.7B actually gets smarter — GSM8K math more than doubles (0.200 → 0.433). Fixing the loop didn't break the brain.Honest caveat: this fixes the thinking loop. Broader agent tool looping can still happen on some tasks — that's a separate beast I'm still characterizing.Links (GGUF, delta + offline reproduce script included): 4B: Spark-X2.6-4B-heretic-noloop-headrow-GGUF · huggingface.co 4b heretic Spark Model 1.7B: Spark-X2.6-1.7B-noloop-headrow-GGUF · huggingface.co 1.7b 97% reduction + math increase. Both cards include the exact delta and a CPU-only script that reproduces the checkpoint bit-for-bit from the upstream quant. Full credit to XHToken (base), the heretic finetune, and the upstream quantizers — this is a derivative that adds only the head-row edit. Credit where it's due: the diagnosis, the activation surgery, and the benchmark rig were done by Qwen4, who did the genuinely hard part — finding the causal lever in the weights instead of guessing at sampling knobs. I just pointed it at the problem and reviewed the work.   submitted by   /u/opUserZero [link]   [comments]