I've been reading the turn logs from PokeBench. My benchmark where vision models play Pokemon Red. Grok 4.20 non-reasoning left the house at turn 36 and didn't get a starter until turn 755. The whole time its notes say things like "We've been oscillating uselessly for 90+ turns" and later "The persistent loop of walking into walls and Mom's dialogue has continued for hundreds of turns." Then it walks into the wall again. So it knows it's stuck and still can't get out. Grok 4.6 beat Brock in 316 turns for about $10, while 4.20 burned $30 and barely got its starter. Why do you think a model can describe its own loop correctly and still not break it? Vision problem or planning problem? pokebench.tv   submitted by   /u/VibeCodyH [link]   [comments]