So I created this simple test (for myself) to see how good the new models are in analytic output. This is not asking them to draw a SVG (which just tests how much SVG or HTML they saw) but creating a prompt that explains the simple syntax for a new graphic language with a few examples and the instruction what to draw. Then my program interpreting it visually. The model needs to understand far more than just how to draw something, it's more cold coding in a syntax they never been trained in. Qwen 3.8 and Ornith-1.5 beat everything else I could run on my putter. They both showed great understanding of the syntax (correct use of gradients) and a great deal of imagination. Shockingly these were among the smallest models tested. 27B - that is insane quality from it, IMHO. In fact they beat Haiku 5.5 in imagination. Gemma 4 -31b didn't get the syntax correctly how to draw text (of course!). It's imagination is Gemma too. Way too Gemma, hahaha. Now the behemoths of the not so distant past: The one that says deepseek v4 flash is actually Qwen3 coder (sorry) - it followed the instructions - not pretty results though. The one marked Deep Seek 4 is the Deepseek 4 flash. It's a timeline I guess, but it didn't figure out how to center text. Qwen 3.5 122b - followed instructions - lack of imagination (I won't hold the \n against it) Behemoth 123b can't really follow the instructions (missed some milestones) and has limited imagination. GLM 4.7 is still doing it - writing entire War and Peace volume about it. 66k token of thinking as of now and nowhere near the end. LOL. These are the big models for local inference and it shows that the smaller models now beat them. The big boys,Chat, Claude... as expected, understand the syntax and have various degrees of (wild) imagination. But they also try to being fancy and add stuff I didn't necessary asked for, such as slogans. Haiku did the job cleanly and precisely, Opus added slogans I didn't ask for - but they are on topic. Gemini 3.8 - tries so hard - throws at you everything it can - many of these are things I really didn't ask for and didn't want. Basically made up stuff. ChatGPT - tried some embellishments, and created cheesy slogan, but as expected. It's funny how such simple test shows how these models work in general.   submitted by   /u/FPham [link]   [comments]