Hey all! I've been working on a custom agentic harness called Rivermark for social, marketing & print design. Whoever has tried to use LLMs (even frontier ones) for this knows that they can be a bit hit or miss, compared to website and app design, where their skills are much better. The benchmark consisted of 112 requests for various social and marketing images that a business might ask an agent for. Some contain a proper brandkit, some have a website reference for the brand, and some start from nothing. Well, the first results, based on 4700+ pairwise blind human judgments, are in! My headline finding, and one I didn't quite expect, is that open models, such as DeepSeek v4.1 Flash 1017 and GLM 5.3 Flash, match or beat plain Claude Code with Sonnet 5.5 and /design. Sonnet 5.5 itself gets the largest lift with Rivermark - it jumps into an impressive lead over everything else. This was also surprising, as I'd expect advantages to be smaller for "smarter" models, as they have this know-how already encoded. I'm writing a blog post and validating the findings with more judges now, but figured I'd share these interesting preliminary results. In the first photo, you can see Bradley-Terry rankings for these model/harness combos. Second photo is the cost vs quality comparison, and in the third photo you can see some representative examples of what models output with Rivermark vs the normal harnesses. Of course, some very important caveats apply: It's a design-specific harness. It contains some domain knowledge (I tried to keep this limited though). With increasingly good frontier models, these advantages tend to get wiped away, but for the time being, they provide a way to achieve similar results with a fraction of the cost. The harness has been running in production on some of these models, and so is tuned for them. (Not on testcases from the benchmark, however) It's tied very tightly to the product itself (https://rivermark.ai) at the moment. I'm planning to release an open source version, but it still would not be a general purpose coding agent. The product itself is optimized for speed and pleasantness of iteration, not one-shotting designs, and this is reflected in the harness. For example, we use the faster but more expensive FlashX GLM API endpoint, and it's a bit more conservative about self-reviews. I'm sure if I made something focused on just the perfect one-shot, I could beat these results on both quality and cost. Open to questions!   submitted by   /u/JannVanDam [link]   [comments]