Source: whatmodel.sh, an open source AI leaderboard I built (and launched here yesterday). Every point comes from published benchmark results: 22,113 of them across 987 benchmarks, from 1,011 sources, each one linked to where it was published. This chart uses data for the text models: 18,755 results for 1,211 models on 617 benchmarks. - 14,436 results are independent evaluations: papers whose authors ran the models themselves (mostly CC BY papers from the ACL Anthology and arXiv), Epoch AI, the LMArena dataset, EvalPlus, the EleutherAI harness and others. - 7,677 are scores labs report for their own models, labelled as lab-reported. Everything is open: the dataset (CC BY-SA 4.0) and the scoring code (MIT) are at https://github.com/jakemanger/whatmodel, with CSV downloads at https://whatmodel.sh/data/manifest.json. Every number links back to its source, and each source's licence is recorded. Interactive version (click any model to see its scores and sources): https://whatmodel.sh/race Please star on github if you find it useful!   submitted by   /u/jakecoolguy [link]   [comments]