I needed a cheap way to compare search agents on BrowseComp-Plus without burning hours of agent time every run. So I built a 15-query subset of the full 830 that ranks systems roughly the way the full benchmark does. - weighted Spearman rank agreement 0.88 averaged over 30 splits, versus 0.75 for a random 15 - two runs more than 10 points apart on the full benchmark keep their order about 96% of the time; 5 to 10 points apart, about 75% - all answers are 1-4 words, so it grades by normalized exact match, no LLM judge - costs under 2% of a full run https://huggingface.co/datasets/LocalLLaMA/browsecomp-plus-mini   submitted by   /u/asankhs [link]   [comments]