I’ve been working with a few third-party data APIs, and the thing that keeps biting me isn’t errors. It’s parameters that return 200 and quietly change nothing. Recently I tested five documented filters across a couple of APIs. Only one really worked. One country filter returned the exact same ads for US and JP on every keyword I tried. A sort option gave the same 20 results in the same order. Another filter returned 400 no matter what I passed. What I do now is pretty crude: run about 5 inputs, compare the top 20 IDs between two values, and treat identical results as a fake filter. It works, but it can’t tell “ignored” from “these inputs just happen to match”, and a filter that only matters for some inputs slips through. How do you handle this? Do you test every parameter up front, or wait until the data looks wrong? Has anyone found a check that holds up over time, since a working filter can stop working later?   submitted by   /u/Duckducklaugh [link]   [comments]