been messing around with small models on actual finance ops stuff, not just chat. this time it was payout review in kill bill (open source billing thing), running locally with fake data. video is a real screen recording, not sped up setup is pretty simple: 10 pending payouts, and the model has to decide for each one whether to release, hold or block took about 4.6 sec for all 10. released 4, held 3 till the card settles (~$5.2k), blocked 3 (~$11k). the blocked ones were a chargeback and two account takeovers, which is exactly what i was hoping it’d catch. it also leaves a note on each account saying why, which tbh is the part i liked most. whoever looks at it next doesn’t have to dig around got 10/10 vs my answer key. then i ran the same cases 3x each on three models: model correct cost per 10 payouts time drex 30/30 $0.00018 0.9s gpt-6 luna 30/30 $0.0008 1.5s claude sonnet 5.5 30/30 $0.019 3.6s so accuracy was the same, all three nailed it. the difference is cost. sonnet was ~106x more expensive for the exact same answers, and slower too. (0.9s is just model time, the 4.6s in the video includes kill bill updating) my guess is this kind of task is mostly structured signals, like chargeback history, recent account changes and whether the card settled. you don’t really need a frontier model for that. i’d expect the big models to pull ahead on messier cases where the signals contradict each other did loan decisions the same way before this, doing bank reconciliation in erpnext next   submitted by   /u/m4rkGS [link]   [comments]