voi-slop-detect-380M: A tiny model to detect AI slop

Wait 5 sec.

I've fine-tuned an UltraBERT-380M classifier to score how much your text feels like AI slop. Demo: https://voightkampfflabs.github.io/slop-detector-demo/ (runs fully in browser via ONNX + WebGPU/WASM) Why? Got tired guessing if I'm reading human or slop on twitter/linkedin/reddit, so trained my own AI detector. Like Pangram, but local - so all your perverted texts stay on your device. Because I can (and because I bought 2x 5090 when they were cheap). Training As it was made for learning purposes, I made my own dataset based on the Pangram v4 recipe: Collect a sample of pre-2023 human texts (rn it's around 11 public datasets, reddit included: 170k texts, 825k pairs) Slopify: summarize the text, and then "decompress" it back with an LLM to produce pure slop. Rewrite: ask LLM to rewrite it in a different way (e.g. more polished, simpler, punchy). Pivot: translate it back and forth, so the content is the same, but the style is different. The target label is how many words/ngrams were changed by the LLM. So it's not a yes/no, but more like a slopification rate guessing. My personal texts land around 0.1-0.2. Trained with HF Transformers on 2x RTX 5090. Considering the model size and hardware, the whole training run is ~3h. Training data is public: https://huggingface.co/datasets/VoightKampffLabs/slop-detect-dataset-v6 Results Not as good as Pangram (yet), but as good as GPTZero and Originality on most things I could compare. voi-380m Pangram GPTZero Originality EditLens, F1 0.86 0.92 0.73 EpochAI, human flagged 0.2% 0.0% 0.0% EpochAI, Opus/GPT/Gemini miss 0.0% 0.0% 0.7% UChicago, 2025 models miss 0.2% 0.0% 1.4% UChicago, StealthGPT rewrite miss 74% 0.0% 56% Non-native essays, flagged 0.6% 0.0% - HF model card has more details and more benchmarks. Where it's bad I'd rather tell you upfront: Humanizers and skills like no-ai-slop beat it. I guess that's why StealthGPT bench numbers are so bad as it's completely out of domain for the model. Asking Opus to rewrite its own slop until detector score goes down to 0.3 takes around 3-4 rounds for me. Old models and some legacy slop made with GPT-3.5 and Llama 2/3. Never trained on that. Poems and non-english texts. Too lazy to add this to the dataset. So what I'm currently training bigger 4B/27B brothers of this one. Yes they're much smarter, but good luck running them via WebGPU/WASM on device. And yes, the end goal is to have a browser extension which marks AI slop on reddit completely on device. PS. the text above scores ~0.1 in the detector (to save you a click).   submitted by   /u/InvadersMustLive [link]   [comments]