I’m working with roughly 5 million public Polish court decisions. The goal is structured extraction: who the parties are, what was requested, what the court decided, obligations, amounts, relationships, etc. Eventually we want to run interesting statistics over the results, like "who gets usually the custody of the child?" A strong LLM can produce useful JSON graphs, we can run statistics on. The problem is getting similar extraction at corpus scale. We have a small 13-document pilot. It already contains 160 distinct entity-type strings, 90 appearing only once. These aren’t just PERSON / ORG / DATE. They include abstract legal concepts, obligations, procedural events, and things like “increase in child support.” here's our demo output from astra: { "effects": [ {"id":"e_zastavenie","change":"zastavit","target_ref":"konanie"}, ], "entities": [ {"id":"this","kind":"uznesenie"}, {"id":"vyrok","kind":"vyrok"}, {"id":"zahlavie","kind":"zahlavie"}, {"id":"odovodnenie","kind":"odovodnenie"}, "effects": [ {"id":"e_zastavenie","change":"zastavit","target_ref":"konanie"}, ], "entities": [ {"id":"this","kind":"uznesenie"}, {"id":"vyrok","kind":"vyrok"}, {"id":"sud","kind":"sud","value":"Okresný súd Bratislava "relations": [ {"id":"r_vlastnik_vyroku","relation":"patri_do","from_ref":"vyrok","to_ref":"this"}, {"id":"r_vlastnik_zahlavia","relation":"patri_do","from_ref":"zahlavie","to_ref":"this"}, Some of this is real conceptual diversity. Some is inconsistent naming or different levels of specificity, but as you can see complex and diverse stuff with a clear "language" i came up with with the help of astra. i previously tried a big "god schema" (20k chars) but the decisions are simply just too diverse. The pipeline I’m considering: Something that extracts the entities from source texts into some kinds of "tags" a classifier like jev (but probably fine-tuned) will get a window of source text and already named relations and decide on the next relations done The candidate model would need overlapping spans and possibly multiple labels per span. Slovak inflection adds another wrinkle: exact source wording often differs from the canonical concept name. Lemmatization helps with grammatical variation, but not synonyms, paraphrases, or concepts inferred from context. I’ve previously tried Jina embeddings → nearest-neighbor candidates → LLM decides which terms to merge, and it worked reasonably well. But that was on a smaller corpus and i'd be damned if i process 250k docs and then figure out thing X was wrong and i have to do it all over again. i'd love to know if you guys have any pointers. THANKS for reading. AI TL;DR: I want to turn 5M court decisions into entity/relation graphs without running an expensive LLM on every document. Thinking small entity extractor + relation classifier, but the legal concepts and naming get messy fast. Anyone built something similar? Looking for ways to train this and normalize concepts without discovering a design mistake 250k documents in and having to redo everything.   submitted by   /u/SignificantZebra5883 [link]   [comments]