TLDR: training my first general law model and unsure what to do with my data PRE-PS: it will be a replacement for opus in my current AI LAW harness, I know it will be hard but I wanna do this project and find it geniuenly fun. I’m building a Polish legal model using Qwen3.5-9B-BASE (I plan to scale up to 27b qwen4 later). I’ve done CPT and have quite a bit of data available, but I’m getting lost about what should come next or even if I should do more CPT. My main question is how to turn this (maybe) domain-adapted Base model into an agentic+chat assistant that reasons through legal problems. But more importantly, how do I get reasoning, teach my in-harness tool-use training, and where does SFT vs RL belong in that process and in what way (especially confused about reasoning and SFT)? What I’ve already done Three sequential CPT stages, roughly 80M tokens each, in this order: Complex legal commentaries → legislation (EU+local) → court decisions I didn’t mix the sources. Each stage continued from the previous checkpoint. On a small Polish financial/legal multiple-choice quiz (really bad eval, just a PoC for now). They all perform very slightly better than base and legislation checkpoint did the best. But again, it's a base model. I haven’t done assistant SFT yet. What data I have (I can get more data of each of these types if you guys recommend) Besides the legislation, commentaries, and decisions used for CPT, I have: Facebook legal Q&A groups: real, messy questions, follow-up clarifications, and answers of very mixed reliability. Certified/institution-authored Q&A: my highest-confidence answer material. Some answers contain detailed analysis; others are just a conclusion and a citation from random institutions like the national bank. Law study materials: from a local college. it has essays, powerpoint presentations stuff like that, very little "tests" tho, but some practice material (like the essays). A/B/C/D questions with answers: relatively little of this; currently being used for evaluation. Thousands of Claude Code–style harness conversations: multi-turn conversations from an existing application/harness. These would need filtering, anonymization, and quality checks. Contracts: government published contracts it can learn from Previous "ML" experiments (models and their training data): NER, generating complex legal court decision summaries, generating graphs from legislature or court decisions (actually got some models that do this successfuly If you guys could point me what data is ideal I could maybe get it too! I can use the basically unlimited codex to clean, restructure, or generate material from these sources, but I don't know how to. What I don’t know is which material should be trained on directly (CPT or bare SFT), which should be transformed (SFT or RL?). For example: 1. Do I continue CPT on the remaining sources, or stop and start SFT? Should I expose the model to the Facebook discussions, study materials, and expert Q&A through CPT first, possibly mixed with the authoritative material? How many tokens for this 9B model and how much for the 27B I plan later? Or should I skip further CPT and use those sources only to build targeted SFT examples? For example the Facebook QnA stuff is real people diverse speech but I can't really reliably verify if the answers are correct (I can only measure likes or high quality people in the group) so idk if I should cherry pick from this? Have an LLM turn this into SFT chat examples? Or just CPT on this? 2. SFT on what, specifically, to teach REASONING? I could start with my highest quality Q&A and teach the model to answer questions. That seems like an obvious chatbot baseline. But if I just train on QnA there won't be any reasoning! The HQ QnA is pretty verbose but I wouldn't bet it could just be re-used as a "reasoning chain". I was thinking of asking Astra to workout those questions and give an answer with its reasoning BUT I assume it will be wrong on 90% of the questions (cuz no internet search, and I can't give it internet search cuz that's not clean QnA data and I would have to transform it into my harness function calling anyways with credible sources not just google searches for legislature) And those 10% it gets correct are trivial and the reasoning could be bad anyways and I would still have to check correctness of the answers, and giving it the answer or hints is imo also not ideal. Or injecting "synthetic reasoning", it suffered from post-hoc in my tests. Another problem is the High Quality QnA is quite rare, most QnA is short and possibly wrong. And generating the missing reasoning is even harder. Because again, I don’t trust a frontier model to solve difficult Polish legal questions from memory. It would need relevant source passages or access to the harness (which would just the gun into in-harness training?). So I'm really unsure on what to do, I was thinking of doing stuff like this, but this is just brainstorming: A) Take verbose long answers train on that, then ask the qwen or another model to make a "summary" at the end and then put the verbose long answer into its block B) Take those 10% of QnA where frontier model actually gets it right with correct thinking (rare) and SFT on that. then we have our finetune try answering more QnAs, we verify if it's correct if it is it goes into the next SFT run and we iteratively (almost like an RL gym) train reasoning from the dumbest examples Both seem like bad options, but kinda funny to think about. 3. Chatbot first, then tools or teach reasoning through tools from the start? The existing harness lets the model write Python and call functions such as vector_search_laws() and graph_search_legislature(), inspect results, and continue researching. I’m leaning toward getting a regular chatbot working first. But the teacher can produce much more trustworthy legal solutions when it has access to sources. (as I've already mentioned I have many in-harness examples, except they could also be incorrect and I don't have their "thinking blocks" (cuz it's a proprietary frontier model) so I again have to inject thinking post-hoc which from my tests doesn't work for law) I was thinking that, if we make chatbot first, the model will at least learn intuition on WHERE TO LOOK, without looking through sources but at least will know a general idea of where to search based on being a chatbot first (even tho possibly incorrect chatbot sometimes, cuz law is really complex) and teach him to function call in-harness to get sources, unsure about this. 4. When does RL enter, and on which task? AFAIK, RL is what develops the model's reasoning on its own (deepseek R1) but I suspect it would be very costly and we could give it some example reasoning first anyway to kick it off(?) So like, instead of the whole fiesta of injecting reasoning or growing it, we could have RL develop it (afaik RL is the only thing that "creates" reasoning unless you distill it) but then I would like to have reasoning even in the "chatbot" and right now I can just jump from base model -> RL, right? Also for legal Q&A, matching the reference conclusion doesn’t necessarily mean the reasoning is correct. What would you actually reward? Like a law similiarity judge LLM + same paragraphs used? What I’m trying to decide One possible route is: Current CPT checkpoint → gold-Q&A SFT → grounded reasoning/tool-use SFT → RL in the harness But I’m not committed to that order. I don’t know whether I’m missing an important stage or inventing unnecessary ones, first time I'm training a GENERAL model from "base". What I would like the finished model to do: It will be an opus replacement so general QnA in and out of harness but especially: have a good "hunch" and understanding of polish law will be good at anonymizing documents generating complex legal summaries for court decisions (heavy reasoning task) generate graphs from legislature and court decisions create/edit/review contracts These were all my initial ML experiments and I actually trained good models for those specific tasks, but I figured, hey I have a lot of this data around why not just combine them and make a generalistic model? Thanks a lot for reading this mess of my thoughts! if you could point me towards good resources for this I would also be glad, for example unsloths RL docs are kinda ass imo. I really like andrej karpathy's videos but they're quite old now by AI standards!   submitted by   /u/SignificantZebra5883 [link]   [comments]