Since the frontier labs have changed to not sharing their thinking traces, how would you go about making a distillation dataset from the current frontier models for a specific task? For example lets say we want to fine-tune a small Qwen/Gemma model to output only one reply from a photo to extract co-ordinates of 3D spatial data which some of the current frontier models do well. How would I be able to make a dataset that is still useful if I only get the model's answer? I am guessing this might work: Send photo and task text to frontier API = Question Get back answer only --> Answer Now send question and answer pair to a Chinese frontier model and ask it what it would think to get to the answer --> Use as thinking trace I'm guessing using a thinking trace of a different flavour of model might give bad results, but I am curious how one could approach to making such a dataset? I'd love to hear if someone has thought of a better method edit: Fixed my markdown, now bullet points are correct   submitted by   /u/thoquz [link]   [comments]