Hello! I just released wyat-2, a fine-tune of Ouro-1.4B (a looped transformer) trained on a single RTX 4070 Ti. The link is the report. As far as I know it’s the first decision model built on a looped transformer: not the best, but the first. I realized I would need heavy data work to make it the best, so I’d prefer not to spend too much time on it for now. Maybe later, if it catches interest. Report : https://wyat.io/wyat2-brief.pdf Model: https://huggingface.co/teilomillet/wyat-2-1.4b   submitted by   /u/Fit_Strawberry8480 [link]   [comments]