I have trained more than a dozen FLUX LoRAs over the last several months, almost all of them aimed at illustration styles. Editorial flat vector, brush-painted, line-art-with-fills, hand-drawn cartoon. A few worked. Most hallucinated in reproducible ways the existing public write-ups skip past or hide behind cherry-picked sample grids.This is a write-up of how they fail. There is no shortage of “here is the recipe” tutorials and they are mostly fine if your use case is a face or a product photo. Illustration is where LoRA training breaks down in interesting ways, and the public engineering writing does not reflect that yet. I burned multiple weeks of dataset iteration and compute time getting wrong outputs and would have paid real money to read this article on day one. I am going to be mildly grumpy about it.Why FLUX is good at photos and bad at illustrationThe short version: pretraining mix. FLUX 1 dev, FLUX 2 dev, and every base checkpoint I have looked at have strong photographic priors. The training corpus is dominated by photographs, depth-of-field, skin texture, film grain. When you ask base FLUX for a photo of a person in a kitchen, you get something convincing because you are reinforcing what the model already knows.When you train a LoRA on an illustration style, you are not reinforcing priors. You are fighting them. The base model has a strong opinion about what a person looks like and that opinion is photographic. Your LoRA is a small low-rank perturbation that has to shove the model far enough away from its photo defaults that “flat vector illustration of a person” looks actually flat and vector, not like a photograph with a stylization filter applied.This shows up at every layer. Attention layers want to put photographic detail back in. The text encoder, if you trained it, partially associates “person” with photo-of-person and forgets that association slowly. The denoising trajectory at low noise levels reintroduces high-frequency texture that pulls the output back toward photo. Realistic LoRAs do not have this problem. Illustration LoRAs fight the prior, converge slowly, and even when they converge they leak the prior in subtle ways. That asymmetry is what most of the failure modes below are really about.The five failure modesHit each of these more than once across different datasets and different styles. There are other failure modes, but these are the ones that show up almost every time and that the tutorials gloss over.Photo bleed-throughThe model places photographic features into images that should be flat. Skin gets pore texture. Backgrounds get bokeh. Hair gets specular highlights. Fabric gets cloth grain. The prompt says flat illustration and the image looks like an illustration with a Photoshop noise filter on top.Base model priors win at any reasonable LoRA strength. You can crank strength up to 1.4 or 1.5 and start to break through, but at that strength everything else collapses. Composition goes weird. Color shifts. Hands fall apart. The strength range where photo bleed-through is fully suppressed and the rest of the model still functions is narrow, sometimes empty.What helps. Train longer than you think, especially on attention layers. A common starting config is 1e-4 learning rate on UNet and 5e-5 on text encoders. For illustration LoRAs I have ended up with attention-layer rates around 2e-4 and let it cook for more steps. Be careful, because this also makes the next failure mode worse.Captioning hygiene matters more than the tutorials let on. Every caption should hammer the style. Flat vector illustration of X , Editorial illustration of Y. Style word in every caption. The model associates the style words with the style only if they are consistent.If your style is genuinely flat, preprocess your training images to remove residual noise. LoRAs trained on JPEG-compressed illustrations can bake the JPEG ringing into the style and show up in outputs as a faint compression halo.Style memorization vs style transferThe LoRA reproduces training-set compositions instead of learning the style. You ask for“a person waving at the camera” and get something structurally identical to one of your training images, with minor cosmetic variation. Ask for “a dog in a park” and the dog is in a pose your training set contained once.This is the failure mode that makes people think their LoRA is working when it is not. The outputs look good as long as the prompt matches a training composition. The moment you ask for something compositionally novel, the LoRA reverts to base-model output or produces a Frankenstein of remembered compositions.The root cause is some combination of: training set too small, training set too compositionally uniform, captions too vague. If you have 20 training images that all show a person standing centered facing the camera, you have not trained a style. You have trained person standing centered facing the camera in this style If your captions all say an illustration of a person, the model has no signal to disentangle subject from composition from style. Recent academic work like ConsisLoRA and K-LoRA tackles this from the training-objective side, but the dataset-level fix is still the highest-leverage move in practice.Mitigations are dataset work, not training work. Aim for 50 to 100 images, deliberately varied in composition. Wide shots, close-ups, multiple subjects, single subjects, different angles, different framings. Captions should describe the subject and the action explicitly, leaving the style for a consistent trigger phrase. Caption-writing is the bottleneck. I have spent more time on captions than on any other part of the pipeline and I still think I undercaption.If you cannot vary composition enough (your source material is 30 illustrations and they are all editorial portraits, say), be honest that you are training a portrait-style LoRA, not a general one. Calling it general is how you ship something that breaks on its second prompt.Subject hallucination at low CFGCharacter anatomy gets worse after LoRA training. Base FLUX largely solved hands and reliably gives you the right number of limbs. After training a strong illustration LoRA, extra fingers come back, faces distort, hands turn into anatomical impossibilities. More pronounced at low CFG values, which is where you would normally generate for stylistic flexibility.I do not have a complete root-cause story. The pattern I see is that the LoRA shifts attention away from the conditioning paths the base model uses for anatomy. If your training illustrations contain stylized hands (which they often do in editorial styles, simplified to two or three fingers as a deliberate aesthetic choice), you have taught the LoRA that two-finger hands are correct. The model is doing its job. Just not the job you wanted.What helps. Include illustrations that show realistic finger counts and clearly-rendered faces, even if those are not your most iconic examples. You are training a style and also reminding the model not to forget what hands look like. Some practitioners caption hand-prominent images with explicit hand descriptions (“five fingers visible on the right hand”). I do not know whether it actually helps or whether it is folk knowledge, but it does not seem to hurt.Generate at slightly higher CFG than for base FLUX. I have ended up running illustration LoRAs at CFG 4 to 5 where I would run base FLUX at 3 to 3.5. Higher CFG pulls anatomy back toward correctness at the cost of some stylistic flexibility. Inpainting is a viable last resort and I no longer feel bad about using it.Color palette collapseThe LoRA learns one specific palette and refuses to deviate. Prompt for the same illustration style but in a red-and-orange palette and get the original palette. Prompt for monochrome version and get the original palette with slightly reduced saturation. Prompt against the palette directly and the LoRA will fight you.This happens when your training set has a narrow color distribution. If every training image is in the same brand palette (often why someone trains a LoRA in the first place), the model entangles style and palette into one inseparable concept.Sometimes that is fine. If the LoRA exists specifically to enforce a brand palette, collapse is a feature. The pain shows up when you want a LoRA that captures a stylistic technique (line weight, fill style, composition language) that can be applied across palettes. That requires engineering the dataset to vary palette while holding technique constant, which is hard because the source material rarely cooperates.If palette-varied training data is not available, treat palette as part of the style and stop pretending otherwise. The downstream user needs to know the LoRA is locked to its palette, because they will eventually ask for a variation and someone has to tell them no.Prompt-following degradationBase FLUX follows long, detailed prompts well. After LoRA training the same model is worse at it. Subjects in multi-subject prompts get dropped. Attributes get reassigned (the man in the red shirt and the woman in the blue dress produces both subjects swapped or one of them in a purple shirt). Spatial relations (the cat to the left of the lamp) fail more often.The likely cause is overfit, especially if you trained text encoders. Text-encoder training is where a lot of the prompt-following capability lives, and small training sets push that capability sideways fast. A LoRA that performs well on prompts you tested during training but worse on prompts you did not test is the classic overfit signature.What helps. Train UNet only as a starting point. Add text-encoder training only if you genuinely need stylistic prompt sensitivity UNet-only cannot deliver, and use a very low learning rate (5e-5 or below).Keep a held-out eval prompt set of 10 to 15 prompts not in the training distribution. Spatial-relation prompts, multi-subject prompts, attribute-binding prompts, long descriptive prompts. Run these every few hundred steps and watch for regression. The moment prompt-following degrades, stop training. Do not wait for visual style to look “done,”because the LoRA will overshoot and you will lose prompt-following you cannot easily get back.The hallucination problem nobody writes aboutThis one I want to spend extra time on, because the public write-ups gloss past it and it is the failure mode that has cost me the most production rework.Illustration LoRAs hallucinate scene content in a way photo and character LoRAs do not. The model invents objects that were not requested. Extra chairs at the table, extra windows on the building, extra background figures. Invented text on signs and posters. Invented logos on packaging. Invented brand marks on clothing. The output contains the requested subject plus a constellation of ambient detail the prompt did not authorize.In photographic outputs this would get caught. Photo realism is unforgiving. A photo with the wrong logo on a coffee cup is obviously wrong. The viewer’s eye expects photographic plausibility and notices when it breaks.Illustration style provides cover. The viewer’s eye is already calibrated to expect stylization and artistic license. An invented chair reads as artistic choice rather than as model failure. An invented logo reads as “the artist made up a brand.” The hallucinations are camouflaged by the medium, which means they ship.The mechanism, as far as I can tell, is that illustration training sets often contain compositional density. Editorial illustrations love putting “stuff” in scenes. Background detail, ambient objects, environmental storytelling. The model learns “this style has scenes with a lot of small objects” and then helpfully adds small objects whether the prompt asked for them or not. The dataset taught it that the style is partially defined by ambient density.I do not have a clean mitigation. The cleaner and sparser your training compositions, the less of this you get. If your style permits dense scenes, you will get hallucinated content, and you will have to either inpaint or filter at the eval stage. Captioning training images extremely densely (describing every object in every scene) helps marginally, because the model learns that objects are nameable rather than ambient. It does not fully solve it. This is the failure mode that has most often caused me to throw out a generation that was otherwise on-brief.What I’d do differently next timeRoughly in the order I would tackle them if I started over today.Smaller dataset, more iterations of curation. I used to start with whatever images I had access to and iterate. I now spend the first day or two picking and rejecting images. Composition variety, palette variety where possible, anatomical reliability, clean source files. 60 carefully picked images beats 200 indiscriminate ones, every time.Captioning audit before training, not after. Read every caption out loud before you start. If half of your captions are “an illustration of a person,” your dataset is going to memorize compositions. Rewrite until each caption describes the subject, the action, the framing, and uses a consistent style word. Tedious, and the highest-leverage thing you can do.Eval prompts from step one, not step two thousand. Pick a held-out set of 15 eval prompts that cover the kinds of generations you actually need to ship. Run them every few hundred training steps. Do not wait for a final checkpoint to find out you have been overfitting since step 500.Train UNet first, text encoders maybe. Default to UNet-only for the first run. Add text-encoder training only if you have a specific reason and a budget to retrain when it goes wrong.Accept that some styles are not LoRA-tractable. There are illustration styles where LoRA on top of FLUX does not converge to acceptable output, no matter how clean your data. Highly geometric vector styles. Single-line continuous-stroke art. Styles that depend on negative space the model does not respect. For those, full fine-tunes or a different base model are the realistic answers. Knowing when to give up saves more time than tuning hyperparameters does.Treat the training run as the cheap part. The expensive part is curating the dataset and writing the captions. The training itself is a couple of hours on a rented H100 via fal.ai or equivalent.Honest verdictLoRA on FLUX is the right tool for illustration when the use case is narrow: a consistent stylistic motif, a known brand asset, a limited range of prompts. If you need “always this exact style of editorial illustration applied to a constrained set of subjects”, a well-trained LoRA does that job cheaply.It is the wrong tool when the use case is “broad illustration capability across diverse subjects with palette flexibility and reliable prompt-following.” For that you need a bigger fine-tune, a base model pretrained with more illustration coverage, or an orchestrated pipeline where the LoRA handles one slice and other models handle the rest. Public LoRAs on Civitai and Hugging Face can give you a sense of what styles are tractable before you commit to your own training run. I have ended up at the orchestration pattern more often than I expected to, and now treat LoRAs as one stylistic component of a larger system.If you are about to train an illustration LoRA, expect at least one of the five failure modes above. The dataset work is going to take longer than the training run, the eval prompts matter more than the visual checkpoints, and some of the failures you just have to accept because no amount of hyperparameter tuning will get you past them.\