Five failures, one survivor at 1:40 a.m. — fixing the two-headed character

It's a little embarrassing to write as one line, but I didn't save the fullbody by bolting on more — I saved it by removing what I'd been bolting on. So this note isn't "how I designed the 6 stages." It's "why it became 6 stages, and what I turned off in the sixth one."

Why it became 6 stages — this wasn't supposed to be one

The original plan was a single pass. Take the base portrait, uncrop into a fullbody frame, inpaint the legs and background. Thirty minutes of work.

Five hours went by.

The kinds of failure I watched across those five hours roughly went: peg-leg (smooth cylinder legs, no shoes), body shredding (shirt material changing three times within one image), ghost heads (a second head fogging in next to the shoulder), severed limbs (a hand cut off at the wrist, floating in midair), and giant familiars at tall aspect (the palm-sized familiar from the story rendered as a creature taller than the protagonist). Five different failures. That was almost the only mercy — if it had been the same failure five times, the issue would have been my prompt. Five different failures meant my tool stack was the wrong shape.

I didn't see that signal until 1:40 a.m. I could have seen it earlier.

The numbers from that batch

11,147 isn't only from these 6 stages. It's a cumulative count over the same batch window including other tasks, and I should be straight about that. But the weight of R3/R5 rejects I watched on screen that night was in the same range as the cumulative count suggests.

The decision at 1:40 a.m.

When the fifth pipeline finished, the verdict came back as one line.

verdict: variant 9/9 — all R5 (AI artifact) or R3 (severed)
recommend: rebuild pipeline from scratch

That one line was the real fork. I had two options.

(1) The ControlNet skeleton sometimes reads twice — patch in more variables to stabilize it. The familiar path. (2) Turn off every helper, change only the aspect ratio and one prompt line, draw it fresh. The throw-five-hours-away path.

(1) looked more reasonable. All the ControlNet tuning I'd built up was sitting right there. But when I looked at the screen again, every one of the five failures had come out of something I had switched on. Turn ControlNet on, get two heads. Turn U2Net on, get shredded shirt material. Stretch the aspect ratio, get a giant familiar. I had been adding helpers to assist the model, and the model had been filling the empty space those helpers made with its average human anatomy.

So I went with (2). Threw out five hours of ControlNet config and started over from one line.

Why the sixth one worked

The Step 6 settings were simple.

aspect_ratio: 512x832
segmentation: isnet-anime
controlnet: off
regional_prompter: off
base_prompt_suffix: "extremely zoomed out, full body framing, single character"
base_image_ref: required (hash)

Out of those five lines, the only one I actually added was base_image_ref: required (hash). The rest are either off lines or back to defaults lines. ControlNet off, regional prompter off, aspect less tall (512x832), segmentation changed to isnet-anime — and isnet-anime is just a segmenter that fits anime-style bases better, not something I invented.

Step 6 produced a live character on its first try. 2:11 a.m. That carried forward: 6 of 9 variants passed on first generation. R5/R3 reject rate on the same character dropped from a 64% average to 12%.

Why it worked — I don't have a clean answer. The most honest version is: I stopped forcing the model away from its average anatomy and let it draw what it wanted to draw. Every time ControlNet tried to pin the skeleton, the model regressed toward "average human" inside that pin, and my character was off-average. Removing the pin gave the model room to be off-average too. Whether this generalizes to every SDXL-class model — I'd want another week before I claimed that.

A longer hypothesis: helper tools give the model "you can't draw here" pressure more than "you should draw here" pressure. ControlNet's skeleton lock looks like a positive signal — "stay inside this skeleton" — but to the model it reads as a negative one, "every pixel outside this skeleton costs you score." Stack enough negative pressure and the model retreats to the narrow zone where it can score safely. That narrow zone is average anatomy. So my off-average character lost the room to score. If this is generally how SDXL's RLHF works, then any character with a non-mainstream skeleton is safer with fewer helpers on. It's a hypothesis I keep wanting to call a finding, but two characters isn't enough. A third and a fourth will turn it from hypothesis to finding, or kill it.

Five hours, one chart

A diagram is shorter than the words I just wrote. The night collapses to this:

flowchart LR
    A[Step 1<br/>portrait + uncrop<br/>30m planned] --> B[Step 2<br/>+ inpaint<br/>peg-leg]
    B --> C[Step 3<br/>+ ControlNet<br/>ghost heads]
    C --> D[Step 4<br/>+ U2Net<br/>body shredding]
    D --> E[Step 5<br/>+ regional<br/>giant familiar]
    E -. 1:40 a.m. .-> X{kill every helper}
    X --> F[Step 6<br/>nothing on<br/>1 survives]

    style A fill:#1f2937,stroke:#374151,color:#e5e7eb
    style B fill:#7f1d1d,stroke:#991b1b,color:#fecaca
    style C fill:#7f1d1d,stroke:#991b1b,color:#fecaca
    style D fill:#7f1d1d,stroke:#991b1b,color:#fecaca
    style E fill:#7f1d1d,stroke:#991b1b,color:#fecaca
    style X fill:#0e1117,stroke:#B5F23D,color:#B5F23D
    style F fill:#14532d,stroke:#16a34a,color:#bbf7d0

What's worth looking at isn't the arrows — it's the text in each box. Step 1's "30m planned" and Step 6's "nothing on" describe the same job. I spent five hours to land on a setting simpler than what I started with. Work graphs like this don't look like a clean GitHub commit graph; the more honest shape is a curve that loops back to where it started. At least for AI image work.

Trying the same rule on one more character

Right before writing this note I tried the same rule — "ControlNet off + isnet-anime + 512x832 + extremely zoomed out + base hash" — on a different character. Not one with a similar build; I picked one with a deliberately different skeleton.

7 of 9 variants passed first try. R3/R5 reject rate: 22%. Worse than Character #1's 12%, but in the same regime. I think the base hash reference works differently per character — characters with a sharp, single-source base portrait seem to anchor better than ones whose base is a mixture. Two samples is barely evidence, so seems is all I'll say. The third character will tell more.

The one thing I'm sure of: this rule lets me always start from scratch cheaply. The "5-hour stack then discard" failure mode is gone. It's not that the work got faster from 5 hours to 30 minutes — it's that I stopped paying the discard cost. The 30 minutes for Character #2 included running Step 6 once and reviewing 9 variants, and the cost of deciding to discard that batch is 200× smaller than the cost of discarding a 5-hour ControlNet stack. The work didn't speed up; the cost of throwing wasted work away got cheaper.

What I'll try next time

The next fullbody job, I won't rebuild the 6-stage pipeline. I'll start from one line — "aspect 512x832 + isnet-anime + extremely zoomed out + ControlNet off." If that doesn't pass on the first try, then I'll add helpers one at a time, ControlNet alone first, then regional prompter alone. Never again do I want to start with a five-hour-thick stack already loaded.

What you can take to your workflow isn't the 6 stages. It's the last line: when an AI image task breaks on you, try remove before add. In my work, the simplest combination drew the most correct body more often than the loaded one did. Whether the same holds for your character — I'll check on the next one and report back.

Editor's note: the 7-bucket failure counts (error 3,115 / timeout 2,549 / failure 2,378 / nonzero_exit 1,949 / permission 874 / exception 227 / fatal 55) are direct counts from local failure_category_counts. The 6-stage pipeline, each failure mode, the 1:40 a.m. verdict, the 512x832 isnet-anime pass, and the R5/R3 drop from 64% to 12% are recorded in work log §D (21ededc6+2657bd88). The 11,147 isn't attributable to these 6 stages alone — same-window cumulative across other tasks. [GAME_BETA] is a project codename.