We built a dress-up app entirely out of generated video, with user-triggered outfit changes taking place while the character dances, all in a seamless loop. It took 1,548 clips that all had to be right; far too many to check by human eye. Nobody has time (or resources) for that. So we needed that oversight to be baked in. The majority of it, at least…
Here’s how we made the process reliable at scale: diffusion ‘recipes’ that make the right pre-generated clip likely to appear (as well as in perfect sequence), and self-healing systems that measure every clip, fixing the ones that need fixing.
Every clip has to be right
A photoreal character dances on a loop. You get to pick and modify her outfit on the fly by scrolling through various clothing choices for each body section; head, torso, legs, and feet. Tap a new hat and she pulls the old one off, throws it away, hops, and lands wearing the new one. Nothing is generated while you play.
We create every outfit’s loop, and every change between outfits, up front, and the app cuts between them on the exact frame where they line up. How did we do it?
So. We have 10 items of clothing in total to choose from. This doesn’t sound like much, but those ten items amount to:
36 unique outfit combinations/loops
1,512 transitions:
1,548 clips (built from roughly 12,000 videos once every intermediate step is counted)
Because the app cuts straight from one clip to the next, a single sketchy one is enough to break the illusion. Thankfully, generated video goes wrong in familiar ways, which gave us clear problems to solve:
On its own, a generation pipeline is what a control engineer (the human at the steering wheel) would call an open loop. Following the prompt, you put a seed in and get a video clip out. Every attempt is a bit of a dice role, and there’s no automatic way to check the result. That job is left to a human; fine for a handful of clips but you can appreciate how it’s completely unfeasible when we’re dealing with thousands (and with a releasable dress-up app like this we’d likely incorporate many more clothing items than feature in this demo; we could be talking tens or even hundreds of thousands of clips).
We closed the loop in two ways: first, we make the plant (the video generation model) better behaved, so the correct clip is the most likely one. Basic probability. Second, we wrap every generation in a feedback loop that measures the clip and repairs it when it falls short.
1 · Recipes that make the correct clip likely
Prevention is better than cure. Before measuring anything, we build the generation process using specialised diffusion recipes (steps that guide a diffusion model from random noise to a coherent, motion-consistent video) that prevent entire categories of problems from happening in the first place.
Ring denoising for seamless loops
A video model normally generates from a first frame to a last frame, and the last frame has never ‘seen’ the first. Ask it for a loop and you get a clip that almost comes back to where it started, then jumps. Not good. Generating a bridge back to the start afterwards doesn’t fix it either: the bridge jumps too.
So our loop recipe changes how denoising works. It treats the clip as a ring, where the frames at the end of the clip are the same as those at the beginning. At every denoising step, the model sees the join from both sides, refining the frames at the beginning and the end at the same time (as one) ensuring there’s no sudden jump when the video loops.
The same thinking runs through the rest of the pipeline:
Recipes like these reduce the scope of things that can go wrong, but they don’t remove the dice roll. A model can still drop a hat or draw the wrong jacket. That’s where the feedback loop comes in.
2 · Self-healing Systems
Every clip in our workflow is made by a System: a generation step wrapped in a measurement and a controller. Drawn the way a control engineer would draw it, it’s a feedback loop:
The sensor combines two kinds of measurement. Exact checks handle whatever can be computed: hard cuts, blank frames, and how visible a loop’s restart is compared with the clip’s own motion. These are cheap, and they give the same answer every time. Everything else has to be looked at, so we ask a small, fast vision model (Cloudflare’s Clef) to act as a judge. It gets a single frame and a few plain yes/no questions: Is there exactly one person? Is she wearing the black beanie? Is she wearing the leather jacket? A small judge is reliable when its questions are easy: one well-chosen frame rather than the whole video, and wording that leaves no room for doubt.
The controller doesn’t just reroll the seed and hope. It passes the prompt, the seed and every finding to a language model, which works out what went wrong and rewrites the prompt to address it. Here’s a real case. She’s meant to swap her cream sweater for a leather jacket and keep everything else:
The judge doesn’t need to know why the beanie disappeared; it simply reports that it did. The controller works out the likely cause (pulling a sweater over your head takes your hat with it. We’ve all been there) and writes the one sentence that addresses it.
The System generates again, measures again, and the clip passes. Roughly half of the clips featuring the beanie healed this way, and nobody had to find them first.
A clip that can’t be healed isn’t shipped. It’s escalated, because a fault the controller can’t fix usually means something upstream needs a person’s attention. Human supervision and intervention is still a thing (at Anorak, anyway).
Composable by design
A System doesn’t care what’s inside it; it just provides the overall structure to complete a task. It can use any generation recipe, the sensor any set of checks, and the controller any policy for improving inputs. This makes Systems building blocks in their own right; self-contained units that can be configured for differnt outcomes.
Outfit loops and transitions use different recipes behind the same measurement and controller, and the workflow simply runs one System for every clip it needs:
That’s what makes the method repeatable. A new character, a new dance or a bigger wardrobe (more clothing items) is a new input file, not a new pipeline. One more option in every category takes the same workflow from 1,512 transitions to 4,536. Adding a check means improving one sensor, and every clip gets the benefit. A better recipe can be dropped in as the plant, and the loop around it stays the same.
Quality control usually means people reviewing output. Here it’s designed into the workflow itself: recipes that make the right clip likely, and a feedback loop that turns a poor clip into a good one. A level of 1,548 clips is about 80 GPU-hours of generation, and nobody’s watching…*
So here’s the demo! Enjoy. Built using AGO: our in-house generative AI orchestration platform that allows us to render complex workflows containing of thousands of videos.
*We do still watch the clips, just not all of them. People still decide what good looks like (see Shaun’s post on the crucial role of taste). The Systems make sure every clip hits the mark.