Remember Nintendo Punch-Out? In this experiment we took a Nano Banana (Google’s image generation model) background, projected it onto a 3D boxing ring, and dropped in a boxer generated as video with MiniMax H3 Max (a high-speed generative video model) — then keyed him, composited him into the scene, and lit the whole thing in real time with shadows, bloom (reproducing the imaging artifacts of real cameras), depth of field and a few other bells and whistles to bring it to life.
Pick your battles (and battleground)
The fun part of the generative aspect is how little of it is fixed. If we’d fancied Vegas or Madison Square Garden instead of a back-street gym, that would have simply required a different prompt. The fighter takes a bit more work to produce — he’s grown from a set of authored key poses that the video model animates between — but swap those out and you could just as easily be squaring up to your old school bully in the playground.
I’ve exposed the controls that show exactly how the scene is constructed. This is essentially what we would call a set of debug menus that would normally be hidden to the player (you lucky people! Invited into the Magic Circle for a quick tinkering). I spent quite a while building out the controls and tuning them to get a particular look and movement.
It’s open ended normally, which means it’s quite easy to break things if you don’t know what it all does (most but not all of you, I’d wager), so I have put in restrictions with this version to stop that from happening. But it’s a great peak under the bonnet and highlights some of the methods and technology we’re experimenting with.
Just a glimpse of what’s possible
When you get hit, the camera jolts back. In a finished product we would obviously go to town on this and incorporate far more complex movement that synchronises with what happens to the character. All of that is controllable in code. It can all be animated and made to spring back, rotate, change FOV (field of view) and do endlessly different things.
Similarly, all the post effects can be animated as well, so they can all be synchronized to work with different actions in the game. This is where the synergy between the generative aspect and traditional coding happens; the code understands the start/end points, position of the characters feet/fists and frame rate of the generative video component to achieve the look and gameplay we want.
Stick your gumshield in and try it for yourself here.