Watch is eating play. MIDiAS research shows gamers watch 8.5 hours over 7.4 hours per week playing. At Anorak we think that’s not a content problem but a format one. Video got frictionless; it never got interactive. We’re building the medium that closes the gap: gamified generative video - content as easy to consume as the feed it lives in, and as engaging as the games it borrows from.

In this post, we want to share some of the architectural choices and technical breakthroughs behind optimizing our generative pipelines. By bringing high-quality generative models down to sub-second and low-latency thresholds, we are turning heavy offline generation into real-time, interactive game-time experiences.

Over the last few years, generative models built around Diffusion Transformers (DiT) have made breakthroughs in image and video generation. Yet, that power comes with a price - a heavy inference cost. The combination of multi-step iterative denoising and massive Transformer calculations means a single generation often takes dozens of seconds, or more. Solving how to drastically cut inference latency and VRAM usage without sacrificing quality is the key to shipping these models in real-world products.

This article is divided into three parts: First, we break down the basics of DiT structures and mainstream acceleration methods. Next, we introduce anorak-inference, our proprietary inference framework. Finally, we’ll use our app called LookStar (currently in closed testing) to show how we used this framework to accelerate a complete “multi-image reference-to-image + image-to-video” workflow and deploy it on a single GPU.

I. DiT Models and Acceleration Methods

1.1 The Basic Structure of DiT

A typical DiT generation workflow consists of three components:

• Encoder: Encodes input (text prompts, reference images) into conditional representations. Text usually goes through a large encoder (like T5 or Gemma), while images are encoded into the latent space via VAE.

• Transformer (Backbone): DiT replaces the U-Net of earlier diffusion models with a Transformer to predict denoising in the latent space. Compared to U-Net, Transformers scale significantly better, allowing for model sizes with billions of parameters.

• Diffusion (Denoising Process): Starting from pure noise, the model iteratively denoises over multiple steps, eventually decoding back into an image or video.

1.2 Where are the Bottlenecks?

DiT inference overhead generally concentrates in two areas:

1. Multi-step Denoising: Every sample generated requires many denoising steps, and every step requires a full forward pass of the Transformer. The more steps, the longer the wait.

2. Transformer Computation: As sequence lengths grow (especially in video) and model sizes increase, the computational and memory overhead for attention and feed-forward (FFN) layers skyrockets.

Therefore, the core strategy for accelerating DiT is two-fold: reduce the number of denoising steps and lower the computation/memory cost of each individual Transformer step.

1.3 Mainstream Acceleration Methods

Based on those two pillars, the industry has developed several primary acceleration techniques:

• Distillation: Methods like DMD or Consistency models compress a denoising process that originally took dozens of steps down to just a few—or even one. This is the most direct and effective way to reduce step counts.

• Caching: By leveraging the similarity of features between adjacent denoising steps (e.g., temporal redundancy in attention/features), we can cache and reuse intermediate results, skipping repetitive calculations.

• Quantization: Reducing weights and activations from FP16/BF16 to FP8, INT8, or even NVFP4. This slashes VRAM usage and leverages the low-precision compute power of new hardware (like the FP8/NVFP4 on Blackwell).

• Multi-resolution / Cascaded Inference: Perform the bulk of the denoising at low resolution, then upscale and refine at high resolution. This puts the heaviest math where the resolution is smallest.

• Efficient Attention (Sparse / Linear Attention): Standard attention computation grows quadratically with sequence length, which is painfully expensive for long video sequences. Sparse Attention skips redundant calculations by only computing a subset of token pairs, while Linear Attention reduces complexity from quadratic to linear.

These methods aren’t mutually exclusive. In practice, you often need to combine them, making trade-offs based on the model and hardware—which is precisely the problem anorak-inference is built to solve.

II. anorak-inference: Anorak’s Proprietary DiT Inference and Orchestration Framework

anorak-inference isn’t just about serving one model efficiently—that’s a solved problem with tools like LightX2V or sglang. Instead, we’re tackling the gaps those tools haven’t covered:

1. Workflow Orchestration: How do you chain multiple disparate models (image-to-image, video generation, super-resolution, frame interpolation, face-swapping) into a single, production-ready workflow?

2. Single-GPU Multi-Model Deployment: How do you run that entire complex workflow on a single card?

We designed this as a model-agnostic, workflow-agnostic foundation where acceleration techniques can be plugged in. Its core design features:

• Unified Service Abstraction: Single-model pipelines and multi-model workflows use the same interface. To the service layer, they are identical—serving a “naked” model is just a one-step workflow. Adding new models or workflows follows the same path.

• Model-Agnostic and Extensible: Need a new model? Just implement a pipeline. Building a new workflow? Write a declarative spec that chains pipelines and transformation operators together. The framework isn’t tied to any specific model. Whether it’s open-source or trained/distilled via our anorak-dflow (which supports RL, SFT, and distillation), everything integrates seamlessly.

• Single-GPU Multi-Model VRAM Management: This is the key to running multi-model workflows. Take LookStar: Flux.2-klein and LTX-2 cannot fit on a single 80GB card simultaneously. The framework follows the declared step sequence, swaps unused models to pinned host memory, and fetches them back when needed—maintaining near-”GPU resident” latency. This mechanism allows the workflow to run on an H100 or squeeze into a 32GB RTX 5090.

• Plug-and-Play Acceleration: Distillation, quantization (built-in FP8 toolchain), multi-resolution inference, and various pre/post-processing ops are all composable components. The framework doesn’t force one acceleration method; it provides a unified base to host all of them.

In short: anorak-inference focuses on bridging the gap in “multi-model orchestration + single-card VRAM scheduling + pluggable acceleration,” while anorak-dflow handles the training and distillation. Together, they cover the entire chain from training to deployment.

III. Case Study: Accelerating LookStar

LookStar is the perfect example of using anorak-inference to accelerate a full-stack DiT workflow. Lookstar is an app where you can use the power of Generative Image and Video to dress yourself in a number of virtual clothing and see yourself swoon around in different environments looking fab. You take a selfie and choose your items and wait for your resulting video to bve generated. The generation process is split into two parts: Multi-image Reference-to-Image (i2i) and Image-to-Video (i2v).

3.1 Multi-image Reference-to-Image (i2i)

The goal: Synthesize an image of a person wearing a full set of items, given an Identity photo and multiple reference photos.

• 4-step Flux-Klein Model: Compresses the denoising process significantly.

• Multi-image Support: Through two rounds of inference, we support up to 9 reference images by combining the Selfie Photo with item references.

• Resolution Downscaling: Since reference items don’t need the same resolution as the final image, we scale them down, drastically reducing compute without affecting visual quality.

Item Reference Image Resolution Time (2-Step Flow) seconds

Full resolution 15.35

Short side 512px (Production) 7.19

• FaceFusion for Consistency: Since reference-based generation sometimes drifts on facial consistency, we added FaceFusion post-generation to align the output with the Identify photo.

Face Change Method: ArcFace Distance (smaller the better)

No face change: 0.800

hyperswap, weight 0.5: 0.264

inswapper, weight 1.0: 0.195

inswapper, weight 0.5（Production): 0.162

Here are two of our developers using this method and as you can see with the above improvements both speed, item consistency and face likeness is beating out NanoBanana2:

3.2 Image-to-Video (i2v)

We currently use Lightricks LTX2.5 Video model to generate our videos.

• 8-step LTX-Video: Uses variable resolution (low-res denoising first, then upsample) to keep heavy lifting in the low-res stage.

• 1-Step High-Res Denoising: In the high-res stage, we cut denoising from 3 steps to 1. Testing shows no perceptible difference in output, but significant compute savings.

• Optimization: Using torch.compile() and FP8 scaled-matmul to accelerate the denoising graph.

3.3 System Level: Getting the Workflow on One Card

The biggest constraint is VRAM. We did two critical things:

1. FP8 Quantization for Gemma3: The text encoder consumes significant VRAM, so we quantized it to FP8.

2. Pinned Memory Management: Using anorak-inference ops, we offload inactive models to pinned memory and swap them in only when needed.

Thanks to these optimizations, the entire LookStar workflow runs not just on an H100, but on a 32GB RTX 5090 too! Although this is a great start - we’ve found LTX2.5 is still a relatively weak model compared to the newer Minimax-H3, and we are currently hard at work making optimisations for that to be production ready.

We Are Just Getting Started

LookStar is just the first application of anorak-inference. By combining distillation to reduce steps, quantization and multi-resolution to reduce per-step costs, and clever VRAM scheduling to fit on one card, we turned a heavy generation workflow into one that is deployable on a single GPU with controlled latency. More importantly, these capabilities are framework-level and reusable. As our optimisation framework evolves, we can replicate these acceleration wins across more models and business scenarios. This is just the beginning.