> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sglang.io/llms.txt
> Use this file to discover all available pages before exploring further.

# MiniMax-H3

> Run native MiniMax-H3 video-and-audio generation with SGLang Diffusion.

## 1. Quick start

Follow the [SGLang Diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation.md),
then choose a verified recipe below. Docker is recommended where the picker offers it.
Setup changes the deployment; Server and Request expose
orthogonal startup and sampling choices.

> [!NOTE]
>
> The generated Server command already includes the recommended encoder policy.
> Change a Server option only for a deliberate trade-off; Request options do not
> reload the model.

The Docker form uses the image's bundled diffusion dependencies without reinstalling
them at startup. For conditioned requests, set **Host media directory**
under **Variables**; the builder mounts it read-only at `/data/minimax-h3`.
AMD and Grace Blackwell currently offer the Python form; the other NVIDIA
recipes also offer Docker.

To use ModelScope through the same normal `sglang serve` path, prefix the copied
command with `SGLANG_USE_MODELSCOPE=true` and replace the model path with
`MiniMax/MiniMax-H3`. Keep the selected variant and topology flags unchanged.

For platform-specific installation details, see the
[SGLang Diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation.md).

## 2. Model capabilities

[MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) is a native joint video-and-audio model for text-to-video-and-audio, first/last-frame control, and multimodal reference conditioning. Its main strength is producing the picture and stereo soundtrack together, so speech, music, ambient sound, and visible events can stay aligned without a separate audio-generation pass.

Choose H3 when synchronized audiovisual output or reference-driven generation matters more than a lightweight deployment. The released recipe targets a 768-pixel short edge at 24 fps for 4–15 seconds, and its capabilities are split across two checkpoint partitions; serving every mode therefore requires separate FL2VA and Ref2VA deployments.

| Task | `task` value | Conditioning |
| - | - | - |
| Text to video and audio | `t2va` | Text prompt only |
| First/last frame to video and audio | `fl2va` | First frame, last frame, or both |
| Reference to video and audio | `ref2va` | Image, video, and audio references, optionally combined with first/last keyframes |

Video-to-video (V2V) is a supported `ref2va` use case, not a fourth task
value. Run the `Ref2VA` partition and provide a video reference in
`conditions`. A hybrid `ref2va` request may also include the same ordered
first/last keyframes accepted by `fl2va`, but it must still contain at least one
reference condition.

Use the selected Hub's root model ID: `MiniMaxAI/MiniMax-H3` on Hugging Face
or `MiniMax/MiniMax-H3` on ModelScope. Select the checkpoint variant with
`--model-variant`: `fl2va` serves both `t2va` and `fl2va`, while `ref2va`
serves reference-conditioned requests. SGLang owns the checkpoint-directory
mapping; do not point `--model-path` at a manually downloaded subdirectory.
The same FL2VA / Ref2VA split applies in
[ComfyUI integrated mode](#comfyui-integrated-mode).

> [!WARNING]
>
> Review the license and usage terms in the MiniMax-H3 model card before production or commercial use. SGLang support does not grant additional model usage rights.

## 3. Deployment details

The builder accepts legal custom GPU counts and topologies, marking them
**Unverified** until the exact recipe has completed end-to-end validation.
Static H3 head or partition violations disable Copy before they reach
`sglang serve`.

### Grace Blackwell

Select **GB300** or **GB200** in the builder. The default is one Linux ARM64
host with four GPUs, Ulysses4, and resident weights. This follows the
[four-GPU compute-tray layout](https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html);
an NVL72 rack is not a single 72-GPU process host.

For two four-GPU hosts, set **Nodes = 2** and **GPUs / node = 4**. The builder
emits Ulysses4 inside each host, Ring2 across hosts, and replicated encoders.
Set the same reachable **Head address** on both hosts and use node ranks 0 and

1. Run the generated command once per host, with the same SGLang revision and
   dependencies. Let `sglang serve` resolve and download the selected weights.

GB200 is a **derived, unverified recipe**, not a measured GB300-equivalent
performance claim. Keep the native precision and platform-default attention
for the baseline; the B200/B300 quantization measurements and H200 Cache-DiT
quality audit do not establish coverage on Grace Blackwell.

GB300 end-to-end coverage currently includes single-host FL2VA text-to-video
with audio, native precision, eager execution, and `quality="exact"`.
Ref2VA and multi-host commands remain **Unverified**. See the
[GB300 measurements](#gb300-single-host) for the tested workload.

### Checkpoint and adapter formats

Start with the command emitted by the builder. Every row below is an overlay on
the same native SGLang pipeline; component repositories contribute their own
config and weights, while weight files retain the base component config. Storage
layout and inference behavior are separate contracts: for example, a PEFT file
may be either a normal style adapter or a timestep-distilled Turbo adapter.

| Scope | Format or variant | Add to the base command | Contract |
| - | - | - | - |
| Full model | Official mixed BF16/FP32, CFG-distilled | `--model-variant fl2va` or `--model-variant ref2va` | Lossless reference and consistency GT path. CFG distillation removes the negative branch; it is not the few-step timestep distillation used by Turbo releases. |
| DiT | Official Diffusers component layout | `--component-paths.transformer MiniMaxAI/MiniMax-H3/transformer` (`fl2va`) or `.../transformer_ref` (`ref2va`) | Loads the official component through the native SGLang graph; no Diffusers runtime fallback. |
| DiT | [AdaLN-pruned Diffusers component](https://huggingface.co/multimodalart/MiniMax-H3-Pruned) | `--component-paths.transformer multimodalart/MiniMax-H3-Pruned/transformer` or `.../transformer_ref` | Approximate curve-AdaLN architecture; its config and basis metadata are loaded natively. |
| DiT | [Full or AdaLN-pruned](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models), or [LoRA-merged/remixed](https://huggingface.co/FX-FeiHou/MiniMax-H3-Remix) BF16 safetensors | `--component-weights-paths.transformer OWNER/REPO/path/FILE.safetensors` | Weight-only override for a native full/pruned H3 layout. Match the FL2VA/Ref2VA partition; pruned, merged, or dtype-converted exports are approximate, and any author-specific sampler remains a separate requirement. |
| DiT | FL/Ref hybrid fine-tunes, including [Singularity v1.3](https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity) (full or pruned INT8) | Replace `--model-variant` with `hybrid`; add `--component-weights-paths.transformer OWNER/REPO/FILE.safetensors` | Explicit merged-weight deployment serving `t2va`, `fl2va`, and `ref2va` on one native pipeline. Quantization and pruning are detected from the file; INT8 requires `comfy-kitchen`. See [Singularity](#singularity-hybrid-weights). |
| DiT | Comfy FP8 or self-describing MXFP8 safetensors | `--component-weights-paths.transformer OWNER/REPO/path/FILE.safetensors` | Per-layer metadata selects static/dynamic FP8 or MXFP8 automatically. |
| DiT | ConvRot INT8, W4A8, W4A4, or mixed W4A4+INT8 safetensors ([INT8](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models), [W4A8](https://huggingface.co/Winnougan/MiniMax-H3-INT4_Convrot_ComfyUI), [W4A4](https://huggingface.co/Merserk/MiniMax-H3-INT4-ConvRot)) | `--component-weights-paths.transformer OWNER/REPO/path/FILE.safetensors` | Auto-detected. INT8 layers run on SGLang's JIT-compiled ConvRot ops on CC 9.0, 10.0, 12.0 and 12.1 and on `comfy-kitchen` elsewhere; W4A8 and W4A4 layers require `comfy-kitchen`. TP must preserve each file's ConvRot group boundaries. |
| DiT | [NVFP4, optionally mixed with INT8 or FP8](https://huggingface.co/Abiray/Minimax-H3-nvfp4-INT4-INT8-Convrot) | `--component-weights-paths.transformer OWNER/REPO/path/FILE.safetensors` | Auto-detected; NVFP4 execution requires NVIDIA compute capability 10.0+. |
| DiT | [AutoRound W4A16 component](https://huggingface.co/Ar4ikov/MiniMax-H3-transformer-W4A16-RTN) | `--component-paths.transformer Ar4ikov/MiniMax-H3-transformer-W4A16-RTN` | Self-describing Diffusers component; SGLang reuses the SRT GPTQ/Marlin backend. The linked export is FL2VA. |
| DiT | GGUF, full or AdaLN-pruned ([full](https://huggingface.co/leejet/MiniMax-H3-GGUF), [pruned](https://huggingface.co/unsloth/MiniMax-H3-GGUF)) | `--component-weights-paths.transformer OWNER/REPO/FILE.gguf` | CUDA capacity path; aligned TP and layerwise offload are supported, FSDP and LoRA are not. |
| Text encoder | Architecture-compatible Qwen3-VL BF16 finetune ([Heretic example](https://huggingface.co/llmfan46/Qwen3-VL-32B-Instruct-ultra-uncensored-heretic)) | `--component-paths.text_encoder OWNER/REPO` | Reuses the native H3 extractor, including its vision tower and layer-50 hidden-state selection. Finetuning changes conditioning, not the sampling schedule. |
| Text encoder | [Serialized FP8 component](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8) | `--component-paths.text_encoder Qwen/Qwen3-VL-32B-Instruct-FP8` | Only eligible language-model linears use FP8; embeddings, norms, and the vision tower keep their declared precision. |
| Text encoder | ConvRot INT8, W4A8, or W4A4 safetensors ([INT8](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders), [Heretic INT8](https://huggingface.co/ethanfel/Qwen3-VL-32B-Ultra-Heretic-H3-ComfyUI-INT8-ConvRot), [W4A8](https://huggingface.co/Winnougan/MiniMax-H3-INT4_Convrot_ComfyUI), [W4A4](https://huggingface.co/Merserk/MiniMax-H3-INT4-ConvRot)) | `--component-weights-paths.text_encoder OWNER/REPO/path/FILE.safetensors` | Auto-detected. INT8 layers run on SGLang's JIT-compiled ConvRot ops on CC 9.0, 10.0, 12.0 and 12.1 and on `comfy-kitchen` elsewhere; W4A8 and W4A4 layers require `comfy-kitchen`. INT8/W4A8 files may include a scalar-scale INT8 embedding, which requires `comfy-kitchen`. Unmarked tensors retain their declared precision. |
| Text encoder | Comfy NVFP4 with dynamic activation quantization ([Heretic example](https://huggingface.co/sakamakismile/Qwen3-VL-32B-Heretic-MiniMax-H3-NVFP4)) | `--component-weights-paths.text_encoder OWNER/REPO/path/FILE.safetensors` | Requires `comfy-kitchen` and NVIDIA compute capability 10.0+. Per-layer metadata selects native NVFP4 matmul; scalar/row-wise INT8 embeddings remain packed. |
| Text encoder | [NVFP4-AWQ](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders) or [Quanto qint8](https://huggingface.co/DeepBeepMeep/MiniMax-H3/tree/main/Qwen3-VL-32B-Instruct) safetensors | `--component-weights-paths.text_encoder OWNER/REPO/path/FILE.safetensors` | Memory-oriented formats: compressed storage is restored, then each active matrix uses BF16/FP16 compute. |
| Text encoder | [GGUF Qwen3-VL](https://huggingface.co/DeepBeepMeep/MiniMax-H3/tree/main/Qwen3-VL-32B-Instruct) | `--component-weights-paths.text_encoder OWNER/REPO/FILE.gguf` | CUDA capacity path with encoder TP/layerwise support; encoder FSDP is not supported. |
| Text encoder | [Compact Qwen3-VL 4B/8B + ClipProj](https://huggingface.co/NicoLab28/ClipProj-MiniMax-H3) | `--component-paths.text_encoder ENCODER_REPO --component-paths.conditioning_projection PROJECTION.safetensors` | Approximate conditioning replacement. A separate weight-only override may quantize the selected small encoder. |
| DiT or adapter | Timestep-distilled Turbo, as merged weights or LoRA ([Larry](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora), [LightX2V](https://huggingface.co/lightx2v/Minimax-h3-Turbo), [merged Ref2VA INT8 example](https://huggingface.co/PulpCut/MiniMax-H3-Ref2VA-Turbo-INT8-ConvRot)) | Use `--component-weights-paths.transformer ...` for merged weights, or `--lora-path OWNER/REPO --lora-weight-name FILE --lora-merge-mode auto` for LoRA | Few-step semantic variant. Pin the exact FL2VA/Ref2VA file and its NFE/sigma schedule, scale, and alpha; storage-format detection does not infer sampling behavior. See [LoRA recipes](#5-lora-recipes). |
| Adapter | Style, subject, or behavior LoRA ([example](https://huggingface.co/fal/MiniMax-H3-Realism-People-LoRA)) | `--lora-path OWNER/REPO [--lora-weight-name FILE] --lora-merge-mode auto` | Native fused and Diffusers/PEFT layouts are normalized at load time. Keep the base schedule unless the author specifies another one, and preserve any trigger phrase, scale, and alpha metadata. |

The rows compose rather than enumerate every cross-product. A Turbo-merged INT8
ConvRot checkpoint, for example, must satisfy both the Turbo sampling contract
and the ConvRot storage/backend contract.

H3's Qwen3-VL text encoder also handles image understanding; there is no separate
`image_encoder` component. Select the conditioning checkpoint, not an optional
generation-tail file for prompt rewriting. The `uncensored` or `Heretic` label
describes a weight modification, not a separate loader or a guarantee of output quality.

For H3, the registered component names are `transformer`, `text_encoder`,
`video_vae`, and `audio_vae`. The shorter `--transformer-weights-path` and
`--text-encoder-path` aliases remain supported. `conditioning_projection` is an
H3 text-encoder sidecar key, not a standalone model component. Plain
video/audio VAE safetensors can use `--component-weights-paths.video_vae` or
`--component-weights-paths.audio_vae`, but SGLang does not currently advertise
a native quantized H3 VAE format.

Pre-quantized files are self-describing: do not combine those rows with
`--quantization` or `--component-quantizations.*`. Only byte-identical official
full weights—whether loaded from the original model, Diffusers component, or a
weight-only layout—belong to the consistency GT. Pruned, quantized,
compact-encoder, and LoRA routes are outside that baseline and the audited
`quality="high"` contract. Packed and per-layer mixed formats reject FSDP unless
their row says otherwise; see
[Quantization](https://docs.sglang.io/docs/sglang-diffusion/quantization.md) for backend-wide constraints.

LoRA tensors alone do not make an execution-coupled release portable. Sparse-
attention/SLA adapters and causal-streaming adapters such as RAVEN also require
their matching attention or streaming pipeline; they are not standard H3 LoRA
overlays in SGLang. Likewise, a remixed checkpoint that prescribes a custom
sampler is only covered when that sampler contract can be reproduced—the fact
that its safetensors layout loads is not sufficient.

For a four-card H200 host, keep the full BF16/FP32 model resident by default.
The model fits without FSDP, so this path avoids the per-block parameter
all-gathers of the memory-oriented FSDP profile:

```bash 4×H200 resident theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010
```

Pure Ulysses4 is also the faster measured topology on H200, not just a
capacity default. The 4×H100 TP2 + Ulysses2 recipe below fits on 141 GB H200
cards, but it replaces the Ulysses all-to-all exchange with two per-block
tensor-parallel all-reduces and measured slower end-to-end, at about 30 GB
lower peak memory per GPU. See the **H200 topology comparison** in the
Benchmarks section for the measured numbers; treat TP2 + Ulysses2 on H200 as
a deliberate memory trade, not a latency default.

For 4×H100 80 GB, balance the large packed activation with resident weight
sharding. TP2 + Ulysses2 was the fastest measured lossless topology while the
Qwen encoder still folds across all four GPUs:

```bash 4×H100 fastest theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010
```

Pure Ulysses4 could not keep the full pipeline resident on 80 GB H100s. Use
`--tp-size 4 --ulysses-degree 1` when lower resident memory matters more than
the last few percent of latency. FSDP remains a verified capacity option, but
its per-block weight all-gathers do not make it the H100 speed default:

```bash 4×H100 FSDP capacity theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --encoder-parallel auto \
  --performance-mode speed \
  --use-fsdp-inference true \
  --port 30010
```

For a two-card RTX 5090 host, use TP2 and keep 20 DiT blocks
resident. Layerwise placement is lossless: it changes parameter placement and
transfer scheduling, not the BF16/FP32 denoising or VAE math. This is the
fastest measured 32 GB operating point:

```bash 2×RTX 5090 fastest lossless theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 2 \
  --tp-size 2 \
  --ulysses-degree 1 \
  --encoder-parallel auto \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 20 \
  --enable-torch-compile false \
  --port 30010
```

The DiT residency and prefetch knobs apply only to the repeatedly executed DiT
blocks. The text encoder and the video VAE decoder blocks use one-layer
prefetch with zero resident layers. The video VAE encoder stays resident
because its indexed down blocks cannot host executable layerwise hooks; the
roughly 577 MiB audio VAE also stays resident because offloading it only adds
transfer overhead. This exact recipe was validated on
2× RTX 5090 (32 GB each) and a 377 GiB host; use a 384 GiB-class machine. The
latency and memory comparison is collected in the benchmark section below.

For a single 24 GB consumer card (RTX 4090), stream the DiT and text encoder
and quantize DiT linear layers online with `convrot_int8`. Keep `vae` out of
`--layerwise-offload-components`: putting the VAE decoder in layerwise
offload re-streams about 9 GiB on each of 167 decode tiles. Default
attention stays `fa` (exact). Approximate backends are opt-in; see
[Attention Backends](https://docs.sglang.io/docs/sglang-diffusion/attention_backends.md#sage-then-sol-hybrid).
On the RTX 4090 (CC 8.9) `convrot_int8` runs on `comfy-kitchen`, so install it
first (`pip install comfy-kitchen`); on CC 9.0, 10.0, 12.0 and 12.1 it uses
SGLang's JIT-compiled ConvRot ops instead (built once per machine on first use).

```bash 1×RTX 4090 24GB theme={null}
sglang generate \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --quantization convrot_int8 \
  --attention-backend fa \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 0 \
  --enable-torch-compile false \
  --prompt "A cat walking on a sunny beach, gentle waves." \
  --save-output
```

The same flags work on `sglang serve`. Drop `--quantization` for the BF16
baseline; everything else stays identical. GPU peak stays about 18 GB
either way because streaming offload is set by the offload buffers and VAE
decode, not the weight dtype.

To skip DiT steps on 4090 while keeping `fa`, add Spectrum to the generate
command above. Do not combine it with `quality="high"` or Cache-DiT.
On this card, 50-step BF16+FA (**1008.6 s**) drops to **285.3 s (3.53×)**
with exact attention, up to **219.0 s (4.61×)** with Sage→Sol hybrid +
Spectrum. See the 4090 table in Benchmarks.

```bash theme={null}
  --enable-spectrum \
  --spectrum-warmup-steps 11 \
  --spectrum-window-size 5 \
  --spectrum-flex-window 1.0
```

The first launch resolves every selected source through the normal Hub path. If
a repository requires authentication, export a Hugging Face token in the
server environment; no manual pre-download is required.

For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager.
The current `torch.compile` path changes the model's numerical output, so no
recommended lossless preset enables it implicitly. An explicit
`--enable-torch-compile true` remains available for controlled experiments, but
do not use it to generate consistency ground truth.

### ComfyUI integrated mode

The [ComfyUI plugin](https://docs.sglang.io/docs/sglang-diffusion/comfyui.md) can run H3 as a per-step
DiT backend. ComfyUI keeps CLIP, VAE, and the sampler loop. SGLang loads only
the transformer under `--comfyui-mode`.

Copy `python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion` into ComfyUI
`custom_nodes/`. Load a shipped graph and set `SGLDOptions.model_type` to
`minimax_h3`:

| Graph | Task | DiT filename |
| - | - | - |
| `minimax_h3_t2v_sgld.json` | T2V / I2VA / FL2VA | `minimax_h3_fl2va_*.safetensors` |
| `minimax_h3_r2v_sgld.json` | Reference-to-video | `minimax_h3_ref2va_*.safetensors` |
| `minimax_h3_t2v_sgld_upscaler.json` | Low-res then 3D ×2 | `fl2va` DiT |

Pruned BF16 / FP8 files from [Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models)
are the usual loader inputs. GGUF goes in `SGLDOptions.transformer_weights_path`.
On a 24 GB card put the 32B CLIP on CPU and enable DiT layerwise offload; use
`convrot_int8` or GGUF rather than the full unpruned BF16 DiT.

### Singularity hybrid weights

[Singularity](https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity) is an
FL/Ref fusion fine-tune, not a complete pipeline repository. Keep the official
model ID for the encoders and VAEs, and select one exact transformer file:

```bash 2×B300 hybrid checkpoint theme={null}
pip install comfy-kitchen

WEIGHTS=WarmBloodAban/Minimax-h3_Singularity/Minimax-h3_Singularity_ref2va_Pruned_v1.3_int8.safetensors
sglang serve --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant hybrid \
  --component-weights-paths.transformer "$WEIGHTS" \
  --num-gpus 2 --ulysses-degree 2 --performance-mode speed \
  --port 30000
```

For the full INT8 checkpoint, replace the filename with
`Minimax-h3_Singularity_ref2va_v1.3_int8.safetensors`. SGLang downloads the
selected file automatically. Do not add an online quantization override.
`hybrid` requires an explicit weight override; it does not convert the official
Ref2VA weights into a multimode model. Request `task` and `conditions` retain
their [usual meanings](#4-generate-video-and-audio), including `conditions: []`
for `t2va` and keyframes for `fl2va`.

Start with `quality: "lossless"` and the standard 50-point schedule. Here
`exact` disables additional request-time approximations; it does not undo
INT8 quantization, pruning, or fine-tuning. The author's recommended Ref2V
Turbo LoRA is a separate optional adapter, not an automatically inferred
4-step schedule. Reference fidelity can differ from the official model; see
the [author's guidance](https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/discussions/1)
before applying the [LoRA sampling contract](#5-lora-recipes).

### Advanced: precomputed AdaLN cache

The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
13B H3 parameters are AdaLN branches whose outputs can be precomputed for
inference. The public base checkpoint contains the original branches, not a
ready-to-use cache. SGLang therefore keeps the standard path as the default.

> [!WARNING]
>
> This is an experimental deployment path. It is intentionally disabled unless
> you provide an explicitly generated cache; end-to-end numerical and peak-memory
> validation remains required before using it in production.

When an inference-only deployment has a fixed sampling schedule, build a cache
from the already materialized transformer directory on CUDA, then pass it to
the usual `sglang serve` command. This does not alter the denoising formula:
the cache stores the BF16 outputs of the original AdaLN linears.

```bash Command theme={null}
python -m sglang.multimodal_gen.tools.build_minimax_h3_adaln_cache \
  --transformer-path "$TRANSFORMER_PATH" \
  --model-variant fl2va \
  --mode t2va \
  --num-inference-steps 50 \
  --flow-shift 12 \
  --audio-flow-shift 3 \
  --output /models/minimax-h3-fl2va-adaln-50step.safetensors

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --minimax-h3-adaln-cache-path /models/minimax-h3-fl2va-adaln-50step.safetensors \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --port 30010
```

`$TRANSFORMER_PATH` is the `FL2VA/transformer` or `Ref2VA/transformer`
directory in the normal SGLang/Hugging Face snapshot; the builder never
downloads a second copy. A cache only covers the scheduler settings used to
create it, including its mode, step count, flow shifts, and condition noise
values. SGLang rejects a request outside that coverage instead of silently
changing conditioning. Cache mode supports the matching unquantized checkpoint
only.

### Advanced: online AdaLN rebuild with a host cache

`--minimax-h3-adaln-online true` needs no prebuilt artifact: the server drops
the 24.2 GiB of `adaln_proj` weights from the GPU and computes each request's
AdaLN outputs from the checkpoint on demand, bit-exact with the resident-weight
path. Because a rebuild pass streams the whole 24.2 GiB, the first request of
every new `(task shape, num_inference_steps, flow_shift, audio_flow_shift)`
combination pays several seconds; after that the plans are served from a
64-slot GPU slab (LRU per plan) backed by a pinned host cache
(`--minimax-h3-adaln-host-cache-gb`, LRU per schedule, default 8 GB per
rank), so mixed-schedule serving does not re-read the checkpoint. Plan sets
that exceed the host budget are simply recomputed on their next occurrence.
Expert escape hatches live in environment variables:
`SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS` resizes the GPU slab (needed
only beyond 65 inference steps) and `SGLANG_DIFFUSION_MINIMAX_H3_ADALN_FP32`
computes the one-time projections in fp32 (experimental; not bit-comparable
to resident weights, validate end-to-end before production). LoRA adapters
that modify `adaln_proj` are rejected in both cache modes rather than
silently ignored.

Both cache modes hold values derived from `adaln_proj`, so a runtime weight
update is only accepted when the cache can follow it: online mode takes a disk
update whose target directory carries native `adaln_proj` safetensors, and
rejects everything else (tensor updates, and directories without those
tensors) before a single weight is written. A sidecar is built offline and
cannot be regenerated in the server, so weight updates are rejected outright;
rebuild the sidecar against the new weights and restart.

### Serve MiniMax-H3 on Ascend NPUs

For Ascend NPU, follow the
[NPU installation guide](https://docs.sglang.io/docs/hardware-platforms/ascend-npus/getting-started/installation.md)
before starting the server.

The Ascend commands below explicitly enable the Cache-DiT configuration used
for the reported performance measurements. Remove these `SGLANG_CACHE_DIT_*`
variables to use lossless denoising. See the **Ascend NPU topology comparison**
in the Benchmarks section for the measured eight- and four-NPU latency.

The measured latency configuration also passes `--dit-cpu-offload false` to
keep the transformer resident on the NPUs. Omit this flag when lower device
memory usage is more important than avoiding CPU-to-NPU transfer latency.

For an eight-NPU host, the validated topology is TP2 + SP4 with Laser
Attention. Use Ascend Flash Attention by replacing `laser_attn` with `fa`.

```bash 8-NPU theme={null}
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-type diffusion \
  --model-variant fl2va \
  --dit-cpu-offload false \
  --num-gpus 8 \
  --tp-size 2 \
  --sp-degree 4 \
  --attention-backend laser_attn \
  --port 30088 \
  --component-residency text_encoder=layerwise-offload
```

For a four-NPU host, use TP2 + SP2:

```bash 4-NPU theme={null}
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-type diffusion \
  --model-variant fl2va \
  --dit-cpu-offload false \
  --num-gpus 4 \
  --tp-size 2 \
  --sp-degree 2 \
  --attention-backend laser_attn \
  --port 30088 \
  --component-residency text_encoder=layerwise-offload
```

## 4. Generate video and audio

`num_inference_steps: N` executes **N denoising updates** over N+1 sigma
points, including the terminal zero. Reference implementations that count
sigma points need N+1 to reproduce this schedule. Older SGLang versions ran
N-1 updates for the same request; use N-1 here to preserve their sampling
trajectory. Changing the step count changes the trajectory, not just latency.

MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
generation mode below, submit a job, poll its status, and then download the
completed MP4.

**T2VA**

MiniMax-H3 supports output durations from 4 through 15 seconds, inclusive. The
following request keeps the verified 5-second profile at a 768-pixel short
edge. MiniMax-H3 resolves the aligned output canvas and frame count from
`target`.

```bash Command theme={null}
video_id=$(
  curl -sS -X POST http://127.0.0.1:30010/v1/videos \
    -H "Content-Type: application/json" \
    -d '{
      "model": "MiniMaxAI/MiniMax-H3",
      "prompt": "At night, while their owner sleeps in a bedroom, three cats march in loudly playing tiny brass instruments, then abruptly file out.",
      "seconds": 5,
      "task": "t2va",
      "conditions": [],
      "target": {
        "short_edge": 768,
        "aspect_ratio": "16:9",
        "duration_seconds": 5.0
      },
      "num_outputs_per_prompt": 1,
      "num_inference_steps": 50,
      "flow_shift": 12.0,
      "audio_flow_shift": 3.0,
      "seed": 1101
    }' |
    jq -r '.id'
)

while true; do
  status=$(curl -sS "http://127.0.0.1:30010/v1/videos/${video_id}" | jq -r '.status')
  [ "$status" = "completed" ] && break
  [ "$status" = "failed" ] && exit 1
  sleep 1
done

curl -sS -L "http://127.0.0.1:30010/v1/videos/${video_id}/content" \
  -o minimax-h3-t2va.mp4
```

The output contract is an MP4 containing H.264 video at 24 fps and one AAC stereo audio stream at 32 kHz.

**FL2VA**

For `fl2va`, provide one or two image conditions with role `keyframe`. The supported frame-index sets are `[0]`, `[-1]`, and `[0, -1]`.

The following request uses one server-local first frame. Use
`frame_index: -1` for a last frame, or include both entries for first-and-last
conditioning.

Choose FL2VA when the supplied image should be the actual first or last frame
of the generated clip. Use image-based Ref2VA instead when the image should
guide identity, style, or composition without being preserved as an endpoint;
Ref2VA may recompose or crop the reference.

```bash Command theme={null}
curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "The supplied frame continues with calm, natural motion and synchronized ambient sound.",
    "seconds": 5,
    "task": "fl2va",
    "conditions": [
      {
        "type": "image",
        "uri": "file:///data/minimax-h3/first-frame.png",
        "role": "keyframe",
        "frame_index": 0
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "auto",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 2101
  }'
```

**V2V**

V2V uses the reference-conditioning weights. Launch the server with
`--model-variant ref2va`, keep the request `task` set to `ref2va`, and provide a video
reference in `conditions`. There is no separate `v2v` task value.

Use `type: "video"` when the input may be silent. If the file has a soundtrack,
H3 also uses it as an audio reference. Use `type: "video_audio"` only when both
streams are required; that form rejects an input without audio. The prompt tag
for the visual stream is `<Video 1>`; an available soundtrack is exposed as
`<Audio 1>`.

> [!NOTE]
>
> Ref2VA treats the input video as reference material, not as a pixel-aligned
> edit source. It can resynthesize or reorder motion and cuts, and it does not
> expose a denoising-strength control. Do not rely on it to preserve every source
> frame or exact timing.

Set `conditions[].start_time_seconds` to select a segment from a longer source.
The default is `0`. SGLang seeks the visual stream and soundtrack to the same
offset, then decodes at most the requested target duration in one pass; the
source is not re-encoded into an intermediate clip.

```bash Command theme={null}
curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "Follow the motion and appearance of <Video 1>, changing the setting to a moonlit bedroom while preserving coherent timing.",
    "seconds": 5,
    "task": "ref2va",
    "conditions": [
      {
        "type": "video",
        "uri": "file:///data/minimax-h3/input.mp4",
        "role": "reference",
        "start_time_seconds": 35.0
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "16:9",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 4101
  }'
```

Use `conditions[].uri` for H3 V2V. The generic top-level `video_path`,
`video_url`, and `video_reference` upload fields are not lowered into H3
reference conditions.

**Multimodal Ref2VA**

For `ref2va`, first launch the reference-conditioning capability with
`--model-variant ref2va`, then provide conditions with role `reference`.
Image, video, and audio references can be combined. Material tags in the
prompt use the one-based order for each modality.

An image condition here is semantic reference material rather than a
pixel-aligned first frame. Use the FL2VA tab when animating a screenshot from
that exact starting composition.

Set `reference_resize_mode` on the request to choose how reference images are
resized before encoding:

* `diffusers` (default) scales the short edge to 2048 pixels, including upscaling
  small inputs. This preserves the original preprocessing behavior.
* `max` only downsizes references whose short edge exceeds 2048 pixels.
* `match` downsizes to the target canvas pixel area when needed, without
  upscaling smaller references. Use this with the
  [ModelTC Ref2VA Turbo LoRA](https://github.com/ModelTC/Minimax-H3-Turbo#note-on-reference-image-resizing),
  whose training uses this policy.

All modes scale without cropping, then round each dimension to the nearest
multiple of 32. This rounding can slightly change the aspect ratio. The option
affects Ref2VA image references, not FL2VA keyframes, video, or audio references.
Changing it changes the conditioning input, so keep it fixed for quality and
performance comparisons; the server does not infer it from the adapter name.

```bash Command theme={null}
curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "Use <Picture 1> as the visual subject and <Audio 1> as the sound reference, with coherent natural motion.",
    "seconds": 5,
    "task": "ref2va",
    "reference_resize_mode": "diffusers",
    "conditions": [
      {
        "type": "image",
        "uri": "file:///data/minimax-h3/reference.png",
        "role": "reference"
      },
      {
        "type": "audio",
        "uri": "file:///data/minimax-h3/reference.mp3",
        "role": "reference"
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "auto",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 3101
  }'
```

Poll and download any conditioned request with the same job-status and
content endpoints used in the T2VA example. Server-local `file://` URIs must
refer to files visible inside the SGLang server environment.

## 5. LoRA recipes

H3 accepts native fused adapters, standard Diffusers/PEFT adapters, and Kohya
`lora_unet_blocks_*` exports from `networks.lora_minimax_h3`. Native adapters
target modules such as `blocks.*.attn.qkv_proj`; PEFT adapters may instead
provide separate `to_q`, `to_k`, and `to_v` projections and the `default`
adapter namespace. SGLang normalizes all three layouts.

The following pinned FL2VA adapters have distinct purposes:

| Recipe | Repository and pinned file | Request setting | Prompt requirement |
| - | - | - | - |
| Recommended speed/quality balance | [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora), `minimax_h3_turbo_v4_step600_ema.safetensors` | `num_inference_steps: 8` (8 denoiser evaluations), `lora_scale: 1.0` | None |
| Most aggressive speed preset (standard PEFT layout) | [`lightx2v/Minimax-h3-Turbo`](https://huggingface.co/lightx2v/Minimax-h3-Turbo), `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | `num_inference_steps: 4` (4 denoiser evaluations), `lora_scale: 1.0`, `lora_alpha: 8` | None |
| Realistic people style | [`fal/MiniMax-H3-Realism-People-LoRA`](https://huggingface.co/fal/MiniMax-H3-Realism-People-LoRA), `h3-realism-people-t2v-i2v-r2v.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.7` | Include `r34l1sm` in the prompt |
| Spatial and physics | [`Jojocodex/minimax-h3-spatial-physics-lora`](https://huggingface.co/Jojocodex/minimax-h3-spatial-physics-lora), `wushu_spatial_physics_clean_3000_pruned.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.4` | None. Describe collisions, stacking, and gravity-consistent motion in the prompt |
| Wushu action | [`Jojocodex/minimax-h3-wushu-action-lora`](https://huggingface.co/Jojocodex/minimax-h3-wushu-action-lora), `wushu_action_h3_lora_v5_3000_pruned.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.8` | Put `wushu_action` at the start of the prompt |
| Camera motion | [`Jojocodex/minimax-h3-yunjing-lora`](https://huggingface.co/Jojocodex/minimax-h3-yunjing-lora), `camera_motion_h3_lora_v1_3000_pruned.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.8` | Put `camera motion` at the start of the prompt |
| Looping sketch anime | [`Inner-Reflections/MiniMax-H3-Looping-Sketch-Anime`](https://huggingface.co/Inner-Reflections/MiniMax-H3-Looping-Sketch-Anime), `minimax_h3_looping_sketch_anime_v1.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 1.0` | Include the author's hand-drawn sketch style phrase in the prompt |
| 3D-to-real detail slider | [`siraxe/3d_to_real_detail_slider_H3`](https://huggingface.co/siraxe/3d_to_real_detail_slider_H3), `3d_to_real_H3_detail_slider_bf16.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 1.0` | None. Sweep the scale; this is a slider, not a fixed style |

The H3 request field controls the number of denoiser evaluations. An 8-step
adapter uses `8`, and a 4-step adapter uses `4`; the sigma schedule includes
one additional terminal zero.

Every recipe uses the same launch shape. Pinning the filename is required for
repositories that publish multiple revisions, and is also recommended for a
reproducible single-file recipe:

```bash Command theme={null}
LORA_REPO=larryvrh/MiniMax-H3-Turbo-Lora
LORA_FILE=minimax_h3_turbo_v4_step600_ema.safetensors
LORA_NAME=h3-turbo-v4
LORA_SCALE=1.0
LORA_ALPHA_ARGS=()
# LightX2V only: LORA_ALPHA_ARGS=(--lora-alpha 8)

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --performance-mode speed \
  --lora-path "$LORA_REPO" \
  --lora-weight-name "$LORA_FILE" \
  --lora-nickname "$LORA_NAME" \
  --lora-scale "$LORA_SCALE" \
  "${LORA_ALPHA_ARGS[@]}" \
  --lora-merge-mode auto \
  --port 30010
```

`auto` merges an adapter into ordinary resident weights to avoid per-step LoRA
matmuls. It keeps the dynamic path in two cases. One is FSDP-sharded weights,
where a full gather can increase peak memory. The other is an adapter whose
update would mostly round away in a BF16 merge. Distilled adapters such as the
Larry Turbo LoRA sit below a BF16 ulp of most weights, and a merge drops
77–97% of their update in typical layers. Use `dynamic` when one resident
server must switch repeatedly between base and LoRA output, and `merge` to
force a merge.

Use the filename, scale, and request schedule from the table together. The
4-evaluation LightX2V recipe is the more aggressive latency/quality tradeoff.
Its checkpoint has rank 128 but omits the training alpha from both the file and
repository metadata, so `--lora-alpha 8` is required to reproduce the author's
reference implementation. Start with the Larry 8-evaluation recipe when
preserving fine visual detail is more important than minimum latency.

The pinned files above were trained for the **FL2VA** partition and apply to
`t2va` or `fl2va` requests. Some repositories, including LightX2V, publish
separate files for Ref2VA/Ref2V; select one explicitly for a `ref2va` server
rather than reusing an FL2VA file. Those Ref2VA files are not yet a pinned,
validated recipe on this page. Also avoid stacking a distilled adapter with
`quality: "high"`: both alter denoising, and that combination has not been
quality-validated.

AdaLN-pruned Diffusers components that publish `adaln_basis` and `adaln_mean`
can also consume a LoRA trained against the released full-width AdaLN modules:
SGLang projects those adapter factors onto the pruned coordinates at load time.
A structurally modified checkpoint without that metadata still fails closed,
and packed GGUF weights remain incompatible with LoRA.

### PDD acceleration LoRAs

[Alibaba PAI's MiniMax-H3-Acc-LoRAs](https://huggingface.co/alibaba-pai/MiniMax-H3-Acc-LoRAs)
use Parallel Decoding Distillation (PDD) for eight denoiser evaluations. These
checkpoints contain backbone LoRA weights and interval-specific output heads.
Merge the backbone weights offline, then fuse the output heads before serving.
The `--lora-path` recipe above does not perform these preparation steps.

Match the adapter and original MiniMax-H3 transformer weights to the server's
model variant:

| Model variant | Adapter file | Tasks |
| - | - | - |
| `fl2va` | `MiniMax-H3-FL2VA-Acc-8Step.safetensors` | `t2va`, `fl2va` |
| `ref2va` | `MiniMax-H3-Ref2VA-Acc-8Step.safetensors` | `ref2va` |

`BASE_TRANSFORMER_DIR` is the directory containing the original model's
BF16/FP32 transformer weight files, not the model root or the LoRA directory.
If you downloaded `MiniMaxAI/MiniMax-H3` to `/path/to/MiniMax-H3`, use
`/path/to/MiniMax-H3/FL2VA/transformer` for T2VA/FL2VA or
`/path/to/MiniMax-H3/Ref2VA/transformer` for Ref2VA. This directory contains
`model-*.safetensors`, `model.safetensors.index.json`, and `config.json`.
The builder reads these files and writes the merged weights to
`$PDD_DIR/transformer`, leaving the original weights unchanged.

The following example prepares FL2VA
weights and serves them on eight NVIDIA CUDA GPUs with TP2 + Ulysses4:

```bash Command theme={null}
BASE_TRANSFORMER_DIR=/path/to/MiniMax-H3/FL2VA/transformer
LORA_DIR=/path/to/h3-acc-loras
PDD_DIR=/path/to/h3-fl2va-pdd

hf download alibaba-pai/MiniMax-H3-Acc-LoRAs \
  MiniMax-H3-FL2VA-Acc-8Step.safetensors --local-dir "$LORA_DIR"

python3 -m sglang.multimodal_gen.tools.build_minimax_h3_pdd_weights \
  "$BASE_TRANSFORMER_DIR" \
  "$LORA_DIR/MiniMax-H3-FL2VA-Acc-8Step.safetensors" \
  "$PDD_DIR"

python3 -m sglang.multimodal_gen.tools.fuse_minimax_h3_pdd_heads \
  "$PDD_DIR" --video-shift 12.0 --audio-shift 3.0

SGLANG_DIFFUSION_MINIMAX_H3_PDD_HEADS="$PDD_DIR/pdd_fused_heads.safetensors" \
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --transformer-weights-path "$PDD_DIR/transformer" \
  --num-gpus 8 \
  --tp-size 2 \
  --ulysses-degree 4 \
  --performance-mode speed \
  --port 30010
```

Keep `pdd_config.json` beside `pdd_fused_heads.safetensors`, outside the
`transformer/` directory. For Ref2VA, use its adapter and base transformer,
a separate output directory, and `--model-variant ref2va`.

Use the requests in [Generate video and audio](#4-generate-video-and-audio)
with `num_inference_steps: 8`, `flow_shift: 12.0`, and `audio_flow_shift: 3.0`.
The nine sigma grid points give eight denoiser evaluations. These settings
must match the fused heads; the server rejects a different step count or
shift. Use `quality: "lossless"` in requests.

## 6. FastH3: 8-step distilled

[FastVideo/FastVideo-FastH3-8-Step-V2](https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2)
is an 8-step DMD2 distillation of MiniMax-H3, trained data-free with Video
Sparse Attention (VSA) at 0.8 sparsity and 64-token tiles, with video / audio
scheduler shifts 10 / 3. Only the T2VA capability was distilled: requests must
use `task: "t2va"`, and `fl2va` / `ref2va` requests are rejected. The
checkpoint inherits the MiniMax-H3 Community License.

Pass the repository directly to `--model-path`. The flat native-Diffusers
upload is materialized into the base-H3 layout through a registered model
overlay; the only non-symlink step is a one-time re-serialization of the
roughly 10 GB video VAE on first launch.

```bash 4×GB300 VSA-H3 theme={null}
sglang serve \
  --model-path FastVideo/FastVideo-FastH3-8-Step-V2 \
  --num-gpus 4 \
  --component-attention-backends transformer=video_sparse_attn_h3 \
  --port 30010
```

Requests use the same asynchronous video endpoint as the base model, with
`task: "t2va"`, `conditions: []`, and a target such as
`{"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 5.0}`. The
request default is `num_inference_steps: 8`: nine sigma grid points, i.e. the
eight distilled DiT evaluations on the checkpoint's trained rungs
`[999, 874, 749, 624, 500, 375, 250, 125]` (unshifted timesteps over 1000,
shifted once per modality, as FastVideo runs them). Any other step count is
rejected. The trained sparsity 0.8 is the default; `--attention-backend-config
'{"VSA_sparsity": ...}'` overrides it. FastH3 requests default to
`quality: "lossless"`, which fuses the VAE decoder's per-head QK RMSNorm and
RoPE and runs its attention on cuDNN SDPA; `quality: "exact"` keeps the
reference decoder kernels. Under `"lossless"` FastH3 decodes each GPU's
spatial VAE tiles in one call rather than one at a time (base MiniMax-H3
batches up to 64 per call), which changes GEMM shapes and so the output at
rounding level; `"exact"` decodes tile by tile on both.

`video_sparse_attn_h3` (VSA-H3) is the trained sparse policy: in-tree
block-sparse kernels (FastVideo's native tcgen05 forward on SM100 / SM103, a
Triton tile-64 kernel on SM90) over segment-pure prefix tiles and (4, 4, 4)
video tiles, driven by the checkpoint's trained
`to_gate_compress` compression branch. Select it for the transformer only, as
with cube sparse attention: the packed DiT blocks run sparse while the token
refiner falls back to dense FA inside the backend, and the text encoder and
VAEs keep their own backends. Ulysses sequence parallelism is supported. See
[Attention Backends](https://docs.sglang.io/docs/sglang-diffusion/attention_backends.md) for
`VSA_sparsity`, `vsa_mode`, `vsa_dense_first_n_steps`, and
`vsa_dense_layers`. On CUDA, FastH3 selects VSA-H3 for the transformer when no
transformer backend is given, unless torch.compile, the breakable CUDA graph or
ring attention is on (VSA-H3 serves none of them). Every dense backend that
runs on base H3 also runs on the FastH3 weights, without the compression
branch: select it with `--component-attention-backends transformer=fa` (or
`torch_sdpa`, ...). On 4× H200 dense FA is about 25% slower end to end than
VSA-H3 (12.6 s against 10.0 s). `sglang generate` takes the same flags as
`sglang serve`.

Measured latencies for the 4× GB300 recipe are in
[FastH3 on GB300](#fasth3-on-gb300).

FastH3 rejects deployment options that do not apply to the distilled
checkpoint instead of silently ignoring them: `--model-variant`,
`quality: "high"`, `fl2va` / `ref2va` requests, and, with VSA-H3,
`--ring-degree` greater than 1, `torch.compile`, and breakable CUDA graph
execution.

> [!NOTE]
>
> Quality gaps versus base H3 on hard motion and fine detail are properties of
> the released distillation, not of the SGLang port. Use base MiniMax-H3 when
> output quality matters more than latency.

### FastH3 Omni: Ref2VA, 8-step PDD

[FastVideo/FastVideo-FastH3-Omni-8-Step-V1](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1)
is a Parallel Decoding Distillation (PDD) student of the MiniMax-H3 Ref2VA
partition: eight denoiser evaluations over a 32-interval grid, with VSA at 0.9
sparsity on 128-token (4, 4, 8) tiles and each reference video keeping 0.1 of
its tiles. Only `task: "ref2va"` is served, with up to 9 images, 3 videos and 3
audio clips as ordered references (12 in total), at least one of them visual.

Pass the repository directly to `--model-path`. Its registered model overlay
maps `transformer_ref/` into the base-H3 layout, fuses the 32 interval output
heads into 8 per-step heads (the math of `tools/fuse_minimax_h3_pdd_heads.py`)
and links the text encoder and VAEs from the `MiniMaxAI/MiniMax-H3` Ref2VA
partition pinned by the release. The served model is the native H3 pipeline:
the release's `pdd_fused_heads` load through the same path as the PDD LoRA
recipe above, and the PDD grid is the 8-step shifted grid scaled to the
release's `sigma_max` of 0.999 (FastGen's `max_t`).

```bash 2×H200 theme={null}
sglang serve \
  --model-path FastVideo/FastVideo-FastH3-Omni-8-Step-V1 \
  --num-gpus 2 --ulysses-degree 2 \
  --dit-cpu-offload true --text-encoder-cpu-offload true --vae-cpu-offload true \
  --port 30010
```

```json Request theme={null}
{
  "model": "FastVideo/FastVideo-FastH3-Omni-8-Step-V1",
  "prompt": "<Picture 1> is the appearance reference for <Subject 1>. [reference generation] ...",
  "task": "ref2va",
  "conditions": [{"type": "image", "role": "reference", "uri": "/path/to/subject.png"}],
  "target": {"short_edge": 480, "aspect_ratio": "16:9", "duration_seconds": 5.0},
  "seed": 0
}
```

Requests default to `num_inference_steps: 8` and clean references
(`imgvid_cond_noise_aug_for_inference` and `audio_cond_noise_aug_for_inference`
of 1.0, the release's target-only flow). On CUDA the transformer selects VSA-H3
when no backend is given; Ref2VA layouts tile every reference video as its own
region and take a per-region top-k, while text, reference images and audio
rows stay dense. `--attention-backend-config` keys `vsa_tile_size`,
`vsa_ref_keep_rate` and `VSA_sparsity` override the trained values.

Warm request latency on 2× H200 (Ulysses 2, DiT / text encoder / VAE CPU
offload; median of three requests after one full warmup, 124 frames at 24 FPS
with audio, until the MP4 is saved):

| References | Short edge | Latency |
| - | - | - |
| Image | 480 | 26.9 s |
| Image + audio | 480 | 27.8 s |
| Video with audio | 480 | 23.2 s |
| Image | 768 | 46.6 s |
| Video with audio | 768 | 37.1 s |

## 7. VDN-H3: hybrid attention, 8-step distill

[Video DeltaNet](https://openvdn.github.io/) (VDN) is a hybrid attention
architecture that approaches the speed of linear attention while maintaining
near-lossless quality. It splits the video-video attention into two
complementary branches: a sliding-window softmax branch for local frame pairs
(five consecutive latent frames form a chunk, and each chunk attends to
itself, the previous chunk, and the following chunk; the first and last
frames, text and audio stay dense) and a bidirectional linear branch, Video
Delta Attention, for long-range context.
[OpenVDN/vdn-minimax-h3](https://huggingface.co/OpenVDN/vdn-minimax-h3)
applies it to MiniMax-H3: the released `stage-dmd-step-250` checkpoint adds a
4.3 GB linear branch and OpenVDN's own 8-step DMD2 LoRA on the unchanged H3
backbone. It is the FL2VA partition, so it serves `t2va` and `fl2va` (first,
last, or both keyframes); `ref2va` is rejected. The weights inherit the
MiniMax-H3 Community License.

Pass the repository directly to `--model-path`. On first launch a registered
overlay prefuses both adapters into the transformer weights (a 62 GB write)
and attaches the linear branch; the Qwen3-VL conditioner and the VAEs are
hard-linked from `MiniMaxAI/MiniMax-H3`. Keep `SGLANG_DIFFUSION_CACHE_ROOT` on
the same filesystem as the Hugging Face cache with at least 90 GB free.

```bash 8×B200 theme={null}
sglang serve \
  --model-path OpenVDN/vdn-minimax-h3 \
  --num-gpus 8 \
  --attention-backend hybrid_window_attn_h3 \
  --performance-mode speed \
  --warmup-num-frames 345 \
  --warmup-resolutions 1344x768 \
  --port 30010
```

`--num-gpus 4`, `2` or `1` gives the smaller Ulysses recipes in the benchmark
table; `sglang generate` takes the same flags. `--warmup-num-frames` /
`--warmup-resolutions` warm up at the clip length and canvas you will serve;
without them the first long request pays 2 to 3 s of allocator growth and
kernel setup.

Requests use the same video endpoint as the base model with `task: "t2va"`
and `conditions: []`, or `task: "fl2va"` with the
[keyframe conditions](#4-generate-video-and-audio), and a target such as
`{"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 14.375}` (the
345-frame paper workload). The default `num_inference_steps: 8` is the only
accepted value: nine sigma grid points, eight DiT forwards. Each keyframe adds
about 2,000 dense rows, so an `fl2va` forward costs a little more than `t2va`.

`hybrid_window_attn_h3` is required: a dense backend would silently skip the
linear branch and the gates and produce the wrong model. On Blackwell
(SM100 / SM103 / SM120) the transformer defaults to online MXFP8;
`--quantization bf16` opts out. Before SM100, `--quantization fp8` is base
H3's per-channel fp8 path, and SM80 runs the bf16 DiT (62 GB; budget for
`--layerwise-offload-components`). Ampere and Ada are enabled but not
benchmarked. Ulysses sequence parallelism is supported; `--model-variant`,
`quality: "high"`, `--ring-degree` greater than 1, `torch.compile`, and
breakable CUDA graph execution are rejected. The backend's
`--attention-backend-config` keys are listed in
[Attention Backends](https://docs.sglang.io/docs/sglang-diffusion/attention_backends.md).

Measured latencies for 1, 2, 4 and 8 B200s are in
[VDN-H3 on B200](#vdn-h3-on-b200); single-card and PCIe multi-card numbers
for the RTX PRO 6000 are in [VDN-H3 on RTX PRO 6000](#vdn-h3-on-rtx-pro-6000).

## 8. Sampling and output controls

MiniMax-H3 supports more than one output per prompt. The video API accepts
`num_outputs_per_prompt` (or OpenAI-compatible `n`) from 1 through 10. Offline
generation accepts `--num-outputs-per-prompt N`; `--num-outputs N` is the short
alias. A scalar seed is expanded deterministically as `seed + output_index`, so
the outputs do not reuse the same noise.

Same-prompt fan-out reuses text conditioning. On the verified 2× RTX 5090
recipe, a 5-step two-output request completed in 155.39 seconds versus 78.11
seconds for one output, while producing two distinct valid MP4 files. The
independent denoise and decode passes remain sequential on this 32 GB profile
to keep peak memory bounded; the grouped path adds essentially no orchestration
overhead. Use server replicas when lower wall-clock latency for many variants
matters more than per-server memory efficiency.

For example, set `"num_outputs_per_prompt": 2` in any request above. After the
job completes, download both outputs by selecting each zero-based variant:

```bash Command theme={null}
video_id="<completed-job-id>"
for variant in 0 1; do
  curl -sS -L \
    "http://127.0.0.1:30010/v1/videos/${video_id}/content?variant=${variant}" \
    -o "minimax-h3-${variant}.mp4"
done
```

### Choose the quality level

`quality` is a cumulative request-scoped optimization parameter with three
levels:

* `"exact"`: the exact reference path. Output is bit-exact
  against the reference implementation and the CI ground truth. On SM100+
  the video VAE's temporal blend and output denormalization run as single
  kernels that reproduce the reference arithmetic bit for bit and verify that
  against the eager chain on first use.
* `"lossless"` (default): includes the global fusion-only tier but does not enable
  Cache-DiT or another approximate optimization. Each DiT block runs its
  RMSNorm and AdaLN modulation as one kernel that rounds once instead of after
  every op, and VAE decode fuses the ViT decoder's per-head QK RMSNorm and
  RoPE into one launch, runs its attention on cuDNN SDPA and, on CUDA, decodes
  several spatial tiles per decoder forward (rounding-level differences only).
* `"high"`: the audited accelerated path. Quality is guaranteed (the audited
  Cache-DiT configuration measures SSIM 0.931 / PSNR 28.16 dB against
  `exact`), but output is no longer bit-identical to the reference.

One resident server serves all three levels; a `quality: "high"` request
mounts its audited Cache-DiT policy at the batch boundary, and a later
`quality: "exact"` or `quality: "lossless"` request removes the hooks
before denoising.

Start the validated server once:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --tp-size 1 \
  --sp-degree 4 \
  --ulysses-degree 4 \
  --ring-degree 1 \
  --encoder-parallel auto \
  --performance-mode speed \
  --use-fsdp-inference false \
  --enable-torch-compile false \
  --port 30010
```

Then choose a request level:

**exact**

Native denoising and VAE decode with no feature-cache approximation or
quality-gated VAE fusions.

```json Request field theme={null}
{
  "quality": "exact"
}
```

**lossless (default)**

The global fusion-only tier. It does not enable MiniMax-H3 Cache-DiT. It
differs from `exact` only in rounding: the DiT blocks' fused RMSNorm + AdaLN
modulation, and the VAE decoder's fused QK RMSNorm + RoPE and cuDNN SDPA
attention.
Omitting the field selects this level.

```json Request field theme={null}
{
  "quality": "lossless"
}
```

**high**

The audited accelerated path. Use it when you can trade bit-exactness for
latency while keeping output closest to the same-seed exact trajectory.

```json Request field theme={null}
{
  "quality": "high"
}
```

The measured trade-off is:

| `quality` | Mean inference latency | Speedup | SSIM vs exact | PSNR vs exact | Expected trade-off |
| - | -: | -: | -: | -: | - |
| `exact` | 75.10 s | 1.00× | 1.000 | exact | Native reference path |
| `lossless` | Not separately measured | — | Same H3 denoise path | Same H3 denoise path | Fusion-only tier; VAE decode fuses QK RMSNorm + RoPE and uses cuDNN SDPA (rounding-level differences) |
| `high` | 53.70 s | 1.40× | 0.931 | 28.16 dB | Smallest same-seed visual change |

These numbers use 1344×768, 124-frame, 24 fps T2VA with 50 inference steps,
video flow shift 12, audio flow shift 3, and three fixed prompt/seed pairs on
4×H200. The prompts cover a quiet detailed scene, fast multi-subject action,
and a moving close-up portrait. `inference_time_s` is averaged across the three
prompts; the quiet-scene point is itself the mean of two repeats.

SSIM and PSNR compare decoded, frame-aligned output with the `exact`
result for the same prompt and seed. They measure trajectory deviation, not
absolute perceptual quality: the `high` path can produce a different but
still plausible realization. It also changes the joint audio-video denoise
trajectory, while these two metrics cover video only.

`quality: "high"` currently accepts only the exact workload and 4×H200
deployment above; other hardware, task modes, request shapes, step counts, or
flow shifts fail before denoising. Offline generation uses the same level
name, for example `sglang generate --quality high`.

> [!NOTE]
>
> `quality` selects a model sampling level and can change generated content.
> `output_quality` controls only output-file compression; it is a separate field.

For manually tuned Cache-DiT experiments outside that validated path, omit
the request `quality` field and set `--enable-cache-dit` or the
process-wide `SGLANG_CACHE_DIT_*` defaults. Any explicit `quality`, including
`"exact"` and `"lossless"`, takes H3 off the generic Cache-DiT path. The
24 GB layerwise recipe above can use the same switch; skipped blocks are
not streamed.

```bash Command theme={null}
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=1 \
SGLANG_CACHE_DIT_BN=0 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.12 \
SGLANG_CACHE_DIT_MC=2 \
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010
```

> [!WARNING]
>
> Cache-DiT skips selected block computation and is approximate. It cannot
> be combined with FSDP inference. DiT layerwise offload is compatible:
> skipped blocks are not streamed, and the first layer after a skip may
> sync-load. Breakable CUDA graph execution takes precedence and leaves
> Cache-DiT disabled. Tune the cache thresholds only after comparing both
> video and audio quality on the target task profile. A real B200 request
> has completed, but the `quality: "high"` path above remains fail-closed
> to the audited 4×H200 workload.

## 9. Feature contracts and advanced recipes

The generated command already contains the recommended topology and encoder
setting. Use the detailed reference below only when applying an optional
override or checking its installation, topology limits, and validation evidence.

**Lossless runtime**

The recommended `speed` launch already combines resident components with
Ulysses sequence parallelism. Validation status below applies only to the
listed hardware and topology; it is not inherited by a similar GPU family.

| Feature | Validation status | Notes |
| - | - | - |
| Ulysses sequence parallelism | Verified: 8× B200, 4× H200, 4× H100, and Ulysses1/2/4/8 on MI300X and MI355X | Use `--ulysses-degree`. Combine with Ring for cross-node scaling; see the next row. |
| Ring sequence parallelism (cross-node) | Verified: 2 nodes of 8× H200 each (Ulysses8 × Ring2) | Use `--ring-degree` together with `--nnodes`/`--node-rank`/`--dist-init-addr`. Ring shards the sequence across nodes while Ulysses shards heads within a node; H3's packed multi-segment attention only supports Ring across the node boundary, not within a single node's Ulysses group. Requires `--encoder-parallel replicate` — `auto`'s fold decision is not node-boundary aware. See the benchmark section below. |
| Tensor parallelism | Verified: B200 TP2 + Ulysses4; H100 TP2 + Ulysses2 and TP4 + Ulysses1 | `--tp-size` may be combined with Ulysses when the TP-local head count remains divisible by the Ulysses degree. On 4×H100, TP2 + Ulysses2 is the measured speed default. |
| FSDP inference | Verified: 4× B200 and 4× H100 + Ulysses4 | Preserves H3's mixed BF16/FP32 parameter policy. B200 completed the exact eager comparison; H100 completed consecutive real requests at about 57 GB peak memory per GPU. |
| Resident components | Verified: B200, H200, 4×H100 with TP, and 1/2/4/8× MI300X and MI355X | This is the recommended single-request latency path when the complete workload fits. |
| CPU and layerwise offload | Verified: 2× RTX 5090 TP2; 1× RTX 4090 24 GB | The 5090 lossless recipe keeps 20 DiT blocks plus both VAE encoders resident, streams the remaining DiT blocks, text encoder, and video VAE decoder blocks, and leaves the small audio VAE resident. The 4090 recipe streams DiT and the text encoder with zero resident DiT layers and **omits `vae`** from `--layerwise-offload-components`. Compatible with Cache-DiT; skipped blocks are not streamed. |
| Breakable CUDA graph | Verified: B200 Ref2VA, opt-in | Matching eager output was observed for the captured signature, without a measured speedup. Re-capture for other shapes and reference sets. |
| `torch.compile` | Measured: H200, opt-in | Steady-state benefit was below measurement noise, while startup increased and numerical output changed. Do not use it for consistency ground truth. |

The verified parallel, placement, and matching-signature BCG paths keep the
BF16/FP32 weights and denoising math. `torch.compile` is the exception called
out above. Always use the eager BF16/FP32 launch when producing CI consistency
ground truth.

For the validated 1344×768 Ref2VA profile, use a 5504-row text bucket so both
the server warmup and reference-conditioned requests share the captured
signature:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --enable-breakable-cuda-graph true \
  --warmup-resolutions 1344x768 \
  --bcg-text-buckets 5504 \
  --port 30010
```

BCG is lossless for a matching captured signature, but capture reserves extra
GPU memory. Re-measure the live H3 text length before reusing this bucket for a
different task profile, reference set, resolution, or prompt template.

**Attention backends**

Leave `--attention-backend` unset for the platform default. Use
`--attention-backend fa` only for an explicit FlashAttention comparison.

SageAttention uses quantized attention math and is not a consistency mode. To
select H3's native packed-varlen Sage path, install the dependency and add
`--attention-backend sage_attn`. On Hopper, install the upstream SM90 binding
fix rather than the PyPI 2.2.0 build:

```bash Command theme={null}
pip install --force-reinstall \
  git+https://github.com/thu-ml/SageAttention.git@d9704247a5139ab4c03bf7fc6b35cc0e2cbb5ea4 \
  --no-build-isolation
```

The backend is a server-wide default. Use
`--component-attention-backends` only when a measured component needs a
different kernel, and keep the platform default for every component not named
in the override.

**Online quantization**

On the verified 8× B200 topology, quantize the BF16 transformer at server load:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --quantization fp8 \
  --port 30010
```

H3 automatically keeps its video/audio patch projections, timestep MLP, and
final video/audio heads in FP32. All other linear layers have stable full
module prefixes, so additional layers can be kept unquantized:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --quantization fp8 \
  --quantization-ignored-layers blocks.0.attn token_refiner \
  --port 30010
```

> [!WARNING]
>
> Online FP8 is approximate and is not a consistency ground-truth mode. It can
> be combined with Cache-DiT, but the two approximations compound. Validate
> visual quality, audio quality, memory use, and latency on the target workload.
> This recipe is limited to the resident B200 and B300 topologies used for real
> H3 validation runs.

On a single 24 GB card, use `convrot_int8` instead of FP8. It quantizes the
four GEMMs per DiT block online from the Hub BF16 weights (data-free, no
calibration) and runs them on `comfy_kitchen.int8_linear` on the 4090 (CC 8.9)
or on SGLang's JIT-compiled ConvRot ops on CC 9.0, 10.0, 12.0 and 12.1
(see [Quantization](https://docs.sglang.io/docs/sglang-diffusion/quantization.md#convrot-int8)).
Quantization happens after H3's grouped `qkv` reorder, so do not load an
externally pre-quantized INT8 checkpoint here.

```bash Command theme={null}
sglang generate \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --quantization convrot_int8 \
  --attention-backend fa \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 0 \
  --enable-torch-compile false \
  --prompt "A cat walking on a sunny beach, gentle waves." \
  --save-output
```

`fa` keeps exact attention. For a faster, approximate DiT path, use
`--attention-backend sol_attn` with
`--attention-backend-config dense_backend=sage_attn,dense_steps=10` and
`--component-attention-backends text_encoder=torch_sdpa,transformer=sol_attn`.
See [Quantization](https://docs.sglang.io/docs/sglang-diffusion/quantization.md#convrot-int8)
and [Attention Backends](https://docs.sglang.io/docs/sglang-diffusion/attention_backends.md#sage-then-sol-hybrid).

> [!WARNING]
>
> `convrot_int8` changes Linear numerics. `sol_attn` / `sage_attn` also change
> the attention algorithm. Neither is a consistency ground-truth mode. The
> BF16 path never depends on `comfy-kitchen`; `convrot_int8` needs it only on
> GPUs the JIT ops do not cover (any CC other than 9.0, 10.0, 12.0 and 12.1) or
> when no CUDA toolchain is available to build them.

Pre-quantized and compact H3 text encoders are listed once in
[Checkpoint and adapter formats](#checkpoint-and-adapter-formats). They use
component-local paths and never inherit the DiT's `--quantization` setting.
SGLang reads their metadata before constructing the native Qwen3-VL encoder and
fails closed when the selected format, projection, or topology is incompatible.

**Cube sparse attention**

Cube sparse attention applies TopK sparsity only to H3's 3D visual streams.
Text, audio, standalone reference images, and the text-only token refiner stay
dense. It runs on pure PyTorch plus FlexAttention, so it has no third-party
kernel dependency.

Select it for the H3 transformer with
`--component-attention-backends transformer=cube_sparse_attn` and pass
`--attention-backend-config` with both `local_cube_size` and
`topk_ratio_list`. Scoping the backend leaves the text encoder on its native
backend:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --performance-mode speed \
  --component-attention-backends transformer=cube_sparse_attn \
  --attention-backend-config '{"local_cube_size": [4, 4, 4], "topk_ratio_list": [1.0, 1.0, 0.8, 0.7, 0.6, 0.5, 0.5]}' \
  --port 30010
```

* `local_cube_size` is the `(T, H, W)` cube that groups neighboring latent
  tokens into one attention block. It must have exactly three entries.
* `topk_ratio_list` sets the per-step keep ratio and must have exactly one
  entry per denoise step, each in `(0, 1]`. A ratio of `1.0` keeps a step
  on H3's native dense attention path; smaller values select the FlexAttention
  sparse path and drop more blocks. The example above matches a request with
  `num_inference_steps: 7`, whose endpoint-inclusive sigma schedule has seven
  denoise updates.

Cube labeling is coordinate-driven. FL2VA keyframes share the target video's
position grid, so a keyframe token and a target token at the same `(T, H, W)`
coordinate receive the same semantic cube label. Duplicate coordinates do not
extend the temporal grid; a semantic cube can therefore span multiple physical
attention blocks. In Ref2VA, standalone reference images remain dense, while
reference videos and the target video contribute to one global TopK candidate
pool rather than receiving separate per-stream quotas.

> [!WARNING]
>
> Cube sparse attention is an approximate backend and is not a consistency
> ground-truth mode. `topk_ratio_list` length must equal the denoise step count
> or the server rejects the request. Cube sparse attention does not support Ring
> parallelism; use `--ulysses-degree` without `--ring-degree`. FlexAttention's
> routing overhead can outweigh sparse-kernel savings on short sequences, so
> benchmark latency as well as visual and audio quality on the target workload.

**SubBlock sparse attention**

SubBlock sparse attention applies training-free block sparsity to MiniMax-H3's
long, non-causal DiT self-attention. The BF16 path supports CUDA SM90, SM100,
and SM120. Other compute capabilities, including SM103/B300, are rejected.
Ulysses sequence parallelism is supported; Ring parallelism is not.

The global backend selects the H3 DiT lazily, so keep the Qwen text encoder on
its compatible dense backend with a component override:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --attention-backend subblock_sparse_attn \
  --component-attention-backends text_encoder=fa \
  --attention-backend-config '{"sparsity": 0.75, "skip_first_steps": 10}' \
  --port 30010
```

Use `text_encoder=torch_sdpa` on SM120. The default configuration drops at
most 75% of key blocks after the first 10 denoise forwards; short sequences,
cross-attention, and unsupported shapes continue on the dense path. Tune
`sparsity` and `skip_first_steps` together with visual and audio quality checks.

On H100/H200 (SM90), `compute_mode=sage_fp8` switches the sparse kernel to
online INT8 Q/K and FP8 P/V compute. Install the optional kernel and add the
compute mode:

```bash Command theme={null}
pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --attention-backend subblock_sparse_attn \
  --component-attention-backends text_encoder=fa \
  --attention-backend-config '{"compute_mode": "sage_fp8", "sparsity": 0.75}' \
  --port 30010
```

> [!WARNING]
>
> Both SubBlock routing and `sage_fp8` are approximate. Use dense BF16 attention
> for consistency ground truth, and validate the selected sparsity on the target
> resolution, duration, task, and checkpoint.

**Encoder scheduling**

The picker explicitly writes `--encoder-parallel auto` in every single-node
recipe. At the default request batch size of one, H100/H200/B200/B300 servers
with peer-to-peer access fold the Qwen encoder across otherwise idle Ulysses
ranks. A pure-TP recipe keeps the encoder inside its TP group, while a
PCIe-only host can avoid an expensive world fold. Keep `auto` unless one of
the cases below applies.

Encoder DP is a throughput policy for compatible request batches. It requires
TP1 and DiT DP1, replicates the encoder weights, and does not improve a batch
of one:

```bash Overlay theme={null}
--encoder-parallel dp \
--batching-max-size 2
```

The cross-node picker recipe already uses replication because the automatic
fold decision is not node-boundary aware:

```bash Overlay theme={null}
--encoder-parallel replicate
```

## 10. Configuration notes

* MiniMax-H3 produces the canonical 24 fps output; request duration is expressed through `target.duration_seconds`.
* `target.duration_seconds` must be between 4 and 15 seconds, inclusive. The command picker defaults to the verified 5-second profile.
* Use a 768-pixel short edge for the released quality recipe. The aligned output dimensions are derived from `target.aspect_ratio`.
* `flow_shift` controls video diffusion and `audio_flow_shift` controls audio diffusion.
* V2V uses `task: "ref2va"` with a `video` or `video_audio` reference; it is served by the `Ref2VA` partition and is not a separate public task value.
* `conditions[].start_time_seconds` selects a non-negative offset for a video reference. Its visual and audio streams are always sought together.
* Ref2VA condition order is semantic and must match the one-based material tags in the prompt. For Ref2VA, `target.aspect_ratio: "auto"` resolves to the model's 16:9 fallback rather than inheriting a reference asset's geometry.
* The distilled pipeline uses a single denoising branch, so CFG parallelism does not apply. Keep it disabled: a CFG degree greater than 1 is accepted for request compatibility, but the extra ranks only recompute the same branch and do not accelerate the request. Explicitly disabling CFG, or setting its size to 1, remains a valid no-op.
* The released visual VAE quality recipe uses overlapping tiled decode. SGLang keeps that recipe by default and distributes complete tiles across the decode group; this changes scheduling, not the computation inside each tile.
* H3 rejects `--vae-config.parallel-decode-mode spatial` and `spatial_shard`: validation found output mismatches. Use the default released tiled recipe.
* On CUDA, `quality: "lossless"` (the default) decodes up to `vae_config.decoder_tile_batch` (64) spatial tiles and `vae_config.window_batch` (1) temporal windows per video VAE decoder forward, next to the fused Q/K norm+RoPE and cuDNN SDPA; `exact` decodes one tile and one window at a time, `--performance-mode memory` does so at every level, and other platforms keep one tile per forward. On SM100+ the temporal blend and output denormalization run as bit-exact kernels at every quality level.
* Keep the default `--encoder-parallel auto`. With the server’s default `batching_max_size` of 1, single-node H100/H200/B200/B300 recipes with peer-to-peer access fold the Qwen text encoder over otherwise idle Ulysses ranks. This is separate from DiT tensor parallelism. A pure-TP recipe already shards the encoder over its TP group and does not add a world fold.
* Single-node Ulysses recipes pipeline each block's all-to-all against dense attention (FlashAttention on Hopper, cuDNN SDPA on Blackwell) over the copy engine: one head group's q/k/v moves while the previous group attends, every copy (the merge into the output included) runs on the copy engine, and the output is bit-identical to the sequential exchange. It needs peer-to-peer access and stream memory operations on every rank and about 6 × (rows per rank) × heads × head\_dim extra activations per rank (2.1 GB for Ref2VA at Ulysses 2), one set shared by every resolution; when that does not fit on some rank, every rank keeps the sequential exchange. `SGLANG_DIFFUSION_ULYSSES_PIPELINE_GROUPS=0` restores the sequential exchange.
* For throughput-oriented serving, select **DP (batched throughput)**. The picker pairs `--encoder-parallel dp` with an editable `--batching-max-size` greater than 1. Encoder DP stays inside each DiT replica and composes with encoder TP: the H100 TP2 + Ulysses2 recipe has two TP-sharded encoder copies that can split a batch, while the RTX 5090 pure-TP2 recipe has one encoder copy and therefore no additional batch-DP degree. It provides no benefit for a batch of one and is not bitwise-identical to the unsplit deployment.
* Use explicit **Fold** to prioritize single-request latency and encoder memory on a measured high-bandwidth single-node topology. Use **Replicate** as the compatibility path when folding or encoder DP is unsuitable.
* `--use-fsdp-inference true` shards only the DiT. MiniMax-H3 preserves the original FP32 dtype of its patch, time, and output projections during FSDP all-gather, so this path does not trade numerical correctness for memory. On 4×H100, prefer TP2 + Ulysses2 for speed; use FSDP as an explicit capacity policy rather than assuming it is faster.
* `speed` keeps model components resident, while `auto` applies the model-aware 120 GiB residency threshold. `memory` prioritizes avoiding OOM and includes the executable VAE decoder in its default layerwise set. A measured recipe with sufficient headroom can opt into `--component-residency vae=resident`; the 2×H100 CI recipe does this because the VAE's 4.8 GiB/GPU cost avoids repeated decoder transfers during tiled decode. DiT residency and prefetch knobs remain scoped to the DiT. Use `speed` only after confirming that the complete target workload fits.
* Breakable CUDA graph execution is an explicit opt-in, not part of the recommended `speed` preset. It requires `--enable-breakable-cuda-graph`, every served size in `--warmup-resolutions`, and `--bcg-text-buckets` that cover the live H3 condition sequence. The validated 1344×768 Ref2VA recipe uses 5504; other task profiles and reference sets may need a different value. It preserves eager output for matching captured signatures, but graph capture consumes additional GPU memory and may provide little latency benefit when Ulysses attention and collectives dominate, so benchmark it on the target topology before enabling it.
* `--enable-spectrum` skips selected DiT steps. On H3 it cannot combine with Cache-DiT / `quality="high"` or SP > 1. 50-step 4090 recipe is in §3.

## 11. Benchmarks

The picker exposes resident and FSDP profiles on NVIDIA datacenter GPUs. GPU
counts are properties of the selected recipes, not a claim that every platform
requires that many GPUs. The detailed tables below report performance only for
the configurations with collected measurements:

| Hardware | Default resident recipe | Other profile or topology |
| - | - | - |
| B300 | 8× Ulysses8 resident | 8× FSDP + Ulysses8; the 8-GPU sweep is not a minimum-GPU claim. |
| B200 | 8× Ulysses8 resident | 4× FSDP + Ulysses4 |
| GB300 | 4× Ulysses4 resident (FL2VA T2VA) | 2 nodes × 4× Ulysses4×Ring2 is unverified. |
| GB200 | 4× Ulysses4 resident, derived/unverified | No performance measurements. |
| H200 | 4× Ulysses4 resident | 4× FSDP + Ulysses4; 4× TP2 + Ulysses2; 2 nodes × 8× Ulysses8×Ring2 cross-node |
| H100 | 4× TP2 + Ulysses2 resident | 4× TP4 + Ulysses1; 4× FSDP + Ulysses4 |
| Ascend NPU | 8 NPUs, TP2 + SP4, Laser Attention | 4 NPUs, TP2 + SP2, Laser Attention |
| MI300X / MI355X | 8× Ulysses8 resident | 1×, 2×, and 4× scaling runs |
| RTX 5090 | 1× layerwise offload, measured on a physical desktop | 2× TP2 + layerwise offload |
| RTX 4090 24 GB | 1× layerwise offload + `convrot_int8` | Spectrum and approximate attention backends are opt-in |

### GB300 single-host

Four requests on **1 host × 4 NVIDIA GB300**, using Linux ARM64, SGLang main
[`15d2cbcc90fc`](https://github.com/sgl-project/sglang/commit/15d2cbcc90fc66b4d08745994cc707a45b067ffc),
PyTorch 2.13.0+cu130, and NCCL 2.29.7. The source checkout was installed with
`python3 -m pip install -e "python[diffusion]" --upgrade` inside
`lmsysorg/sglang:latest`; this is a source-validation environment, not a claim
that the image's bundled release contains that revision.

The command matches the GB300 default in the builder: Ulysses4, resident
weights, automatic encoder folding, native BF16/FP32, platform-default
Dynamic cuDNN/FA attention, and eager execution. The model was downloaded by
`sglang serve` from `MiniMaxAI/MiniMax-H3`.

The workload uses the builder's cat-band prompt, FL2VA T2VA, a 5-second request
at a 768-pixel short edge, 50 inference steps, `quality="exact"`, one output,
and seed 1101. Every MP4 contained video and audio and passed a full decode check.

| First full request after server warmup | Following three requests | Warm median |
| -: | - | -: |
| 35.16 s | 33.11 / 33.10 / 33.10 s | **33.10 s** |

These are client-observed completion times, including up to one second of
status-polling delay. Download, model loading, and server warmup are excluded.
This is a deployment smoke/latency measurement, not a cross-framework quality
audit, a minimum-GPU claim, or an extrapolation to GB200/NVL72 scaling.

### Ascend NPU topology comparison

Both topologies used Laser Attention and the explicit Cache-DiT configuration
from the Ascend launch commands, with `--dit-cpu-offload false` keeping the DiT
resident. The measured workload was one 5-second T2VA request at 1344×768,
124 frames, 24 fps, and 50 inference steps.

| NPU count | Topology | End-to-end latency |
| -: | - | -: |
| 8 | TP2 + SP4 | **55.07 s** |
| 4 | TP2 + SP2 | **103.57 s** |

These are individual end-to-end measurements for each topology, not averages.
The eight-NPU topology had 46.8% lower end-to-end latency than the four-NPU
topology.

### B300 precision and encoder placement

A 12-configuration sweep on a single 8× B300 host, covering both checkpoint
partitions, both transformer precisions, and all three text-encoder
placements. It answers one question — *how long does one request take, and how
much memory does it need*.

### What was measured

**Hardware.** 8× NVIDIA B300 SXM6, single node.

**Model.** `MiniMaxAI/MiniMax-H3`, both released weight partitions.

**Serve command.** Exactly the recipe the picker emits for B300, plus the one
or two overlay flags under test:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --performance-mode speed \
  --host 0.0.0.0 \
  --port 30010
```

The swept axes are `--model-variant` (`fl2va` / `ref2va`), `--quantization`
(unset for BF16 / `fp8`), and `--encoder-parallel` (`auto` / `fold` /
`replicate`). Nothing else differs between the 12 servers.

This is a single-request latency sweep (`batching_max_size: 1`), so encoder DP
is intentionally excluded: it cannot distribute a batch of one. Use the
**DP for a request batch** setting above for a compatible multi-request
deployment; the table below does not claim a measured H3 DP speedup.

**Driver.**

```bash Command theme={null}
python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
  --host 127.0.0.1 --port 30010 \
  --model MiniMaxAI/MiniMax-H3 \
  --dataset vbench --task text-to-video \
  --num-prompts 1 --max-concurrency 1 \
  --warmup-requests 1 --warmup-inference-steps 50 \
  --extra-body '{"task":"t2va","conditions":[],"target":{"short_edge":768,"aspect_ratio":"16:9","duration_seconds":5.0},"seconds":5,"flow_shift":12.0,"audio_flow_shift":3.0}'
```

**Workload**

| Property | Value |
| - | - |
| Output duration | 5.167 s |
| Resolution | 1344×768 |
| Frames | 124 @ 24 fps |
| Denoising steps | 50 |
| `flow_shift` / `audio_flow_shift` | 12.0 / 3.0 |
| Requests in flight | 1 (`--max-concurrency 1`, server at `batching_max_size: 1`) |
| Requests measured | 1 per cell, after 1 warmup request |

### Results

| Weights | Precision | Encoder | Load | Warmup | Latency | Peak/GPU |
| - | - | - | -: | -: | -: | -: |
| FL2VA | BF16 | auto | 118.1 s | 29.65 s | **19.04 s** | 83,578 MB |
| FL2VA | BF16 | fold | 114.0 s | 28.72 s | **19.04 s** | 83,578 MB |
| FL2VA | BF16 | replicate | 116.0 s | 28.33 s | **19.04 s** | 124,158 MB |
| FL2VA | FP8 | auto | 116.0 s | 27.16 s | **18.03 s** | 51,926 MB |
| FL2VA | FP8 | fold | 116.0 s | 25.99 s | **18.04 s** | 51,926 MB |
| FL2VA | FP8 | replicate | 118.0 s | 27.97 s | **18.04 s** | 92,506 MB |
| Ref2VA | BF16 | auto | 114.0 s | 38.69 s | **29.12 s** | 83,968 MB |
| Ref2VA | BF16 | fold | 118.0 s | 36.58 s | **29.13 s** | 83,968 MB |
| Ref2VA | BF16 | replicate | 116.0 s | 35.17 s | **29.13 s** | 124,490 MB |
| Ref2VA | FP8 | auto | 124.0 s | 34.30 s | **27.12 s** | 52,816 MB |
| Ref2VA | FP8 | fold | 112.0 s | 34.44 s | **27.12 s** | 52,816 MB |
| Ref2VA | FP8 | replicate | 116.0 s | 33.42 s | **27.12 s** | 93,396 MB |

### FastH3 on GB300

A 4× GB300 host served [FastH3](#6-fasth3-8-step-distilled) with the VSA-H3
recipe above (1344×768 at 24 fps with audio, `task: "t2va"`,
eight DiT forwards, the default `quality: "extra-high"` and sparsity 0.8,
eager BF16, Ulysses4, `NCCL_NVLS_ENABLE=0`), measured with the MP4 encoded
while the VAE decodes
([sgl-project/sglang#41819](https://github.com/sgl-project/sglang/pull/41819)).
The workload is 72 requests at
concurrency 1 after one warm request per shape: six scene families, 1,000- and
10,000-token prompts, 5 / 10 / 15 s, two seeds, every prompt unique so no
conditioning cache hits. E2E is the client wall clock from submission to the
last byte of the MP4; the stage columns are the server timings of the same
requests. Each cell is the median of the 12 requests of that shape:

| Prompt / requested / aligned | Text encoder | Denoise (8 forwards) | Decode (video + audio) | E2E | Peak/GPU |
| - | -: | -: | -: | -: | -: |
| 1K / 5 s / 124 | 0.10 s | 3.84 s | 0.56 s | **5.20 s** | 94,468 MB |
| 1K / 10 s / 243 | 0.10 s | 8.45 s | 1.12 s | **10.74 s** | 97,628 MB |
| 1K / 15 s / 362 | 0.10 s | 14.85 s | 1.72 s | **18.24 s** | 100,246 MB |
| 10K / 5 s / 124 | 0.18 s | 6.31 s | 0.55 s | **7.90 s** | 94,510 MB |
| 10K / 10 s / 243 | 0.18 s | 12.35 s | 1.12 s | **14.98 s** | 98,128 MB |
| 10K / 15 s / 362 | 0.18 s | 20.28 s | 1.67 s | **24.01 s** | 100,408 MB |

Over all 72 requests the E2E p50 / p90 / p95 is 12.94 / 23.94 / 24.08 s, and
every MP4 carries 1344×768 H.264 at 24 fps with a 32 kHz stereo AAC track.

### VDN-H3 on B200

An 8× B200 (SM100, 183 GB) host served [VDN-H3](#7-vdn-h3-hybrid-attention-8-step-distill)
on the paper workload: 1344×768 at 24 fps with audio for 14.375 s (345 frames,
102 latent frames, about 104k packed rows), `task: "t2va"`,
nine sigma grid points (8 DiT forwards), seed 1000, `hybrid_window_attn_h3`,
eager, and the warmup run at the served clip shape
(`--warmup-num-frames 345 --warmup-resolutions 1344x768`), so every forward
of the served request is steady state; without those flags the first forward
pays 2 to 3 s of allocator growth and kernel setup. "Steady
s/NFE" is the mean of forwards 2 to 8. The OpenVDN reference rows ran the released
inference stack (`8nfe_tuned_fp8.yaml`, `infer_ulysses.py`) on the same host
and the same clip length, with its Ulysses layout (`parallel.softmax_ranks`)
swept at every GPU count rather than assumed. The SGLang rows were re-measured
in one campaign at `f1a2ee763d`, one arm at a time on an otherwise idle node,
each with the [8×B200 recipe](#7-vdn-h3-hybrid-attention-8-step-distill) and
only `--num-gpus` and `--quantization` changed; the OpenVDN reference rows come
from an earlier campaign on the same host type. A repeated arm moves by about
3% across nodes, so treat smaller differences as noise:

| Config | Steady s/NFE | Denoise (8 forwards) | Decode | Peak/GPU |
| - | -: | -: | -: | -: |
| OpenVDN published, FP8, 8× B200 Ulysses 5+3 ([openvdn.github.io](https://openvdn.github.io/)) | 1.40 | 11.2 s | — | — |
| OpenVDN reference, BF16 (`8nfe_tuned.yaml`), 1× B200 | 7.66 | 61.2 s | — | — |
| OpenVDN reference, FP8, 1× B200 | 6.25 | 50.0 s | — | — |
| OpenVDN reference, FP8, 2× B200 Ulysses (standard / 1+1 branch-parallel) | 4.87 / 5.04 | 39.0 / 40.3 s | — | — |
| OpenVDN reference, FP8, 4× B200 Ulysses (3+1 branch-parallel) | 2.59 | 20.7 s | — | — |
| OpenVDN reference, FP8, 8× B200 Ulysses (5+3 / 4+4 / standard) | 1.40 / 1.47 / 1.52 | 11.2 / 11.7 / 12.2 s | — | — |
| SGLang, `--quantization bf16`, 1× B200, `--performance-mode speed`, served-shape warmup | 7.27 | 58.2 s | 7.0 s | 156,298 MB |
| SGLang, `--quantization fp8` (→ online `mxfp8` on SM100+), 1× B200, `--performance-mode speed`, served-shape warmup | 5.93 | 47.4 s | 7.0 s | 128,420 MB |
| SGLang default (online `mxfp8`), 2× B200 Ulysses2, `--performance-mode speed`, served-shape warmup | **3.25** | **26.0 s** | 3.7 s | 102,664 MB |
| SGLang, `--quantization bf16`, 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | 2.05 | 16.4 s | 2.1 s | 104,358 MB |
| SGLang default (online `mxfp8`), 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | **1.67** | **13.3 s** | 2.1 s | 81,168 MB |
| SGLang default (online `mxfp8`), 8× B200 Ulysses8, `--performance-mode speed`, served-shape warmup | **0.85** | **6.8 s** | 1.4 s | 76,918 MB |
| SGLang, per-channel fp8 weight scales (the online `fp8` path of other models; on SM100+ `--quantization fp8` resolves to `mxfp8` and no flag selects the per-channel path, so this arm forces it), 8× B200 Ulysses8 | 0.91 | 7.3 s | 1.4 s | 76,450 MB |

On 8× B200 the SGLang Ulysses8 path runs the paper workload at 0.85 s/NFE
(6.8 GPU-seconds per NFE) against the published 1.40 s/NFE of OpenVDN's 5+3
branch-parallel layout; the whole request (text encoding, 8 forwards, joint
decode) completes in about 8.2 s after warmup. At this GPU count the step is
launch- and copy-bound in the linear branch rather than FLOP-bound, and the
transformer runs online `mxfp8` by default (e4m3 with one E8M0 scale per 32
elements into cuBLASLt's block-scaled GEMM, with the activation quant fused
into the adaLN and SwiGLU producers).

Against the published 8× B200 headline (1.40 s/NFE, 11.2
GPU-seconds per NFE), the SGLang 4× B200 run spends 6.7 GPU-seconds per
NFE, with half the GPUs and half the all-to-all fan-out. At equal GPU count
SGLang is 1.50× faster at two cards (3.25 vs 4.87 s/NFE), 1.56× at four
(1.67 vs 2.59) and 1.66× at eight (0.85 vs 1.40), while on one card the two
stacks are within 5% of each other (5.93 vs 6.25 s/NFE quantized, 7.27 vs 7.66
in BF16). The advantage is therefore parallel scaling rather than per-GPU
kernel work: SGLang holds 88-91% parallel efficiency from 2 to 8 cards against
the reference stack's 56-64%, and sweeping the reference stack's own layout
does not close it (its best split is 8% better than standard Ulysses at eight
cards, 1.40 against 1.52 s/NFE).

On one B200 both single-GPU rows run with the DiT resident under the auto
memory policy (128 GB peak quantized, 156 GB in BF16, of the card's 183 GB),
so no `--layerwise-offload-components` flag is needed at this clip length and
the single-GPU rows are compute-bound.

### VDN-H3 on RTX PRO 6000

The same paper workload (1344×768, 345 frames, 8 DiT forwards, seed 1000,
`hybrid_window_attn_h3`, `--performance-mode speed`, served-shape warmup) on
RTX PRO 6000 Blackwell Server Edition cards (SM120, 96 GB, PCIe, no NVLink).
The window runs on FA4's sm120 kernel and the transformer defaults to online
`mxfp8` (a cutlass sm120 block-scaled GEMM); `--quantization fp8` maps to it.
One card needs `--layerwise-offload-components text_encoder`: the fp8 DiT
(33 GB) and the bf16 text encoder (48 GB) do not both fit, and the whole-module
`--text-encoder-cpu-offload` re-homes the encoder in one piece and runs out of
memory when it is used. Multi-card Ulysses goes over PCIe; the all-to-all is
the bulk of the step there (54% of GPU time at 8 cards), so the per-card
efficiency drops with count. A 2-card run should use a pair on one PCIe
switch (`nvidia-smi topo -m` shows `PIX`): a pair across the CPU root complex
measured 15.7 to 17.0 s/NFE on a shared host.

| Config | Steady s/NFE | Denoise (8 forwards) | Decode | Peak/GPU |
| - | -: | -: | -: | -: |
| 1× RTX PRO 6000, `--layerwise-offload-components text_encoder` (default `mxfp8`) | **20.27** | **160.0 s** | 20.6 s | 95,790 MB |
| 1× RTX PRO 6000, same, per-channel `fp8` path (before this mapping) | 22.85 | 180.3 s | 20.7 s | 95,160 MB |
| 2× RTX PRO 6000 Ulysses2, one PCIe switch (default `mxfp8`) | 13.61 | 107.4 s | 10.6 s | 93,846 MB |
| 2× RTX PRO 6000 Ulysses2, one PCIe switch, per-channel `fp8` | 14.97 | 118.1 s | 10.6 s | 94,700 MB |
| 4× RTX PRO 6000 Ulysses4 (default `mxfp8`) | 7.98 | 63.0 s | 5.6 s | 82,178 MB |
| 4× RTX PRO 6000 Ulysses4, per-channel `fp8` | 8.23 | 65.1 s | 5.6 s | 82,422 MB |
| 8× RTX PRO 6000 Ulysses8 (default `mxfp8`) | 5.43 | 43.1 s | 3.5 s | 77,704 MB |
| 8× RTX PRO 6000 Ulysses8, per-channel `fp8` | 5.47 | 43.4 s | 3.5 s | 77,236 MB |

For comparison the same code on B200 measures 6.03 / 3.30 / 1.67 / 0.88 s/NFE
at 1 / 2 / 4 / 8 cards (NVLink, 86% parallel efficiency at 8).

### H200 topology comparison

The same four-card H200 host completed both lossless resident placements with
the standard 1344×768, 5-second, 50-step T2VA request (fixed prompt and seed,
eager BF16/FP32, back-to-back runs on an otherwise idle host). Latency is the
warmed-up request; the first pair uses the default warmup request, the second
pair adds `--warmup-resolutions 1344x768` so warmup already covers the served
resolution:

| Topology | Warmup | Denoise | Decode | E2E | Peak/GPU |
| - | - | -: | -: | -: | -: |
| Ulysses4 | default | 79.04 s | 3.77 s | **84.14 s** | 94,288 MB |
| TP2 + Ulysses2 | default | 81.17 s | 2.97 s | 85.51 s | 63,490 MB |
| Ulysses4 | `--warmup-resolutions 1344x768` | 71.73 s | 1.32 s | **74.38 s** | 94,290 MB |
| TP2 + Ulysses2 | `--warmup-resolutions 1344x768` | 75.52 s | 1.29 s | 78.33 s | 63,490 MB |

Ulysses4 stays the H200 latency default: 5.0 % faster end-to-end than
TP2 + Ulysses2 once warmup covers the served resolution (1.6 % with the
default warmup, where first-request cold start masks the topology gap).
TP2 + Ulysses2 shards the DiT weights and holds peak memory about 30 GB per
GPU lower, which is why it remains the 80 GB H100 recipe. Matching the warmup
request to the served resolution removes the cold first-request cost on both
topologies (about 10 s end-to-end on this workload).

### H200 cross-node scaling

Long references and long durations grow the packed sequence length, and
Ulysses alone cannot scale sequence parallelism past the GPU count of one
node without either violating head-count divisibility or exposing
all-to-all traffic across the slower inter-node link. H3 combines
node-local Ulysses with cross-node Ring: Ring's point-to-point KV rotation
is designed to overlap with attention compute, which fits a slower
cross-node link better than an all-to-all does.

**Hardware.** 2 nodes × 8× NVIDIA H200 SXM, same cluster, InfiniBand
between nodes.

**Serve command.** The cross-node cell the picker emits for H200, run
identically on both nodes with `--node-rank` set to 0 and 1:

```bash Command theme={null}
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 16 \
  --nnodes 2 \
  --node-rank {{NODE_RANK}} \
  --dist-init-addr {{NODE0_IP}}:20000 \
  --sp-degree 16 \
  --ulysses-degree 8 \
  --ring-degree 2 \
  --encoder-parallel replicate \
  --performance-mode speed \
  --host 0.0.0.0 \
  --port 30010
```

**What was measured.** A controlled denoise-stage comparison on identical
hardware: 8× H200 single-node (Ulysses8, no Ring) versus the same 16-GPU
cross-node command above (Ulysses8 × Ring2), holding prompt, seed, and
step count fixed:

| Task | Single-node (Ulysses8) | Cross-node (Ulysses8 × Ring2) | Change |
| - | -: | -: | -: |
| T2VA denoise/step | 0.749 s | 0.477 s | −36.3% |
| Ref2VA/V2V denoise/step | 2.572 s | 1.494 s | −41.9% |

The gain grows with sequence length because Ring's per-hop communication
cost stays roughly constant while attention compute grows quadratically
with sequence length, so V2V's longer packed sequence benefits more than
T2VA's shorter one. With the point-to-point KV rotation pipelined against
attention compute, one V2V request's full denoise stage completed in
68.1–68.3 seconds versus 128.6 seconds on the single-node 8-GPU baseline
(−47.0%), with byte-identical output to the unpipelined cross-node path.

Cross-node determinism was confirmed separately: the same request run
twice against the same cross-node deployment produced byte-identical
output. A cross-node run's output is not expected to bit-match a
single-node run of the same prompt and seed — Ring's online-softmax merge
across hops accumulates floating-point operations in a different order
than single-node attention, which is an expected source of bit-level
difference, not a correctness regression.

> [!WARNING]
>
> `--encoder-parallel auto`'s fold decision is not yet node-boundary aware
> and attempts to fold the text encoder across nodes, which crashes the
> Ref2VA reference-conditioned encoder. Always pass
> `--encoder-parallel replicate` explicitly for cross-node H3 deployments.

### H100 topology comparison

The same four-card H100 host completed three lossless placements. TP2 with
Ulysses2 was the fastest; TP4 used the least memory:

| Topology | Pipeline latency | Peak/GPU |
| - | -: | -: |
| TP2 + Ulysses2 | 13.25 s | 66.04 GB |
| FSDP + Ulysses4 | 13.36 s | 57.01 GB |
| TP4 + Ulysses1 | 13.86 s | 49.80 GB |

### RTX 5090 capacity run (two cards)

The single-card recipe is a different one — see *RTX 5090 single card, physical
desktop* below; the twenty resident layers here are half a layer per GPU under
TP2 and do not transfer to one card. The verified two-card RTX 5090 host used TP2 with layerwise offload. The full
50-step, 1344×768, 5-second request completed in 559.67 seconds: 525.05
seconds of denoising and 33.61 seconds of decoding, with a 26.3 GiB sampled
peak per GPU.

| DiT settings | 5-step denoise | Inference | Peak/GPU | Result |
| - | -: | -: | -: | - |
| prefetch 1, resident 20 | 43.48 s | 78.11 s | 26.3 GiB | Selected recipe |
| prefetch 2, resident 20 | 43.37 s | 78.06 s | 27.5 GiB | No measurable gain |
| Ulysses2, prefetch 2, resident 10 | Did not reach warmup | — | — | Rejected |

### Consumer GPU tuning

On consumer hardware the binding question is not which card you have but how much
host RAM sits behind it. H3's weights are about 108 GB — 61.73 GB of DiT and
46.18 GB of text encoder — so no consumer configuration holds them all, and where
the shortfall lands decides the throughput.

**The command** — most consumer machines need exactly one flag beyond the model:

```bash consumer single GPU, lossless theme={null}
sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --layerwise-offload-components dit,text_encoder,vae
```

With 16 GB of VRAM or more, add `--layerwise-resident-layers video_vae=36` for
the 13 s decode; with \~96 GB of host RAM and 16 GB+ of VRAM, add
`--dit-layerwise-resident-layers 4` for the 6 s step. That is the whole flag
surface. The [builder at the top of this page](#1-quick-start) has consumer
cards and a Host RAM selector: pick your budget and it emits this command with
your tier's measured expectations attached as comments. The table below is the
same data in one view.

**Unified-memory machines are the exception**: on a DGX Spark (GB10, 128 GB
shared between CPU and GPU) launch with **no flags at all**. The deployment
still exceeds the pool, automatic offload engages on its own, and its placement
beat the explicit recipe above by 2.1× on the denoise (12.1 vs 25.8 s/it
measured on the same box) — the VRAM/host split that justifies every flag in
this section does not exist there. Expect \~12 min of load, a \~5.5 min text
encoding stage per request (steady-state compute on this chip, not a stall),
and \~12 min per warm 480P request.

**Two budgets, and what each one buys**

| | 12 GB VRAM + 32 GB host | host free, VRAM 16 GB |
| - | -: | -: |
| Recipe | A | B |
| Peak VRAM | ≤ 12 GiB | ≤ 16 GiB (OOMs at 12) |
| Host anonymous (must fit) | 24.5 GiB | 116.7 GB pinned |
| Denoise, 864×480 / 124 frames / 20 NFE | 16.8 - 18.7 s/it | **6.01 s/it** |
| Runs at all | yes | yes |

The left column is one configuration measured twice, at 318.94 s and 356.37 s;
the 12% spread tracked host load on a shared machine, so treat smaller
differences than that as unresolved. The right column is 120.92 s at a 16 GiB
allocator cap. Four resident DiT layers is what Recipe B buys its speed with,
and it is also why 12 GiB is not enough for it.

Read the host row carefully, because the two numbers are not the same kind of
memory. *Anonymous* host memory — pinned buffers and pageable copies — has to fit,
and the kernel cannot reclaim it. Page cache backing a file mapping is
*droppable*, so it does not count against the budget even though it shows up in
`VmRSS`; use `RssAnon` from `/proc/<pid>/status` when checking. Likewise measure
VRAM with `torch.cuda.set_per_process_memory_fraction` and let the allocator fail,
rather than reading `nvidia-smi`, which reports the caching allocator's reserved
pool and overstates the requirement.

Inside 32 GB the weights cannot be pinned, so each denoise step copies about
60 GiB from the checkpoint mapping, and a mapped source is synchronous however
the copy is requested: the driver stages it through its own buffer, so the
transfer neither overlaps compute nor runs at pinned bandwidth. That is where
the step goes, and giving the host room to pin the weights instead is what takes
it to 6.01 s.

Two caveats on the constrained number, both from instrumenting the run rather
than from arithmetic. The machine it was measured on has 2 TB of host memory, so
the kernel kept all 107.7 GiB of mapped checkpoint pages resident: major faults
across a whole request were 6, and `read_bytes` was zero. Nothing was read from
disk. A real 32 GB host cannot cache 107.7 GiB, so it will fault and re-read,
and should be expected to be slower than the figures here rather than equal to
them — an NVMe is a requirement, not a recommendation. Measure your own machine
with major faults (`/proc/<pid>/stat`) on the worker process, not on the
launcher, which holds no weights.

**Recipe A — fits 12 GB VRAM + 32 GB host**

```bash 12 GB + 32 GB, lossless theme={null}
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --layerwise-resident-layers video_vae=36
```

**Recipe B — host memory is free (the fast path)**

```bash unconstrained host, lossless theme={null}
sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --dit-layerwise-resident-layers 4 \
  --layerwise-resident-layers video_vae=36
```

Recipe B pins \~112 GB of host memory (DiT 61.56 GB, text encoder 46.18 GB, VAE
\~4.5 GB in its decode dtype). Do not reach for it on a 32 GB machine.

**What not to change, and why**

* `video_vae=36` holds every decoder block for the decode only — residency
  arms at the decoder's first block and releases when it finishes, so the
  denoise still runs on an empty card. It fits 12 GB because decoder weights
  are held in their decode compute dtype (fp16) from load, which halves them
  to \~4.9 GiB; the rounding was already part of every output (the decode
  computes in fp16 autocast), so the result is bit-identical, and the decode
  drops from 60 s streamed (or 209 s on a busy host) to \~10 s. The
  `expandable_segments` line stays: the decode sits close enough to the cap
  that fragmentation otherwise tips it over.
* Leave `--enable-torch-compile` off, as elsewhere on this page. Layerwise offload
  rebinds `param.data` on every layer, so compiled graphs do not get the benefit
  they would on resident weights.
* Recipe A's flags are what the automatic policy should choose on its own. Until
  the model declares its own placement, `--performance-mode memory` plus the
  explicit component list is what makes it happen; pass them.

**Reading the startup log**

The server prints the memory decisions it made; checking three lines against
your budget catches a mis-set machine in the first minute instead of the first
request.

* `Layerwise offload: host memory available: N GiB` — what the runtime sees
  after loading, not your DIMM size. On a 32 GB host expect single digits here;
  a much larger number means another process's memory accounting (or a
  container limit) is in play.
* `leaving N GiB of weights on the checkpoint mapping` — the expected line on a
  32 GB host: the DiT streams from the checkpoint file. If instead the log
  reports pinned weights, the runtime decided your host has room — which is
  faster, and means the 32 GB figures above do not apply to you.
* `Loaded video_vae: ... host mmap` vs `host pageable` — where the VAE landed
  (decoder weights are \~4.9 GiB once held in their decode dtype).
  `Loaded <component>` lines carry the same buckets for every component.

If a request dies after the denoise finishes, it is the decode colliding with
the cap: keep the `expandable_segments` line, and if it persists drop to
`video_vae=24` and take the partially streamed decode.

**Against ComfyUI, on the same weights**

Same unpruned bf16 checkpoints, same card, same sampler settings (cfg 1.0,
euler\_ancestral, sigma shift 12.0/3.0, seed 1101), 864×480 / 124 frames / 20 NFE:

When host memory is free, the engines are close and sglang is ahead:

| | denoise | host anonymous | peak VRAM |
| - | -: | -: | -: |
| sglang, Recipe B | **6.01 s/it** | 116.7 GB pinned | ≤ 16 GiB |
| ComfyUI KSampler | 6.58–6.59 s/it | 116.5 GiB | 13048 MiB |

Inside 12 GB, both engines run these weights, and one measurement convention
matters on each side. ComfyUI's memory manager reads system RAM and adapts, so
the rows below patch `psutil` to a pretend host size — the same convention the
sglang rows use. Its `--reserve-vram` is also soft: told to keep 12 GiB free it
still peaked at 13.5 GiB, a figure a real 12 GB card cannot give it, so both
engines here run under the same hard allocator cap
(`set_per_process_memory_fraction`), where its peak stays at 12.1–12.3 GiB.
Under that cap, Recipe A wins the whole request at every host size:

| 12 GB VRAM, both engines hard-capped | sglang Recipe A (TE + denoise + decode) | ComfyUI, bf16 (warm) |
| - | -: | -: |
| 32 GB host | 12.4 + 212.8 + 9.4 ≈ **235 s** | 276–302 s |
| 48 GB host | 15.8 + 192.1 + 10.0 ≈ **218 s** | 246–267 s |
| 64 GB host | 7.5 + 162.4 + 10.3 ≈ **180 s** | 194–195 s |

Same GPU, same load window, unpruned bf16 checkpoints, outputs verified. All
figures are anchored at 480P — activations grow with the pixel count, so at
768P drop the resident DiT layers to 0 first, then `video_vae` to 24 if the
decode still collides.

One convention behind that table is the one a home machine does not share: the
host sizes above are `psutil` patches on a 2 TB lab machine, so the kernel kept
the whole checkpoint in page cache and nothing was read from disk. A physical
host of that size re-reads the streamed weights every step, and what that costs
is computable — `bytes per step = (50 - resident DiT layers - layers the host
pin budget covered) x 1.23 GiB`. The courier overlaps that read with compute, so
the drive stops being the denoise clock once its sequential rate exceeds those
bytes divided by the card's step time. A 32 GB host covers roughly 13 layers,
which puts a 24 GB card at \~38 GiB/step and a 12/16 GB card at \~46 GiB/step: a
PCIe 4.0 NVMe (6–7 GiB/s) clears both, a PCIe 3.0 drive (\~3.2 GiB/s) becomes the
clock, and a SATA SSD is not usable for this model. Resident DiT layers remove
1.23 GiB/step each, but only while the streamed set is larger than the pin
budget — once the read fits under the step's compute time, more of them buy
nothing and only cost VRAM.

An earlier revision of this page recorded 38 s/step on a physical 4090 with a
990 Pro, reading 52.9 GB per step — about 1.4 GB/s, and faulted sequentially, so
almost none of it showed in `majflt` (measure `read_bytes`, not major faults).
That rate was the old path, not the drive: the pinned-store pool rounded every
block up to a power of two, so the pin budget over-committed and the machine had
to run with pinning off, and the layers that stayed mapped faulted in 4 KiB at a
time. Exact-size pinning and O\_DIRECT reads through the courier removed both,
and the physical 5090 run below shows the drive fully hidden behind compute. A
4090 on a 32 GB host is derived to return to its compute wall (\~8.5 s/step) on a
PCIe 4.0 drive; that machine has not been re-measured. The
VRAM axis holds too: capped at 16 GiB the same recipe wins \~250 vs 292–301 s,
and at 24 GiB (with `--dit-layerwise-resident-layers 6` — measured at a
22 GiB cap so a desktop's own allocations fit; a headless card can raise it
to 10 for under 1% more) \~8.5 s/step vs ComfyUI's 249–260 s requests. Four changes carry it: the VAE staying on its checkpoint mapping (#35862, root fix
\#35946), per-layer pinning with net-cost accounting (#35867), the courier
thread that ships still-mapped layers through pinned slots (#35882), and
decoder weights held in their decode dtype from load (#35967) — which is what
lets `video_vae=36` fit and turns the decode from the slowest stage (54–96 s
streamed) into the fastest (\~10 s, faster than ComfyUI's own 15–25 s). Output
equivalence is bit-level: the fp16-held decode reproduced the fp32-store run's
video byte for byte, and the audio stream is bit-identical.

Stage by stage under the cap: text encoding is even (both stream the same
48 GB Qwen3VL), the denoise leads at 32–48 GB hosts and sits within
run-to-run variance of ComfyUI at 64 GB (162 vs 159 s), and the decode leads
everywhere. Two ComfyUI notes that still matter: `--fast-disk` measured no
faster than its default on this lab host — though on a physical 60 GB desktop
its default is OOM-killed while loading (55.6 GB of anonymous memory) and
`--fast-disk` is what makes it run at all — and stacking
`--novram --cache-none --disable-pinned-memory` made things strictly worse
(69.1 GiB anonymous, 750 s requests) — the adaptive default is the right
configuration on a small host.

The path ComfyUI ships for 12 GB cards uses
`minimax_h3_fl2va_pruned_int8_convrot` and `qwen3vl_32b_minimax_h3_nvfp4_awq`,
i.e. an int8 DiT and an NVFP4 text encoder, and its pruned bf16 file is 40.2 GB
against the unpruned 66.3 GB. Those are different weights, so it is not a
like-for-like comparison with the recipes above.

### RTX 5090 single card, physical desktop

Every consumer figure above this point was measured with at least one axis
simulated: a hard allocator cap standing in for VRAM, a `psutil` patch standing
in for host RAM, and a lab machine whose page cache held the whole checkpoint.
This run had none of that — an RTX 5090 (32 GB) in a Ryzen 9 9950X desktop with
60 GB of RAM, an 8 GB swap file and a Samsung 9100 PRO on PCIe 5.0, on Ubuntu
24.04. The workload matches the comparison above: FL2VA T2VA, 864×480, 124
frames, 20 steps, cfg 1.0, euler\_ancestral, sigma shift 12.0/3.0, same seeds and
keyframes on both engines, unpruned bf16 weights on both sides.

```bash 32 GB VRAM + 60 GB host, lossless theme={null}
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --dit-layerwise-resident-layers 14 \
  --layerwise-resident-layers video_vae=36 \
  --warmup-resolutions 864x480 --warmup-num-frames 124
```

| | text encoding | denoise step | decode | request |
| - | -: | -: | -: | -: |
| sglang | **4.2 s** | 5.14–5.17 s | **6.4 s** | **112.2 / 112.1 s** |
| ComfyUI `--fast-disk --cache-none` | 11.3 s | 5.17–5.30 s | 7.2 s | 145.9 / 145.6 / 140.9 s |

Three things a capped run cannot settle:

**The denoise is at the card's compute wall, not the drive's.** The two engines
are within noise of each other per step, and prefetch depth 2 and cuDNN SDPA
each measured as no change. The 51.3 GiB the DiT reads from the drive per
request is fully overlapped — across a whole request the compute thread waited
0.9 s for the courier.

**Host RAM decides how much traffic there is to hide.** At 60 GB the pin budget
covers 34 of the 50 DiT layers, so only about two layers per step come off the
disk. The 46 GiB text encoder is never pinned — streamed layers take the budget
first — and is read whole, once per request, at the drive's sequential rate:
that is the 4.2 s stage, against 12.1–12.9 s when the same bytes go through the
page cache.

**Neither engine's default command survives this machine.** ComfyUI's is
OOM-killed during load (55.6 GB of anonymous memory) and needs `--fast-disk`;
sglang's was too, until pinned stores were allocated at their exact size instead
of the allocator's next power of two. Even now the automatic placement keeps no
resident DiT layers and streams the VAE on every decode, so the flags above are
what reach the numbers in the table.

Two caveats to carry into your own machine. The run finished with its 8 GB swap
file full, because the pin budget takes 95% of the memory available after
loading and there is no smaller setting yet — on a machine that is also your
desktop, expect that pressure and give the card more resident layers, since a
resident layer is one the host no longer has to pin. And these are 480P figures:
at 768P drop the resident DiT layers first, then `video_vae` to 24.

### RTX 4090 24 GB single-GPU run

One RTX 4090 D 24 GB completed the 1344×768, 107-frame, 20-NFE T2VA
workload (euler, `torch.compile` and step caching disabled) with DiT and
text-encoder layerwise offload. Same process: load → warmup (seed 0) →
timed (seed 42); only the timed pass is reported. GPU peak stayed about
18 GB.

| Config | Timed e2e | Denoise | vs BF16 | PSNR vs BF16 |
| - | -: | -: | -: | -: |
| BF16 + FlashAttention | 405.6 s | 370.2 s | 1.00× | — |
| `convrot_int8` + FA | 303.3 s | 273.7 s | 1.34× | 24.81 dB |
| `convrot_int8` + `sol_attn` | 223.9 s | 203.9 s | 1.81× | 24.44 dB |
| `convrot_int8` + `sage_attn` | 174.9 s | 154.2 s | 2.32× | 23.51 dB |
| `convrot_int8` + Sage→Sol hybrid | 163.8 s | 143.1 s | 2.48× | 23.04 dB |

`convrot_int8` + FA changes Linear numerics only. The `sol_attn` /
`sage_attn` / hybrid rows also change the attention algorithm, so speed
and pixel fidelity rank in opposite orders there. Default remains
`convrot_int8` + `fa`. Cache-DiT can share this layerwise recipe; omit
`quality` and see the quality-level section.

Same card, 50-step run. Speedup is vs BF16+FA. PSNR is vs same-tree INT8+FA (Spectrum off).

| Config | e2e | vs BF16+FA | PSNR mean / min |
| - | -: | -: | -: |
| BF16 + FlashAttention | 1008.6 s | 1.00× | — |
| `convrot_int8` + FA | 718.5 s | 1.40× | — |
| `convrot_int8` + FA + Spectrum `11 / 5 / 1.0` | **285.3 s** | **3.53×** | **27.4 / 25.3** |
| `convrot_int8` + `sage_attn` + Spectrum `15 / 5 / 0` | 245.3 s | 4.11× | 23.6 / 21.0 |
| `convrot_int8` + Sage→Sol hybrid + Spectrum `15 / 5 / 0.5` | **219.0 s** | **4.61×** | 20.6 / 17.8 |

Prefer FA + `11 / 5 / 1.0` when the bar is FA-like. Sage / hybrid + Spectrum is faster; quality then follows the attention backend, not the skip schedule.

### AMD Instinct task and scaling runs

The AMD recipes keep the released BF16/FP32 precision policy and use AITER
packed attention. The picker emits the fastest measured topology, 8 GPUs with
Ulysses degree 8. All runs below completed full H.264/AAC decoding and
representative-frame inspection.

| Hardware | Task | Denoise | Decode | Peak/GPU |
| - | - | -: | -: | -: |
| MI355X | T2VA | 55.2907 s | 9.5344 s | 97,444 MB |
| MI355X | FL2VA | 53.7978 s | 9.4477 s | 96,922 MB |
| MI355X | Ref2VA | 41.3812 s | 6.8247 s | 94,518 MB |
| MI300X | T2VA | 167.4878 s | 25.3244 s | 97,272 MB |
| MI300X | FL2VA | 150.2311 s | 12.5684 s | 96,750 MB |
| MI300X | Ref2VA | 107.6232 s | 11.3768 s | 94,268 MB |

The task matrix used 8 GPUs and 50 denoising steps. The scaling matrix uses
one 1344×768, 209-frame T2VA request and changes only the GPU count and
matching Ulysses degree:

| Hardware | GPUs | Denoise | Decode | Peak/GPU |
| - | -: | -: | -: | -: |
| MI355X | 8 | 55.2907 s | 9.5344 s | 97,444 MB |
| MI355X | 4 | 104.2294 s | 11.1824 s | 103,350 MB |
| MI355X | 2 | 223.0246 s | 15.5330 s | 115,250 MB |
| MI355X | 1 | 288.7968 s | 24.0472 s | 137,676 MB |
| MI300X | 8 | 167.4878 s | 25.3244 s | 97,272 MB |
| MI300X | 4 | 297.3727 s | 26.5067 s | 103,436 MB |
| MI300X | 2 | 585.5401 s | 29.4909 s | 115,010 MB |
| MI300X | 1 | 978.0886 s | 36.0142 s | 137,626 MB |

For a measured lower-count AMD deployment, set both `--num-gpus` and
`--ulysses-degree` to 4, 2, or 1. AITER packed attention matched segment-wise
BF16 SDPA at cosine similarity `0.9999991655` on MI355X and `0.9999991059` on
MI300X.

## 10. Run in ComfyUI

Run this model from ComfyUI with the
[SGLDiffusion plugin](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion), which ships in the SGLang
repository at `python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion`.

**Server mode** — SGLang runs the pipeline and ComfyUI
sends the request. Start a server as shown above, point the
`SGLDiffusion Server Model` node at it, then generate with
`SGLDiffusion Generate H3`.
This path has been run end to end against a live server.

**Integrated mode** — not available for this model
: H3 denoises a packed video-and-audio sequence in one pass and routes conditioning by task, while ComfyUI's KSampler drives a single latent tensor and has no audio branch
. Use server mode.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.