MiniMax-H3-Fun-Controlnet-Union-2.0.safetensors

---

license: other license_link: LICENSE.md license_name: minimax-h3-community-license-agreement library_name: videox_fun tags:

• controlnet
• controlnet-union
• video-to-video
• image-text-to-video
• text-to-video
• video-inpainting tasks:
• text-to-video-synthesis

---

MiniMax-H3-Fun-Controlnet-Union-2.0

What's new in 2.0

	MiniMax-H3-Fun-Controlnet-Union (v1)	MiniMax-H3-Fun-Controlnet-Union-2.0 (this model)
Control conditions	5 — Canny, Depth, HED, MLSD, Pose	8 — + Scribble, Layout, Gray
Control branch depth	5 control blocks (layers 0, 10, 20, 30, 40)	10 control blocks (layers 0, 5, 10, …, 45) — skips injected every 5 of the 50 transformer blocks
Inpaint masked-pixel recipe	pre_norm (holes ≈ −2 in VAE input space, extreme dark)	post_norm (holes at 0, mid-gray, following Wan 2.1) — cleaner inpaint blending
Checkpoint contents	control_proj_in + 5 control_blocks (~6.8 GB)	control_proj_in + 10 control_blocks (~13.5 GB)
Required config	minimax_h3_control.yaml	minimax_h3_control_inpaint_post_norm.yaml

Everything else is carried over from v1: control_in_dim = 49 (latent + masked latent + mask, so the same branch does control and inpaint), control_apply_audio = false, guidance-distilled (guidance_scale = 1.0), and the same zero-gated skip-add into the main branch.

Loading a v1 config against this checkpoint is a silent failure. With minimax_h3_control.yaml (5 blocks) the model builds only half the control branch; load_state_dict(strict=False) drops control_blocks.5~9 as unexpected keys and misplaces the rest, producing wrong outputs. Always use minimax_h3_control_inpaint_post_norm.yaml.

Model Card

Name	Description
MiniMax-H3-Fun-Controlnet-Union-2.0.safetensors	ControlNet-Union-2.0 branch weights for MiniMax-H3. Holds only the control branch (control_proj_in plus 10 control_blocks, about 13.5 GB) and is loaded on top of the base MiniMax-H3 transformer. One checkpoint supports 8 control conditions (Canny, Depth, HED, MLSD, Pose, Scribble, Layout, Gray) and video inpainting.

Model Features

• Union control over 8 conditions: one checkpoint handles Canny, Depth, HED, MLSD, Pose, Scribble, Layout and Gray control videos for video-to-video generation — no per-condition checkpoint switching.
• Denser control injection: the control branch attaches to 10 of the 50 transformer blocks (layers 0, 5, 10, 15, 20, 25, 30, 35, 40, 45); every control skip is added to the main branch through a zero-gated projection. This is roughly 2× the injection points of v1 and gives tighter structural adherence.
• Guidance-distilled: run with guidance_scale = 1.0, one forward pass per step, no classifier-free guidance needed.
• Inpainting is supported, with the post_norm recipe: the control input is widened to control_in_dim = 49 (latent + masked latent + mask channels). Unlike v1, the masked pixels are zeroed after the ImageNet normalization (holes sit at 0 / mid-gray) rather than before it (holes landed near −2 / extreme dark), which improves how filled regions blend with kept regions. Use examples/minimax_h3_fun/predict_v2v_control_inpaint.py.
• control_context_scale scales every control skip before it is added to the main branch: 1.0 gives the strongest control (used for all results below), values below 1.0 weaken the guidance of the control video, 0.0 switches the control branch off.
• The generation follows the control video: the frame count snaps down to the largest 17 * n + 5 the video VAE can decode (duration capped at 15 seconds), the canvas keeps the control video's own aspect ratio at the height * width pixel budget (both multiples of 32), at a fixed 24 fps.
• Detailed prompts give better stability; we recommend describing the scene, the subject and the camera in the prompt.

Supported control conditions

Condition	Control signal	New in 2.0?
Canny	Canny edge map
Depth	Monocular depth map
HED	HED edge detection
MLSD	Line-segment detection
Pose	DWPose skeleton
Scribble	Free-hand / sketch lines	✅
Layout	Bounding-box layout	✅
Gray	Grayscale (luminance) video	✅

See the full description on the original page