moss-control.safetensors

---

license: apache-2.0 base_model:

• OpenMOSS-Team/MOSS-Image-Base
• OpenMOSS-Team/MOSS-Image-SFT pipeline_tag: text-to-image tags:
• moss-image
• controlnet
• control
• depth
• segmentation
• regional-prompting
• lora

---

MOSS-Image Color-Depth Control (unofficial)

A control adapter for MOSS-Image that makes generation follow a color region map (where each object is) and a depth map (its 3D shape and front/back order). Same idea and recipe as darask0/Krea-2-color-depth-control, on the MOSS-Image DiT. Made for posing scenes with 3D mannequins and for region-painted layouts, together with per-region prompts.

Unofficial: a community adapter, not made or endorsed by OpenMOSS. Apache-2.0 like the base model. The depth maps used for training were made with Depth Anything V2 Large (CC-BY-NC-4.0) and the color maps with Mask2Former Swin-L; the released weights are the adapter only.

Two people from a 3D mannequin render (region prompts + control). Top row: 5,000 steps with the control on for the whole run, ending at 0.5, ending at 0.3, and the base model without control. Bottom row: the same at 10,000 steps. With the control on for the whole run the mannequin surface shows through; ending it at 0.5 gives real people at the mapped places.

Files

File
moss-control.safetensors	Recommended. 10,000 steps.
moss-control-002500 / 005000 / 007500.safetensors	Intermediate steps. Earlier = looser, looks less like the mannequin; later = tighter to the map.
examples/	Two scenes exported from a three.js mannequin editor: clay.png (shaded render), color.png (color pass), depth.png (depth pass), and regions.json for Hayate (edit the /path/to/ prefixes).
images/	Result sheets for the two scenes (control only and with region prompts).
training/config.toml, training/moss_control.py	Training configuration and code (hayate-train).

What it does

Region prompts alone tell MOSS-Image what goes in each region, but placement and scale drift. Image-to-image from a 3D render keeps the layout but carries the render's flat background and gray color. With this adapter the controls are conditions, not image content: generation stays text-to-image from pure noise, and the output follows the map's placement, scale and pose, with a background described only by the prompt.

Inputs

• Color map: an RGB image at the output size where each object or region has its own flat color. The colors are IDs only. Black is fine for "unspecified". Training maps came from COCO panoptic segmentation.
• Depth map: grayscale at the output size, near = white.

Either input may be omitted: the adapter was trained with control dropout (color only 35%, depth only 35%, both 20%, none 10%).

Recommended settings

• Use it together with region prompts (one prompt per color). With only a scene prompt, a mannequin-shaped map is drawn as a mannequin.
• color_end and depth_end = 0.5 for 3D mannequin or other synthetic inputs: the control acts on the first half of the steps (layout, pose), then the model finishes the surfaces. 0.3 gives almost the same result. Use 1.0 for maps from real images.
• MOSS-Image SFT with the 4-step DMAD adapter (darask0/MOSS-Image_4-step_adapter_lora): about 1.8 s per 1024² image on an RTX PRO 6000.

Format

Weights are BF16 safetensors keyed by the MOSS-Image transformer module names.

• video_embedder.weight [3072, 384] and video_embedder.bias [3072] replace the transformer's video_embedder (originally [3072, 128]). The input is the channel concatenation of three 128-channel latents (FLUX.2 VAE latents, 2x2 packed and batch-norm normalized like the image latent):
  1. the noisy latent (as usual),
  2. the color map latent,
  3. the depth map latent (depth replicated to RGB, pixels scaled to [-1, 1]; posterior mean). A zero latent means "no control" for that input. The output head is unchanged.
• *.lora_A.weight / *.lora_B.weight: rank 64, alpha 64, on the block linears (layers.*.proj.{q,k,v,o,ffn_up_gate,ffn_down}) and the text embedder blocks.
• Metadata: hayate_control=1, base_model=moss-base, control_channels=color,depth, control_preprocess=vae.
• In Hayate, strength scales the LoRA (video_embedder is never scaled). Per-input end fractions (color_end, depth_end) zero a control after that fraction of the steps.

Trained on the MOSS-Image Base transformer; used with MOSS-Image SFT and the 4-step DMAD adapter.

See the full description on the original page