Qwen-Image-Edit Face-to-Image Generation Model

This model is re-uploaded from an external source: Original Source Link

Model Overview

This model is a face-guided image generation model trained on top of Qwen-Image-Edit. It can directly generate beautiful full-body photos based on an input face image.

Please note: The input image for this model must be a cropped face image. Do not include any areas or content other than the face in the input image.

• Training Data: Provided by Maiju
• Model Architecture: LoRA
• Base Model: Qwen-Image-Edit
• Training Code: DiffSynth-Studio

Showcase

Prompt: Photography. A young woman wearing a yellow dress, standing in a flower field, with colorful flowers and green grass in the background.

Input Image	Generated Image 1	Generated Image 2	Generated Image 3

Prompt: Photography. A young beautiful woman wearing light green and white traditional Chinese attire with flowing ribbons, holding a long sword, standing in a traditional corridor with dappled light and shadows, elegant and graceful.

Input Image	Generated Image 1	Generated Image 2	Generated Image 3

Prompt: A young woman wearing a black leather jacket and blue jeans, standing in an industrial-style building with red brick walls and metal structures, sunlight pouring in, natural expression.

Input Image	Generated Image 1	Generated Image 2	Generated Image 3

Prompt: A young woman in an elegant red gown, holding a book, wearing a silver necklace, with an elegant and dignified expression, background of the Arc de Triomphe in Paris.

Input Image	Generated Image 1	Generated Image 2	Generated Image 3

Inference Code

Install DiffSynth-Studio:

git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .

Inference code:

from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
import torch
from PIL import Image

pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image-Edit", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    tokenizer_config=None,
    processor_config=ModelConfig(model_id="Qwen/Qwen-Image-Edit", origin_file_pattern="processor/"),
)
pipe.load_lora(pipe.dit, lora_config=ModelConfig(model_id="DiffSynth-Studio/Qwen-Image-Edit-F2P", origin_file_pattern="model.safetensors"))

face_image = Image.open("face_image.png").convert("RGB")
image = pipe(
    prompt="Photography. A young woman wearing a yellow dress, standing in a flower field, with colorful flowers and green grass in the background.",
    negative_prompt="deformed or distorted fingers, enlarged head with abnormal head-to-body ratio, short bobblehead character, harsh glaring sunlight, over-saturated colors, legs twisted into knock-knees or bow-legs",
    edit_image=face_image,
    seed=0,
    num_inference_steps=40,
    height=1152, width=864,
)
image.save("image.jpg")

In addition, we provide the following code to crop the face area from a portrait photo as the input for this model:

from modelscope import snapshot_download
from insightface.app import FaceAnalysis
from PIL import Image
import numpy as np
import cv2

def initialize_face_detector():
    snapshot_download("ByteDance/InfiniteYou", allow_file_pattern="supports/insightface/*", cache_dir="models")
    face_detector = FaceAnalysis(name='antelopev2', root="models/ByteDance/InfiniteYou/supports/insightface")
    face_detector.prepare(ctx_id=0, det_size=(640, 640))
    return face_detector

def crop_face(face_detector, image):
    face_info = face_detector.get(cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR))
    bbox = sorted(face_info, key=lambda x: (x['bbox'][2] - x['bbox'][0]) * (x['bbox'][3] - x['bbox'][1]))[-1]['bbox']
    face_image = image.crop(list(map(int, bbox)))
    return face_image

face_detector = initialize_face_detector()
image = Image.open("photo.jpg")
crop_face(face_detector, image).save("face.jpg")