Diffusers documentation
Anima
Anima
Anima adalah model teks-ke-gambar yang menggunakan ulang CosmosTransformer3DModel dengan encoder teks Qwen3, kondisioner teks T5-token, dan VAE AutoencoderKLQwenImage.
import torch
from diffusers import ModularPipeline
pipe = ModularPipeline.from_pretrained("circlestone-labs/Anima-Base-v1.0-Diffusers")
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda") # or "mps", "xpu", "cpu"
image = pipe(prompt="masterpiece, best quality, 1girl, solo, city lights").images[0]AnimaModularPipeline
kelas diffusers.AnimaModularPipeline
< source >( blocks: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = Nonepretrained_model_name_or_path: str | os.PathLike | None = Nonecomponents_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = Nonecollection: str | None = Noneworkflow: str | None = Nonemodular_config_dict: dict[str, typing.Any] | None = Noneconfig_dict: dict[str, typing.Any] | None = None**kwargs )
ModularPipeline untuk Anima.
AnimaAutoBlocks
Pipeline Modular otomatis untuk pembuatan teks-ke-gambar dan gambar-ke-gambar menggunakan Anima.
Alur kerja yang didukung:
text2image: membutuhkanpromptimg2img: membutuhkanimage,prompt
Komponen:
text_encoder (Qwen3Model) tokenizer (Qwen2Tokenizer) t5_tokenizer (T5Tokenizer) guider
(ClassifierFreeGuidance) vae (AutoencoderKLQwenImage) image_processor (VaeImageProcessor)
text_conditioner (AnimaTextConditioner) transformer (CosmosTransformer3DModel) scheduler
(FlowMatchEulerDiscreteScheduler)
Inputs:
prompt (str):
The prompt or prompts to guide image generation.
negative_prompt (str, optional):
The prompt or prompts not to guide the image generation.
max_sequence_length (int, optional, defaults to 512):
Maximum sequence length for prompt encoding.
image (Image | list, optional):
Reference image(s) for denoising. Can be a single image or list of images.
height (int, optional):
The height in pixels of the generated image.
width (int, optional):
The width in pixels of the generated image.
generator (Generator, optional):
Torch generator for deterministic generation.
num_images_per_prompt (int, optional, defaults to 1):
The number of images to generate per prompt.
image_latents (Tensor, optional):
image latents used to guide the image generation. Can be generated from vae_encoder step.
num_inference_steps (int):
The number of denoising steps.
sigmas (list, optional):
Custom sigmas for the denoising process.
strength (float, optional, defaults to 0.9):
Strength for img2img/inpainting.
latents (Tensor):
Pre-generated noisy latents for image generation.
*denoiser_input_fields (None,optional):
The conditional model inputs for the Anima denoiser.
output_type (str,optional*, defaults to pil):
Output format: ‘pil’, ‘np’, ‘pt’.
Output:
images (list):
Gambar yang dihasilkan.
AnimaTextConditioner
kelas diffusers.AnimaTextConditioner
< source >( source_dim: int = 1024target_dim: int = 1024model_dim: int = 1024num_layers: int = 6num_attention_heads: int = 16mlp_ratio: float = 4.0target_vocab_size: int = 32128use_self_attention: bool = Trueuse_layer_norm: bool = Falsemin_sequence_length: int = 512 )
Text conditioner yang digunakan oleh Anima untuk memetakan hidden states Qwen3 dan token ids T5 ke Cosmos text embeddings.
Anima menggunakan ulang Cosmos Predict2 DiT. Satu-satunya modul kondisioning yang spesifik untuk model adalah adapter LLM ini, yang melakukan cross-attention dari embedding token T5 yang dipelajari ke hidden states encoder teks Qwen3 sebelum loop difusi. target_dim adalah dimensi output kondisioner dan harus cocok dengan text_embed_dim dari transformer.
forward
< source >( source_hidden_states: Tensortarget_input_ids: Tensortarget_attention_mask: typing.Optional[torch.Tensor] = Nonesource_attention_mask: typing.Optional[torch.Tensor] = None ) → torch.Tensor
Parameter
- source_hidden_states (
torch.Tensordengan bentuk(batch_size, source_sequence_length, source_dim)) — Hidden states dari text encoder Qwen3 untuk dijadikan kondisi. - target_input_ids (
torch.Tensordengan bentuk(batch_size, target_sequence_length)) — ID token T5 yang digunakan sebagai token query yang dipelajari. - target_attention_mask (
torch.Tensor, opsional) — Attention mask untuk id token T5 target. - source_attention_mask (
torch.Tensor, opsional) — Attention mask untuk hidden states Qwen3 sumber.
Mengembalikan
torch.Tensor
Embedding kondisioning teks untuk transformer Cosmos.