Diffusers

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.42.1).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

AllegroTransformer3DModel

Un modelo Diffusion Transformer para datos 3D de Allegro fue introducido en Allegro: Open the Black Box of Commercial-Level Video Generation Model por RhymesAI.

El modelo se puede cargar con el siguiente fragmento de código.

from diffusers import AllegroTransformer3DModel

transformer = AllegroTransformer3DModel.from_pretrained("rhymes-ai/Allegro", subfolder="transformer", dtype=torch.bfloat16).to("cuda")  # or "mps", "xpu", "cpu"

AllegroTransformer3DModel

clase diffusers.AllegroTransformer3DModel

< source >

( patch_size: int = 2patch_size_t: int = 1num_attention_heads: int = 24attention_head_dim: int = 96in_channels: int = 4out_channels: int = 4num_layers: int = 32dropout: float = 0.0cross_attention_dim: int = 2304attention_bias: bool = Truesample_height: int = 90sample_width: int = 160sample_frames: int = 22activation_fn: str = 'gelu-approximate'norm_elementwise_affine: bool = Falsenorm_eps: float = 1e-06caption_channels: int = 4096interpolation_scale_h: float = 2.0interpolation_scale_w: float = 2.0interpolation_scale_t: float = 2.2 )

forward

< source >

( hidden_states: Tensorencoder_hidden_states: Tensortimestep: LongTensorattention_mask: typing.Optional[torch.Tensor] = Noneencoder_attention_mask: typing.Optional[torch.Tensor] = Noneimage_rotary_emb: tuple[torch.Tensor, torch.Tensor] | None = Nonereturn_dict: bool = True )

Parámetros

hidden_states (torch.Tensor de forma (batch_size, num_channels, num_frames, height, width)) — Entrada hidden_states.
encoder_hidden_states (torch.Tensor de forma (batch_size, sequence_len, embed_dims)) — Incrustaciones condicionales (incrustaciones calculadas a partir de las condiciones de entrada, como los prompts) que se deben utilizar.
timestep (torch.LongTensor) — Se utiliza para indicar el paso de desruido.
attention_mask (torch.Tensor, opcional) — Máscara de autoatención aplicada a hidden_states.
encoder_attention_mask (torch.Tensor, opcional) — Máscara de atención cruzada aplicada a encoder_hidden_states.
image_rotary_emb (tuple de torch.Tensor, opcional) — Incrustaciones posicionales rotatorias precalculadas.
return_dict (bool, opcional, valor predeterminado True) — Indica si se debe devolver un ~models.transformer_2d.Transformer2DModelOutput en lugar de una tupla simple.

El método forward de AllegroTransformer3DModel.

Transformer2DModelOutput

clase diffusers.models.modeling_outputs.Transformer2DModelOutput

< source >

( sample: torch.Tensor )

Parámetros

sample (torch.Tensor de forma (batch_size, num_channels, height, width) o (batch size, num_vector_embeds - 1, num_latent_pixels) si Transformer2DModel es discreto) — La salida de estados ocultos condicionada a la entrada encoder_hidden_states. Si es discreto, devuelve distribuciones de probabilidad para los píxeles latentes sin ruido.

La salida de Transformer2DModel.

Update on GitHub

←AceStepTransformer1DModel AnyFlowFARTransformer3DModel→