#
# 根据 Apache 许可证 2.0 版本（“许可证”）授权
# 您只能在遵守许可证的情况下使用本文件。
# 您可以在以下位置获取许可证副本
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# 除非适用法律要求或双方书面同意，软件
# 根据许可证分发，本软件按“原样”
# 不附带任何明示或暗示的保证或条件。
# 请参阅许可证中关于权限的具体语言
# 许可证下的限制。-->

  
    
      
    
  

# Helios

来自北京大学、字节跳动等机构的 [Helios：真正的实时长视频生成模型](https://huggingface.co/papers/2603.04379)，作者为 Shenghai Yuan、Yuanyang Yin、Zongjian Li、Xinwei Huang、Xiao Yang、Li Yuan。

*  我们推出了 Helios，这是首个在单张 NVIDIA H100 GPU 上以 17 FPS 运行、支持分钟级生成且质量与强大基线相当的 14B 视频生成模型。我们在三个关键维度上取得了突破：(1) 在长视频生成中具备抗漂移鲁棒性，无需使用自强制（self-forcing）、误差库（error banks）或关键帧采样等常见的抗漂移启发式方法；(2) 实现实时生成，无需使用 KV-cache、因果掩码（causal masking）或稀疏注意力（sparse attention）等标准加速技术；以及 (3) 训练过程无需并行或分片框架，在将最多四个 14B 模型装入 80 GB GPU 显存的同时，实现了图像扩散模型规模的批量大小。具体而言，Helios 是一个 14B 自回归扩散模型，采用统一的输入表示，原生支持 T2V、I2V 和 V2V 任务。为缓解长视频生成中的漂移问题，我们刻画了其典型失效模式，并提出了简单而有效的训练策略，在训练过程中显式模拟漂移，同时从根源上消除重复运动。在效率方面，我们对历史上下文和噪声上下文进行了大幅压缩，并减少了采样步数，使得计算成本与 1.3B 视频生成模型相当甚至更低。此外，我们引入了基础设施层面的优化，在加速推理和训练的同时降低了内存消耗。大量实验表明，Helios 在短视频和长视频生成任务上均持续优于以往方法。所有代码和模型均可在 [this https URL](https://pku-yuangroup.github.io/Helios-Page) 获取。

以下 Helios 模型在 Diffusers 中受支持：

- [Helios-Base](https://huggingface.co/BestWishYsh/Helios-Base)：最佳质量，采用 v-prediction、标准 CFG 以及自定义的 HeliosScheduler。
- [Helios-Mid](https://huggingface.co/BestWishYsh/Helios-Mid)：中等权重，采用 v-prediction、CFG-Zero* 以及自定义的 HeliosScheduler。
- [Helios-Distilled](https://huggingface.co/BestWishYsh/Helios-Distilled)：效率最佳，采用 x0-prediction 和自定义的 HeliosDMDScheduler。

> [!TIP]
> 点击右侧边栏中的 Helios 模型，以查看更多视频生成示例。

### 优化内存和推理速度

下面的示例演示了如何生成针对内存或推理速度进行优化的文本到视频。

有关各种内存节省技术的详细信息，请参阅 [减少内存使用](../../optimization/memory) 指南。

下方的 Helios 模型需要约 6GB 的显存。

```py
import torch
from diffusers import AutoModel, HeliosPipeline
from diffusers.hooks.group_offloading import apply_group_offloading
from diffusers.utils import export_to_video

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32)

# group-offloading
pipeline = HeliosPipeline.from_pretrained(
    "BestWishYsh/Helios-Base",
    vae=vae,
    dtype=torch.bfloat16
)
pipeline.enable_group_offload(
    onload_device=torch.device("cuda"),
    offload_device=torch.device("cpu"),
    offload_type="leaf_level",
    use_stream=True,
    record_stream=True,
)

prompt = """
A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue 
and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with 
a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, 
allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades 
of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and 
the vivid colors of its surroundings. A close-up shot with dynamic movement.
"""
negative_prompt = """
Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
"""

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=99,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_base_t2v_output.mp4", fps=24)
```

[编译](../../optimization/fp16#torchcompile)在首次执行时较慢，但后续对 pipeline 的调用会更快。[注意力后端](../../optimization/attention_backends)（如 FlashAttention 和 SageAttention）可以通过优化注意力机制的计算来显著提升速度。[上下文并行](../../training/distributed_inference#context-parallelism)将输入序列拆分到多个设备上，以并行处理长上下文，从而降低内存压力和延迟。[缓存](../../optimization/cache)也可以通过存储和复用中间输出来加速推理。

```py
import torch
from diffusers import AutoModel, HeliosPipeline
from diffusers.utils import export_to_video

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32)

pipeline = HeliosPipeline.from_pretrained(
    "BestWishYsh/Helios-Base",
    vae=vae,
    dtype=torch.bfloat16
)
pipeline.to("cuda")  # or "mps", "xpu", "cpu"

# attention backend
# pipeline.transformer.set_attention_backend("flash")
pipeline.transformer.set_attention_backend("_flash_3_hub") # For Hopper GPUs

# torch.compile
torch.backends.cudnn.benchmark = True
pipeline.text_encoder.compile(mode="max-autotune-no-cudagraphs", dynamic=False)
pipeline.vae.compile(mode="max-autotune-no-cudagraphs", dynamic=False)
pipeline.transformer.compile(mode="max-autotune-no-cudagraphs", dynamic=False)

prompt = """
A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue 
and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with 
a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, 
allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades 
of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and 
the vivid colors of its surroundings. A close-up shot with dynamic movement.
"""
negative_prompt = """
Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
"""

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=99,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_base_t2v_output.mp4", fps=24)
```

### 使用 Helios-Base 进行生成

下面的示例演示了如何使用 Helios-Base 根据文本、图像或视频生成视频。

```python
import torch
from diffusers import AutoModel, HeliosPipeline
from diffusers.utils import export_to_video, load_video, load_image

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Base", subfolder="vae", dtype=torch.float32)

pipeline = HeliosPipeline.from_pretrained(
    "BestWishYsh/Helios-Base",
    vae=vae,
    dtype=torch.bfloat16
)
pipeline.to("cuda")  # or "mps", "xpu", "cpu"

negative_prompt = """
Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
"""

# For Text-to-Video
prompt = """
A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue 
and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with 
a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, 
allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades 
of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and 
the vivid colors of its surroundings. A close-up shot with dynamic movement.
"""

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=99,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_base_t2v_output.mp4", fps=24)

# For Image-to-Video
prompt = """
A towering emerald wave surges forward, its crest curling with raw power and energy. Sunlight glints off the translucent water, 
illuminating the intricate textures and deep green hues within the wave’s body. A thick spray erupts from the breaking crest, 
casting a misty veil that dances above the churning surface. As the perspective widens, the immense scale of the wave becomes 
apparent, revealing the restless expanse of the ocean stretching beyond. The scene captures the ocean’s untamed beauty and 
relentless force, with every droplet and ripple shimmering in the light. The dynamic motion and vivid colors evoke both awe and 
respect for nature’s might.
"""
image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/wave.jpg"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    image=load_image(image_path).resize((640, 384)),
    num_frames=99,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_base_i2v_output.mp4", fps=24)

# For Video-to-Video
prompt = """
A bright yellow Lamborghini Huracn Tecnica speeds along a curving mountain road, surrounded by lush green trees 
under a partly cloudy sky. The car's sleek design and vibrant color stand out against the natural backdrop, 
emphasizing its dynamic movement. The road curves gently, with a guardrail visible on one side, adding depth to 
the scene. The motion blur captures the sense of speed and energy, creating a thrilling and exhilarating atmosphere. 
A front-facing shot from a slightly elevated angle, highlighting the car's aggressive stance and the surrounding greenery.
"""
video_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/car.mp4"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    video=load_video(video_path),
    num_frames=99,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_base_v2v_output.mp4", fps=24)
```

### 使用 Helios-Mid 进行生成

Helios-Mid 的以下示例展示了如何基于文本、图像或视频生成视频。

```python
import torch
from diffusers import AutoModel, HeliosPyramidPipeline
from diffusers.utils import export_to_video, load_video, load_image

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Mid", subfolder="vae", dtype=torch.float32)

pipeline = HeliosPyramidPipeline.from_pretrained(
    "BestWishYsh/Helios-Mid",
    vae=vae,
    dtype=torch.bfloat16
)
pipeline.to("cuda")  # or "mps", "xpu", "cpu"

negative_prompt = """
Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
"""

# For Text-to-Video
prompt = """
A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue 
and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with 
a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, 
allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades 
of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and 
the vivid colors of its surroundings. A close-up shot with dynamic movement.
"""

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=99,
    pyramid_num_inference_steps_list=[20, 20, 20],
    guidance_scale=5.0,
    use_zero_init=True,
    zero_steps=1,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_pyramid_t2v_output.mp4", fps=24)

# For Image-to-Video
prompt = """
A towering emerald wave surges forward, its crest curling with raw power and energy. Sunlight glints off the translucent water, 
illuminating the intricate textures and deep green hues within the wave’s body. A thick spray erupts from the breaking crest, 
casting a misty veil that dances above the churning surface. As the perspective widens, the immense scale of the wave becomes 
apparent, revealing the restless expanse of the ocean stretching beyond. The scene captures the ocean’s untamed beauty and 
relentless force, with every droplet and ripple shimmering in the light. The dynamic motion and vivid colors evoke both awe and 
respect for nature’s might.
"""
image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/wave.jpg"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    image=load_image(image_path).resize((640, 384)),
    num_frames=99,
    pyramid_num_inference_steps_list=[20, 20, 20],
    guidance_scale=5.0,
    use_zero_init=True,
    zero_steps=1,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_pyramid_i2v_output.mp4", fps=24)

# For Video-to-Video
prompt = """
A bright yellow Lamborghini Huracn Tecnica speeds along a curving mountain road, surrounded by lush green trees 
under a partly cloudy sky. The car's sleek design and vibrant color stand out against the natural backdrop, 
emphasizing its dynamic movement. The road curves gently, with a guardrail visible on one side, adding depth to 
the scene. The motion blur captures the sense of speed and energy, creating a thrilling and exhilarating atmosphere. 
A front-facing shot from a slightly elevated angle, highlighting the car's aggressive stance and the surrounding greenery.
"""
video_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/car.mp4"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    video=load_video(video_path),
    num_frames=99,
    pyramid_num_inference_steps_list=[20, 20, 20],
    guidance_scale=5.0,
    use_zero_init=True,
    zero_steps=1,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_pyramid_v2v_output.mp4", fps=24)
```

### 使用 Helios-Distilled 进行生成

下面的示例演示了如何使用 Helios-Distilled 根据文本、图像或视频生成视频。

```python
import torch
from diffusers import AutoModel, HeliosPyramidPipeline
from diffusers.utils import export_to_video, load_video, load_image

vae = AutoModel.from_pretrained("BestWishYsh/Helios-Distilled", subfolder="vae", dtype=torch.float32)

pipeline = HeliosPyramidPipeline.from_pretrained(
    "BestWishYsh/Helios-Distilled",
    vae=vae,
    dtype=torch.bfloat16
)
pipeline.to("cuda")  # or "mps", "xpu", "cpu"

negative_prompt = """
Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality,
low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured,
misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards
"""

# For Text-to-Video
prompt = """
A vibrant tropical fish swimming gracefully among colorful coral reefs in a clear, turquoise ocean. The fish has bright blue 
and yellow scales with a small, distinctive orange spot on its side, its fins moving fluidly. The coral reefs are alive with 
a variety of marine life, including small schools of colorful fish and sea turtles gliding by. The water is crystal clear, 
allowing for a view of the sandy ocean floor below. The reef itself is adorned with a mix of hard and soft corals in shades 
of red, orange, and green. The photo captures the fish from a slightly elevated angle, emphasizing its lively movements and 
the vivid colors of its surroundings. A close-up shot with dynamic movement.
"""

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_frames=240,
    pyramid_num_inference_steps_list=[2, 2, 2],
    guidance_scale=1.0,
    is_amplify_first_chunk=True,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_distilled_t2v_output.mp4", fps=24)

# For Image-to-Video
prompt = """
A towering emerald wave surges forward, its crest curling with raw power and energy. Sunlight glints off the translucent water, 
illuminating the intricate textures and deep green hues within the wave’s body. A thick spray erupts from the breaking crest, 
casting a misty veil that dances above the churning surface. As the perspective widens, the immense scale of the wave becomes 
apparent, revealing the restless expanse of the ocean stretching beyond. The scene captures the ocean’s untamed beauty and 
relentless force, with every droplet and ripple shimmering in the light. The dynamic motion and vivid colors evoke both awe and 
respect for nature’s might.
"""
image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/wave.jpg"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    image=load_image(image_path).resize((640, 384)),
    num_frames=240,
    pyramid_num_inference_steps_list=[2, 2, 2],
    guidance_scale=1.0,
    is_amplify_first_chunk=True,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_distilled_i2v_output.mp4", fps=24)

# For Video-to-Video
prompt = """
A bright yellow Lamborghini Huracn Tecnica speeds along a curving mountain road, surrounded by lush green trees 
under a partly cloudy sky. The car's sleek design and vibrant color stand out against the natural backdrop, 
emphasizing its dynamic movement. The road curves gently, with a guardrail visible on one side, adding depth to 
the scene. The motion blur captures the sense of speed and energy, creating a thrilling and exhilarating atmosphere. 
A front-facing shot from a slightly elevated angle, highlighting the car's aggressive stance and the surrounding greenery.
"""
video_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/helios/car.mp4"

output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    video=load_video(video_path),
    num_frames=240,
    pyramid_num_inference_steps_list=[2, 2, 2],
    guidance_scale=1.0,
    is_amplify_first_chunk=True,
    generator=torch.Generator("cuda").manual_seed(42),
).frames[0]
export_to_video(output, "helios_distilled_v2v_output.mp4", fps=24)
```

## 文本到视频展示

  
    提示词生成视频
  
  
    一位维京战士驾驶着一辆载满乘客的现代城市公交车。这位维京人留着向后扎起的金色长发，蓄着胡须，头戴毛皮镶边的头盔，身披铠甲。他身穿传统的束腰外衣和长裤，但同时也系着安全带，全神贯注地穿梭于繁忙的街道。公交车内部十分典型，一排排座位上坐满了正在度过日常生活的各色乘客。外部镜头展示了熙熙攘攘的城市环境，包括高楼大厦和车流。中景镜头聚焦于手握方向盘的维京人，并偶尔穿插他坚毅表情的特写。
    
    
      
        
      
    
  
  
    一段纪录片风格的自然摄影画面，镜头从一辆向左移动的车上拍摄，捕捉到一只螃蟹迅速钻进洞穴的瞬间。这只螃蟹有着坚硬、呈绿褐色的甲壳和长长的螯，以坚定的速度在沙地上移动。它的身体微微拱起，钻入沙中，身后留下一条细小的痕迹。背景是一片浅滩，散落着岩石和贝壳，地平线处可见海岸线柔和的弧度。照片具有自然、真实的质感，突出了螃蟹的自然动态和沙子的纹理。这是一个从略微俯视角度拍摄的近景镜头。
    
    
      
        
      
    
  

## 图像转视频展示

  
    图像提示生成的视频
  
  
    
    一辆时尚的红色起亚汽车在阴云密布的天空下沿着乡间公路疾驰，周围延伸至远方的田野和树木因动态模糊而凸显了车辆的现代设计与动感姿态。车身光亮的漆面反射着阴沉的天空，突显出其流线型轮廓与运动姿态。车牌上写着“KIA 626”，车辆的前大灯已开启，进一步增强了运动感与活力。道路平缓弯曲，车辆略微偏离中心位置，营造出向前冲的动势。动态的前侧四分之三视角捕捉到了车辆在连绵起伏的丘陵与零星树木构成的宁静背景下的强劲气场。
    
    
      
        
      
    
  
  
    
    特写镜头捕捉到一只毛茸茸的橙色猫咪，它有着引人注目的绿色眼睛和白色胡须，正专注地凝视着镜头。猫咪的毛发柔软且打理得十分整洁，呈现出温暖的橙色与奶油色的混合色调。它那双大而富有表现力的眼睛是鲜亮的绿色，流露出好奇与警觉。猫咪的鼻子小巧粉嫩，嘴巴微微张开，隐约露出粉色的舌头。背景柔和地虚化，暗示着一个色调中性的舒适室内环境。照片采用了浅景深，焦点清晰地落在猫咪的面部，而背景则保持模糊。这是一个从略微俯视角度拍摄的特写镜头。
    
    
      
        
      
    
  

## 交互式视频展示

  
    提示词生成视频
  
  
    提示词可以在这里找到
    
      
        
      
    
  
  
    提示词可以在这里找到
    
      
        
      
    
  

## 资源

通过以下资源进一步了解 Helios。
- 观看 [video1](https://www.youtube.com/watch?v=vd_AgHtOUFQ) 和 [video2](https://www.youtube.com/watch?v=1GeIU2Dn7UY)，了解 Helios 主要功能的演示。
- 有关更多详细信息，请参阅研究论文 [Helios: Real Real-Time Long Video Generation Model](https://huggingface.co/papers/2603.04379)。

## HeliosPipeline[[diffusers.HeliosPipeline]]

#### diffusers.HeliosPipeline[[diffusers.HeliosPipeline]]

```python
diffusers.HeliosPipeline(tokenizer: AutoTokenizer, text_encoder: UMT5EncoderModel, vae: AutoencoderKLWan, scheduler: HeliosScheduler, transformer: HeliosTransformer3DModel)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios.py#L108)

**参数：**

tokenizer (`T5Tokenizer`)：来自 [T5](https://huggingface.co/docs/transformers/en/model_doc/t5#transformers.T5Tokenizer) 的分词器，具体为 [google/umt5-xxl](https://huggingface.co/google/umt5-xxl) 变体。

text_encoder (`T5EncoderModel`)：即 [T5](https://huggingface.co/docs/transformers/en/model_doc/t5#transformers.T5EncoderModel)，具体为 [google/umt5-xxl](https://huggingface.co/google/umt5-xxl) 变体。

transformer ([HeliosTransformer3DModel](/docs/diffusers/main/zh/api/models/helios_transformer3d#diffusers.HeliosTransformer3DModel))：用于对输入潜变量进行去噪的条件 Transformer。

scheduler ([HeliosScheduler](/docs/diffusers/main/zh/api/schedulers/helios#diffusers.HeliosScheduler))：用于与 `transformer` 配合，对编码后的图像潜变量进行去噪的调度器。

vae ([AutoencoderKLWan](/docs/diffusers/main/zh/api/models/autoencoder_kl_wan#diffusers.AutoencoderKLWan))：变分自编码器（VAE）模型，用于将视频编码为潜在表示，以及从潜在表示解码为视频。

使用 Helios 进行文本生成视频 / 图像生成视频 / 视频生成视频的 Pipeline。

该模型继承自 [DiffusionPipeline](/docs/diffusers/main/zh/api/pipelines/overview#diffusers.DiffusionPipeline)。有关为所有流水线实现的通用方法（下载、保存、在特定设备上运行等），请参阅父类文档。

#### __调用__[[diffusers.HeliosPipeline.__call__]]

```python
__call__(prompt: str | list[str] = None, negative_prompt: str | list[str] = None, height: int = 384, width: int = 640, num_frames: int = 132, num_inference_steps: int = 50, sigmas: list = None, guidance_scale: float = 5.0, num_videos_per_prompt: int | None = 1, generator: typing.Union[torch.Generator, list[torch.Generator], NoneType] = None, latents: typing.Optional[torch.Tensor] = None, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, output_type: str | None = 'np', return_dict: bool = True, attention_kwargs: dict[str, typing.Any] | None = None, callback_on_step_end: typing.Union[typing.Callable[[int, int], NoneType], diffusers.callbacks.PipelineCallback, diffusers.callbacks.MultiPipelineCallbacks, NoneType] = None, callback_on_step_end_tensor_inputs: list = ['latents'], max_sequence_length: int = 512, image: typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor], NoneType] = None, image_latents: typing.Optional[torch.Tensor] = None, fake_image_latents: typing.Optional[torch.Tensor] = None, add_noise_to_image_latents: bool = True, image_noise_sigma_min: float = 0.111, image_noise_sigma_max: float = 0.135, video: typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor], NoneType] = None, video_latents: typing.Optional[torch.Tensor] = None, add_noise_to_video_latents: bool = True, video_noise_sigma_min: float = 0.111, video_noise_sigma_max: float = 0.135, history_sizes: list = [16, 2, 1], num_latent_frames_per_chunk: int = 9, keep_first_frame: bool = True, is_skip_first_chunk: bool = False)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios.py#L445)

**参数：**

prompt (`str` 或 `list[str]`，*可选*)：用于引导图像生成的提示词或提示词列表。如果未定义，请改为传入 `prompt_embeds`。

negative_prompt (`str` 或 `list[str]`，*可选*) : 图像生成过程中需要避免的提示词。如果未定义，请改为传入 `negative_prompt_embeds`。在未使用引导（`guidance_scale` < `1`）时将被忽略。

height (`int`，默认为 `384`)：生成图像的高度（以像素为单位）。

width (`int`，默认为 `640`)：生成图像的宽度（以像素为单位）。

num_frames (`int`，默认为 `132`)：生成视频的帧数。

num_inference_steps (`int`，默认为 `50`)：去噪步数。更多的去噪步数通常能带来更高质量的图像，但代价是推理速度变慢。

sigmas (`list[float]`, *可选*) : 用于去噪过程的自定义 sigma 值。如果未定义，则使用调度器的默认调度。

guidance_scale (`float`，默认为 `5.0`) : 如 [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598) 中定义的引导尺度。`guidance_scale` 定义为 [Imagen Paper](https://huggingface.co/papers/2205.11487) 中公式 2 的 `w`。通过设置 `guidance_scale > 1` 来启用引导尺度。较高的引导尺度会促使生成与文本 `prompt` 紧密关联的图像，通常以牺牲图像质量为代价。

num_videos_per_prompt (`int`，*可选*，默认为 1)：每个提示词生成的图像数量。

generator (`torch.Generator` 或 `list[torch.Generator]`，*可选*)：用于使生成过程确定性的 [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html)。

latents (`torch.Tensor`, *可选*) : 从高斯分布中采样的预生成噪声潜变量，用作图像生成的输入。可用于以不同的提示词微调同一生成过程。如果未提供，则使用提供的随机`generator`进行采样来生成潜变量张量。

prompt_embeds (`torch.Tensor`, *可选*) : 预生成的文本嵌入。可用于轻松调整文本输入（提示词权重）。如果未提供，则从 `prompt` 输入参数生成文本嵌入。

negative_prompt_embeds (`torch.Tensor`，*可选*)：预生成的负向文本嵌入。如果未提供，则从 `negative_prompt` 生成。

output_type (`str`，*可选*，默认为 `"np"`)：生成图像的输出格式。在 `PIL.Image` 和 `np.array` 之间选择。

return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `HeliosPipelineOutput` 而非普通元组。

attention_kwargs (`dict`, *可选*) : 一个 kwargs 字典，如果指定，将作为参数传递给 [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py) 中 `self.processor` 下定义的 `AttentionProcessor`。

callback_on_step_end (`Callable`、`PipelineCallback`、`MultiPipelineCallbacks`、*可选*)：一个函数，或 `PipelineCallback` 或 `MultiPipelineCallbacks` 的子类，在推理过程中每个去噪步骤结束时被调用，其参数为：`callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`。`callback_kwargs` 将包含由 `callback_on_step_end_tensor_inputs` 指定的所有张量列表。

callback_on_step_end_tensor_inputs (`list`，*可选*)：`callback_on_step_end` 函数的张量输入列表。列表中指定的张量将作为 `callback_kwargs` 参数传递。您只能包含流水线类的 `._callback_tensor_inputs` 属性中列出的变量。

max_sequence_length (`int`，默认为 `512`)：文本编码器的最大序列长度。如果提示词超过此长度，将被截断；如果提示词较短，则会被填充至该长度。

image (`PipelineImageInput`，*可选*)：用于图像到视频条件控制的输入图像。

image_latents (`torch.Tensor`, *可选*) : 用于替代 `image` 的预编码图像潜变量。

fake_image_latents (`torch.Tensor`, *可选*) : 在条件化过程中使用的可选伪图像潜变量。

add_noise_to_image_latents (`bool`，*可选*，默认为 `True`)：是否在对图像潜变量进行去噪之前添加噪声。

image_noise_sigma_min (`float`，*可选*，默认值为 `0.111`)：添加到图像潜变量的噪声的最小 sigma 值。

image_noise_sigma_max (`float`，*可选*，默认值为 `0.135`)：添加到图像潜变量的噪声的最大 sigma 值。

video (`PipelineImageInput`，*可选*)：用于视频到视频条件控制的输入视频。

video_latents (`torch.Tensor`，*可选*)：用于替代 `video` 的预编码视频潜变量。

add_noise_to_video_latents (`bool`，*可选*，默认为 `True`)：是否在对视频潜变量进行去噪之前添加噪声。

video_noise_sigma_min (`float`, *可选*, 默认值为 `0.111`) : 添加到视频潜变量的噪声的最小 sigma 值。

video_noise_sigma_max (`float`，*可选*，默认值为 `0.135`)：添加到视频潜变量的噪声的最大 sigma 值。

history_sizes (`list`，*可选*，默认为 `[16, 2, 1]`)：用于自回归分块生成的历史窗口大小。

num_latent_frames_per_chunk (`int`，*可选*，默认为 `9`)：自回归生成过程中，每个块生成的潜在帧数量。

keep_first_frame (`bool`，*可选*，默认为 `True`)：是否跨块保留第一帧。

is_skip_first_chunk (`bool`，*可选*，默认为 `False`)：是否跳过第一个块的生成。

**返回：** `~HeliosPipelineOutput` 或 `tuple`

如果 `return_dict` 为 `True`，则返回 `HeliosPipelineOutput`；否则返回一个 `tuple`，其中第一个元素是包含生成图像的列表，第二个元素是 `bool` 列表，用于指示对应的生成图像是否包含“不适合工作场所”（nsfw）内容。

用于生成的 pipeline 的调用函数。

示例：
```python
>>> import torch
>>> from diffusers.utils import export_to_video
>>> from diffusers import AutoencoderKLWan, HeliosPipeline

>>> # Available models: BestWishYsh/Helios-Base, BestWishYsh/Helios-Mid, BestWishYsh/Helios-Distilled
>>> model_id = "BestWishYsh/Helios-Base"
>>> vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
>>> pipe = HeliosPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
>>> pipe.to("cuda")

>>> prompt = "A cat and a dog baking a cake together in a kitchen. The cat is carefully measuring flour, while the dog is stirring the batter with a wooden spoon. The kitchen is cozy, with sunlight streaming through the window."
>>> negative_prompt = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"

>>> output = pipe(
...     prompt=prompt,
...     negative_prompt=negative_prompt,
...     height=384,
...     width=640,
...     num_frames=132,
...     guidance_scale=5.0,
... ).frames[0]
>>> export_to_video(output, "output.mp4", fps=24)
```

#### encode_prompt[[diffusers.HeliosPipeline.encode_prompt]]

```python
encode_prompt(prompt: str | list[str], negative_prompt: str | list[str] | None = None, do_classifier_free_guidance: bool = True, num_videos_per_prompt: int = 1, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, max_sequence_length: int = 226, device: typing.Optional[torch.device] = None, dtype: typing.Optional[torch.dtype] = None)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios.py#L196)

**参数：**

prompt (`str` 或 `list[str]`，*可选*)：要编码的提示

negative_prompt (`str` 或 `list[str]`，*可选*)：用于引导图像生成的负面提示词。如果未定义，则必须传入 `negative_prompt_embeds`。当不使用引导时（即 `guidance_scale` 小于 `1` 时），此参数将被忽略。

do_classifier_free_guidance (`bool`，*可选*，默认为 `True`)：是否使用无分类器引导。

num_videos_per_prompt (`int`，*可选*，默认为 1)：每个提示词应生成的视频数量。用于放置结果嵌入的 torch 设备

prompt_embeds (`torch.Tensor`, *可选*) : 预生成的文本嵌入。可用于轻松调整文本输入，*例如* 提示词加权。如果未提供，文本嵌入将从 `prompt` 输入参数生成。

negative_prompt_embeds (`torch.Tensor`, *可选*) : 预生成的负向文本嵌入。可用于轻松调整文本输入，*例如* 提示词权重。如果未提供，negative_prompt_embeds 将从 `negative_prompt` 输入参数生成。

device : (`torch.device`, *可选*): torch 设备

dtype : (`torch.dtype`, *可选*): torch 数据类型

将提示词编码为文本编码器的隐藏状态。

## HeliosPyramidPipeline[[diffusers.HeliosPyramidPipeline]]

#### diffusers.HeliosPyramidPipeline[[diffusers.HeliosPyramidPipeline]]

```python
diffusers.HeliosPyramidPipeline(tokenizer: AutoTokenizer, text_encoder: UMT5EncoderModel, vae: AutoencoderKLWan, scheduler: diffusers.schedulers.scheduling_helios.HeliosScheduler | diffusers.schedulers.scheduling_helios_dmd.HeliosDMDScheduler, transformer: HeliosTransformer3DModel, is_cfg_zero_star: bool = False, is_distilled: bool = False)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios_pyramid.py#L121)

**参数：**

tokenizer (`T5Tokenizer`)：来自 [T5](https://huggingface.co/docs/transformers/en/model_doc/t5#transformers.T5Tokenizer) 的分词器，具体为 [google/umt5-xxl](https://huggingface.co/google/umt5-xxl) 变体。

text_encoder (`T5EncoderModel`)：即 [T5](https://huggingface.co/docs/transformers/en/model_doc/t5#transformers.T5EncoderModel)，具体为 [google/umt5-xxl](https://huggingface.co/google/umt5-xxl) 变体。

transformer ([HeliosTransformer3DModel](/docs/diffusers/main/zh/api/models/helios_transformer3d#diffusers.HeliosTransformer3DModel))：用于对输入潜变量进行去噪的条件 Transformer。

scheduler ([`HeliosScheduler`, `HeliosDMDScheduler`]) : 用于与 `transformer` 配合，对编码后的图像潜变量进行去噪的调度器。

vae ([AutoencoderKLWan](/docs/diffusers/main/zh/api/models/autoencoder_kl_wan#diffusers.AutoencoderKLWan))：变分自编码器（VAE）模型，用于将视频编码为潜在表示，以及从潜在表示解码为视频。

使用 Helios 进行文本生成视频 / 图像生成视频 / 视频生成视频的 Pipeline。

该模型继承自 [DiffusionPipeline](/docs/diffusers/main/zh/api/pipelines/overview#diffusers.DiffusionPipeline)。有关为所有流水线实现的通用方法（下载、保存、在特定设备上运行等），请参阅父类文档。

#### __调用__[[diffusers.HeliosPyramidPipeline.__call__]]

```python
__call__(prompt: str | list[str] = None, negative_prompt: str | list[str] = None, height: int = 384, width: int = 640, num_frames: int = 132, sigmas: list = None, guidance_scale: float = 5.0, num_videos_per_prompt: int | None = 1, generator: typing.Union[torch.Generator, list[torch.Generator], NoneType] = None, latents: typing.Optional[torch.Tensor] = None, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, output_type: str | None = 'np', return_dict: bool = True, attention_kwargs: dict[str, typing.Any] | None = None, callback_on_step_end: typing.Union[typing.Callable[[int, int], NoneType], diffusers.callbacks.PipelineCallback, diffusers.callbacks.MultiPipelineCallbacks, NoneType] = None, callback_on_step_end_tensor_inputs: list = ['latents'], max_sequence_length: int = 512, image: typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor], NoneType] = None, image_latents: typing.Optional[torch.Tensor] = None, fake_image_latents: typing.Optional[torch.Tensor] = None, add_noise_to_image_latents: bool = True, image_noise_sigma_min: float = 0.111, image_noise_sigma_max: float = 0.135, video: typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor], NoneType] = None, video_latents: typing.Optional[torch.Tensor] = None, add_noise_to_video_latents: bool = True, video_noise_sigma_min: float = 0.111, video_noise_sigma_max: float = 0.135, history_sizes: list = [16, 2, 1], num_latent_frames_per_chunk: int = 9, keep_first_frame: bool = True, is_skip_first_chunk: bool = False, pyramid_num_inference_steps_list: list = [10, 10, 10], use_zero_init: bool | None = True, zero_steps: int | None = 1, is_amplify_first_chunk: bool = False)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios_pyramid.py#L508)

**参数：**

prompt (`str` 或 `list[str]`，*可选*)：用于引导图像生成的提示词或提示词列表。如果未定义，请改为传入 `prompt_embeds`。

negative_prompt (`str` 或 `list[str]`，*可选*) : 图像生成过程中需要避免的提示词。如果未定义，请改为传入 `negative_prompt_embeds`。在未使用引导（`guidance_scale` < `1`）时将被忽略。

height (`int`，默认为 `384`)：生成图像的高度（以像素为单位）。

width (`int`，默认为 `640`)：生成图像的宽度（以像素为单位）。

num_frames (`int`，默认为 `132`)：生成视频的帧数。

sigmas (`list[float]`, *可选*) : 用于去噪过程的自定义 sigma 值。如果未定义，则使用调度器的默认调度。

guidance_scale (`float`，默认为 `5.0`) : 如 [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598) 中定义的引导尺度。`guidance_scale` 定义为 [Imagen Paper](https://huggingface.co/papers/2205.11487) 中公式 2 的 `w`。通过设置 `guidance_scale > 1` 来启用引导尺度。较高的引导尺度会促使生成与文本 `prompt` 紧密关联的图像，通常以牺牲图像质量为代价。

num_videos_per_prompt (`int`，*可选*，默认为 1)：每个提示词生成的图像数量。

generator (`torch.Generator` 或 `list[torch.Generator]`，*可选*)：用于使生成过程确定性的 [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html)。

latents (`torch.Tensor`, *可选*) : 从高斯分布中采样的预生成噪声潜变量，用作图像生成的输入。可用于以不同的提示词微调同一生成过程。如果未提供，则使用提供的随机`generator`进行采样来生成潜变量张量。

prompt_embeds (`torch.Tensor`, *可选*) : 预生成的文本嵌入。可用于轻松调整文本输入（提示词权重）。如果未提供，则从 `prompt` 输入参数生成文本嵌入。

negative_prompt_embeds (`torch.Tensor`，*可选*)：预生成的负向文本嵌入。如果未提供，则从 `negative_prompt` 生成。

output_type (`str`，*可选*，默认为 `"np"`)：生成图像的输出格式。在 `PIL.Image` 和 `np.array` 之间选择。

return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `HeliosPipelineOutput` 而非普通元组。

attention_kwargs (`dict`, *可选*) : 一个 kwargs 字典，如果指定，将作为参数传递给 [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py) 中 `self.processor` 下定义的 `AttentionProcessor`。

callback_on_step_end (`Callable`、`PipelineCallback`、`MultiPipelineCallbacks`、*可选*)：一个函数，或 `PipelineCallback` 或 `MultiPipelineCallbacks` 的子类，在推理过程中每个去噪步骤结束时被调用，其参数为：`callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`。`callback_kwargs` 将包含由 `callback_on_step_end_tensor_inputs` 指定的所有张量列表。

callback_on_step_end_tensor_inputs (`list`，*可选*)：`callback_on_step_end` 函数的张量输入列表。列表中指定的张量将作为 `callback_kwargs` 参数传递。您只能包含流水线类的 `._callback_tensor_inputs` 属性中列出的变量。

max_sequence_length (`int`，默认为 `512`)：文本编码器的最大序列长度。如果提示词超过此长度，将被截断；如果提示词较短，则会被填充至该长度。

image (`PipelineImageInput`，*可选*)：用于图像到视频条件控制的输入图像。

image_latents (`torch.Tensor`, *可选*) : 用于替代 `image` 的预编码图像潜变量。

fake_image_latents (`torch.Tensor`, *可选*) : 在条件化过程中使用的可选伪图像潜变量。

add_noise_to_image_latents (`bool`，*可选*，默认为 `True`)：是否在对图像潜变量进行去噪之前添加噪声。

image_noise_sigma_min (`float`，*可选*，默认值为 `0.111`)：添加到图像潜变量的噪声的最小 sigma 值。

image_noise_sigma_max (`float`，*可选*，默认值为 `0.135`)：添加到图像潜变量的噪声的最大 sigma 值。

video (`PipelineImageInput`，*可选*)：用于视频到视频条件控制的输入视频。

video_latents (`torch.Tensor`，*可选*)：用于替代 `video` 的预编码视频潜变量。

add_noise_to_video_latents (`bool`，*可选*，默认为 `True`)：是否在对视频潜变量进行去噪之前添加噪声。

video_noise_sigma_min (`float`, *可选*, 默认值为 `0.111`) : 添加到视频潜变量的噪声的最小 sigma 值。

video_noise_sigma_max (`float`，*可选*，默认值为 `0.135`)：添加到视频潜变量的噪声的最大 sigma 值。

history_sizes (`list`，*可选*，默认为 `[16, 2, 1]`)：用于自回归分块生成的历史窗口大小。

num_latent_frames_per_chunk (`int`，*可选*，默认为 `9`)：自回归生成过程中，每个块生成的潜在帧数量。

keep_first_frame (`bool`，*可选*，默认为 `True`)：是否跨块保留第一帧。

is_skip_first_chunk (`bool`，*可选*，默认为 `False`)：是否跳过第一个块的生成。

pyramid_num_inference_steps_list (`list`，*可选*，默认为 `[10, 10, 10]`)：在第二阶段生成过程中，每个金字塔阶段的推理步数。

use_zero_init (`bool`，*可选*，默认为 `True`)：是否在去噪开始时应用 CFG 零初始化。

zero_steps (`int`，*可选*，默认为 `1`)：使用 CFG zero-init 的初始步数。

is_amplify_first_chunk (`bool`，*可选*，默认为 `False`)：是否对第一个块（chunk）放大引导（与 DMD 相关）。

**返回：** `~HeliosPipelineOutput` 或 `tuple`

如果 `return_dict` 为 `True`，则返回 `HeliosPipelineOutput`；否则返回一个 `tuple`，其中第一个元素是包含生成图像的列表，第二个元素是 `bool` 列表，用于指示对应的生成图像是否包含“不适合工作场所”（nsfw）内容。

用于生成的 pipeline 的调用函数。

示例：
```python
>>> import torch
>>> from diffusers.utils import export_to_video
>>> from diffusers import AutoencoderKLWan, HeliosPyramidPipeline

>>> # Available models: BestWishYsh/Helios-Base, BestWishYsh/Helios-Mid, BestWishYsh/Helios-Distilled
>>> model_id = "BestWishYsh/Helios-Base"
>>> vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
>>> pipe = HeliosPyramidPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
>>> pipe.to("cuda")

>>> prompt = "A cat and a dog baking a cake together in a kitchen. The cat is carefully measuring flour, while the dog is stirring the batter with a wooden spoon. The kitchen is cozy, with sunlight streaming through the window."
>>> negative_prompt = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"

>>> output = pipe(
...     prompt=prompt,
...     negative_prompt=negative_prompt,
...     height=384,
...     width=640,
...     num_frames=132,
...     guidance_scale=5.0,
... ).frames[0]
>>> export_to_video(output, "output.mp4", fps=24)
```

#### encode_prompt[[diffusers.HeliosPyramidPipeline.encode_prompt]]

```python
encode_prompt(prompt: str | list[str], negative_prompt: str | list[str] | None = None, do_classifier_free_guidance: bool = True, num_videos_per_prompt: int = 1, prompt_embeds: typing.Optional[torch.Tensor] = None, negative_prompt_embeds: typing.Optional[torch.Tensor] = None, max_sequence_length: int = 226, device: typing.Optional[torch.device] = None, dtype: typing.Optional[torch.dtype] = None)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_helios_pyramid.py#L214)

**参数：**

prompt (`str` 或 `list[str]`，*可选*)：要编码的提示

negative_prompt (`str` 或 `list[str]`，*可选*)：用于引导图像生成的负面提示词。如果未定义，则必须传入 `negative_prompt_embeds`。当不使用引导时（即 `guidance_scale` 小于 `1` 时），此参数将被忽略。

do_classifier_free_guidance (`bool`，*可选*，默认为 `True`)：是否使用无分类器引导。

num_videos_per_prompt (`int`，*可选*，默认为 1)：每个提示词应生成的视频数量。用于放置结果嵌入的 torch 设备

prompt_embeds (`torch.Tensor`, *可选*) : 预生成的文本嵌入。可用于轻松调整文本输入，*例如* 提示词加权。如果未提供，文本嵌入将从 `prompt` 输入参数生成。

negative_prompt_embeds (`torch.Tensor`, *可选*) : 预生成的负向文本嵌入。可用于轻松调整文本输入，*例如* 提示词权重。如果未提供，negative_prompt_embeds 将从 `negative_prompt` 输入参数生成。

device : (`torch.device`, *可选*): torch 设备

dtype : (`torch.dtype`, *可选*): torch 数据类型

将提示词编码为文本编码器的隐藏状态。

## HeliosPipelineOutput[[diffusers.pipelines.helios.pipeline_output.HeliosPipelineOutput]]

#### diffusers.pipelines.helios.pipeline_output.HeliosPipelineOutput[[diffusers.pipelines.helios.pipeline_output.HeliosPipelineOutput]]

```python
diffusers.pipelines.helios.pipeline_output.HeliosPipelineOutput(frames: Tensor)
```

[源代码](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/helios/pipeline_output.py#L9)

**参数：**

frames (`torch.Tensor`、`np.ndarray` 或 List[List[PIL.Image.Image]]) : 视频输出列表 - 它可以是一个长度为 `batch_size,` 的嵌套列表，其中每个子列表包含长度为 `num_frames.` 的去噪 PIL 图像序列。它也可以是一个形状为 `(batch_size, num_frames, channels, height, width)` 的 NumPy 数组或 Torch 张量。

Helios 流水线的输出类。