A ModularPipeline for joint video + audio generation with Pika H3 Flex, the post-trained six-step MiniMax-H3: the t2va (text only) and fl2va (first and/or last keyframe) workflows against the transformer/ checkpoint partition with the fl2va adapter, and the ref2va (omni-reference) workflow against transformer_ref/ with the ref2va adapter.

The base components are MiniMax-H3’s, and the Pika H3 Flex release is a pair of rank-128 LoRA adapters on top of the two transformer partitions, so a pipeline is assembled in two steps: the components are loaded through a modular_model_index.json that names the base components and the two schedulers, and the adapter of the selected workflow is loaded with load_lora_weights().

import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("Pika-Labs/Pika-H3-Flex", workflow="fl2va")
pipe.load_components(dtype={"vae": torch.float32, "audio_vae": torch.float32, "default": torch.bfloat16})
pipe.load_lora_weights("Pika-Labs/Pika-H3-Flex", weight_name="fl2va/adapter.safetensors")

Like MiniMax-H3, Pika H3 Flex denoises one packed sequence that holds the text conditioning, the keyframe or reference conditioning latents, the audio latents and the video latents at once, which is why the blocks pass a row layout around rather than per-modality tensors, and why the pipeline carries two schedulers (shift = 12.0 for video, shift = 3.0 for audio) that are stepped inside a single transformer call. Unlike MiniMax-H3, the two schedulers are MiniMaxH3Scheduler instances with stochastic_sampling over the fixed six-step denoising_step_list: there is no num_inference_steps, and every step draws fresh noise from the request’s generator.

The adapters are guidance-distilled like their base: there is no guider, no negative_prompt and no guidance_scale, and every step runs exactly one forward pass.

This module carries the model facts every block keys off — the config-derived geometry as properties, the canvas, tier and frame-count arithmetic as functions — and the conditioning, encoding and noise contracts live on the blocks themselves.

Auto Modular pipeline blocks for joint video + audio generation with Pika H3 Flex: the t2va (text only), fl2va (first and/or last keyframe) and ref2va (omni-reference) workflows, selected on the references and keyframe inputs. The t2va and fl2va workflows run the transformer partition with the fl2va adapter and ref2va the transformer_ref partition with the ref2va adapter, so load the adapter of the workflow the request takes. Without a workflow=, loading the components pulls both 61.7GB transformer partitions — pass one to load only what the task needs.

Supported workflows:

• t2va: requires prompt
• fl2va: requires prompt, image or prompt, last_image
• ref2va: requires prompt, references

Components: image_processor (VaeImageProcessor) text_encoder (Qwen3VLForConditionalGeneration) tokenizer (Qwen2Tokenizer) processor (Qwen3VLProcessor) vae (AutoencoderKLMiniMaxH3) audio_vae (AutoencoderKLMiniMaxH3Audio) scheduler (MiniMaxH3Scheduler) audio_scheduler (MiniMaxH3Scheduler) transformer_ref (MiniMaxH3Transformer3DModel) transformer (MiniMaxH3Transformer3DModel) video_processor (VideoProcessor)

Configs: canvas_short_edge (default: 768) canvas_max_pixels (default: 1032192) denoising_step_list (default: [1000, 750, 500, 250, 100, 40, 0])

Inputs: references (list, optional): The references to condition on, in the order the model should read them: the order labels them in the prompt presentation and lays them out on the shared rotary clock, so a different order is a different request. One dataclass per modality, all holding in-memory media — a PikaH3FlexImageReference (at most 9), a PikaH3FlexVideoReference at its own fps (at most 3, whose audio soundtrack is conditioned on as well), or a PikaH3FlexAudioReference at its own sample_rate (at most 3) — for at most 12 references in total, and audio references cannot be the only ones. Every video or standalone audio reference runs 2 to 15 seconds, and the video durations and the standalone audio durations each sum to at most 15 seconds. These blocks never open a media file: decode with each class’s from_file classmethod, which brings the rates along. height (int, optional): Height of the generated video in pixels, a multiple of 32. width (int, optional): Width of the generated video in pixels, a multiple of 32. num_frames (int, optional): Number of frames to generate, at the fixed 24 fps: 120 to 360, i.e. 5 to 15 seconds, snapped up to the next 17 * n + 5 the video VAE can decode. To generate a video as long as a reference soundtrack, pass round(samples / sample_rate * 24). image (Image, optional): Keyframe the video starts from. It is center-cropped to the canvas’ aspect ratio and resized onto it; the canvas by default is derived from its own aspect ratio. last_image (Image, optional): Keyframe the video ends on. Can be passed on its own to generate up to a frame. It is center-cropped to the canvas’ aspect ratio and resized onto it, like image. prompt (str): The prompt to guide generation, a single string. normalized_references (list, optional): The references normalized by the setup step, in packed order. keyframes (list, optional): The keyframes put onto the target canvas, in packed order. condition_latents (list, optional): The encoded video conditioning latents, one per image and video reference in packed order. Their shape is where every reference block’s geometry comes from. audio_condition_latents (list, optional): The encoded audio conditioning rows, one per audio-bearing reference in packed order. generator (Generator, optional): The generator of the request. The conditioning noise is drawn from it first, one draw per condition, before the noise of the generated rows. latents (Tensor): Pre-generated video noise of shape (1, 24, num_latent_frames, latent_height, latent_width), used instead of the draw. audio_latents (Tensor): Pre-generated audio noise of shape (2, 32, num_audio_latents). *denoiser_input_fields (None,optional): The structural description of the packed sequence the transformer reads by name: token_tags, position_ids and the three row-index tensors. attention_kwargs (dict,optional): Additional kwargs for attention processors. keyframe_anchors (tuple,optional, defaults to ()): Which end of the video every keyframe is anchored to, in packed order. output_type (str,optional*, defaults to pil): Output format: ‘pil’, ‘np’ or ‘pt’.

Outputs: videos (list): The generated video. audio (Tensor): The generated soundtrack, of shape (1, 2, num_samples). sampling_rate (int): Sample rate of the generated soundtrack in Hz.

A subject, style or scene reference: at most 9 per request.

A motion and camera reference: at most 3 per request, conditioned on together with its own soundtrack.

A voice or music reference: at most 3 per request, and never on its own — an audio reference has to be paired with at least one image or video reference. It never reaches the conditioner and is encoded by the audio VAE alone.