Pika H3 Flex: synchronized video and audio in six steps, driven by up to 12 references

What Pika H3 Flex is

Pika H3 Flex is a set of open adapters post-trained by Pika Labs on top of MiniMax-H3. It does not run a video model and an audio model as unrelated stages. Video and stereo audio are generated together in one diffusion sampling process, then decoded from a shared timeline into an H.264 video stream and AAC audio. A request uses six model evaluations, not the usual much longer denoising ladder.

The more consequential idea is not text-to-video alone. The release puts two kinds of creative control behind one inference package. FL2VA handles a prompt with an optional opening frame, ending frame, or both. Ref2VA handles ordered multimodal references, allowing a creator to assign images, videos, and audio to specific jobs such as identity, motion, camera movement, environment, source footage, or sound.

Open weights are not the hosted API

The first boundary to get right is the difference between local open-weight inference and Pika's hosted service. The downloadable package supports native 480p and 768p output for requested durations of 5 to 15 seconds. Pika's API separately offers draft-to-1080p, 2K, and 4K generation, audio stems, and prompt enhancement. Those hosted extensions are not included in the local adapters.

Open weights also do not mean unrestricted commercial use. The inference and loader code is Apache-2.0, but the two adapters are model derivatives of MiniMax-H3 and use the MiniMax H3 Community License. The applicable territory excludes the European Union, the United Kingdom, the Republic of Korea, and the United States. There are additional conditions around downstream users, commercial revenue, model-improvement restrictions, safety measures, and required notices. A deployment review has to cover those terms rather than reading the Apache code license in isolation.

Two adapters with different control semantics

FL2VA is the endpoint-conditioned branch. With text only, it generates the scene, action, dialogue, ambience, and music. With a first image, that image establishes the opening state. With a last image, the model organizes motion toward the supplied ending. With both, the prompt describes a continuous transformation between the two anchors. Pika's guidance recommends a single continuous shot between endpoint frames unless a cut is explicitly part of the story.

Ref2VA is the reference-conditioned branch. It accepts up to nine images, three videos, and three standalone audio clips, with at most twelve files in total and at least one image or video. Input order is preserved. The prompt refers to assets as <Picture N>, <Video N>, <Audio N>, and <Subject N>, making each reference's role explicit. A video's embedded soundtrack remains part of that video reference and does not consume a separate standalone-audio slot.

References are conditions, not deterministic compositing operations. Visual relationships distinguish full preservation, partial preservation, attribute transfer, and weak reference. Audio can be fully copied, partially copied, used as a timbre reference, or treated only as broad atmosphere. Video continuation is likewise a prompting intent; this CLI does not expose a dedicated seamless-extension mode.

Task	CLI entry	Conditioning inputs	Representative use
Text to video and audio	t2va, FL2VA	One prompt of 1 to 8,000 characters	Multi-shot stories, motion graphics, product films, dialogue and ambience
First or last frame	i2va, l2va, FL2VA	Prompt plus one endpoint image	Start from an approved composition or land on a specified frame
First and last frame	fl2va, FL2VA	Prompt plus both endpoint images	Design the action and camera path between two fixed states
Multimodal references	ref2va, Ref2VA	Up to 9 images, 3 videos, and 3 audio clips	Identity and product retention, motion transfer, editing, sound reference

From references to an encoded file

The pipeline first resolves the task, canvas, duration, and reference specifications. It then downloads a pinned adapter and its pinned MiniMax-H3 base. Before loading the transformer, it validates the adapter format, LoRA rank and alpha, sampling knots, video and audio shifts, numerical policy, base repository, base revision, and adapter checksum. A mismatch stops the run before expensive model loading, which prevents a successful-looking but non-reproducible output from the wrong contract.

flowchart LR
  A["Prompt and optional endpoint frames"] --> B["Text and visual conditioning"]
  C["Ordered image / video / audio references"] --> D["Reference preflight and encoding"]
  B --> E["FL2VA or Ref2VA adapter"]
  D --> E
  E --> F["Six joint denoising evaluations"]
  F --> G["Video latent decode"]
  F --> H["Audio latent decode"]
  G --> I["H.264 video"]
  H --> J["32 kHz stereo AAC"]
  I --> K["MP4 plus generation metadata"]
  J --> K

The text and media conditions pass through the H3 conditioner, which is released before transformer sampling to reduce memory pressure. The transformer is built on CPU, injected with the adapter, and either moved to the GPU or configured for block-level CPU offload. After sampling, separate video and audio VAEs decode the latents. The final MP4 is then reopened with PyAV to verify the encoded canvas, exact frame count, 24 fps rate, 32 kHz sample rate, and two-channel audio. That last step checks the real container rather than only tensor shapes in memory.

Native output contract

Local generation supports 480p and 768p across 21:9, 16:9, 3:2, 4:3, 5:4, 1:1, 4:5, 3:4, 2:3, 9:16, and 9:21. The exact pixel size comes from the packaged aspect-ratio and resolution buckets; it is not an arbitrary resize from a fixed short edge. Output is 24 fps with 32 kHz stereo audio in an H.264/AAC MP4. A sidecar JSON records the prompt, seed, adapter hash, geometry, input order, sampling peak allocation, dependency versions, and timing breakdown.

The requested duration is an integer from 5 to 15 seconds, but generation follows a 17n+5 frame grid and rounds upward. A 5-second request produces 124 frames, about 5.17 seconds; a 15-second request produces 362 frames, about 15.08 seconds. The distinction matters when timing cuts or evaluating throughput. It is equally incorrect to describe the aspect-ratio control as free cropping: both canvas shape and pixel dimensions are resolved through the release's buckets.

Property	Open local inference	Important boundary
Resolution	480p / 768p, 11 aspect ratios	1080p, 2K, and 4K belong to Pika's hosted API
Duration	Requested 5 to 15 seconds	Rounded up to the 17n+5 frame grid
Picture and sound	24 fps, 32 kHz stereo	Muxed H.264 plus AAC MP4, not separate deliverables
Prompt	1 to 8,000 characters	Read verbatim; local inference does not enhance prompts
References	At most 9 images, 3 videos, 3 audio clips	12 files total, with at least one image or video

Six steps is a specific sampler, not six-step Euler

The adapter contract fixes six denoising evaluations with knots at 1000, 750, 500, 250, 100, 40, and 0, video shift 12, audio shift 3, and fresh intermediate noise. The implementation calls the method renoise. Replacing it with a generic six-step Euler sampler is not equivalent, even though both expose six model calls.

Seeds constrain the noise sequence but do not guarantee bit-exact output. Pika warns that attention kernels, devices, processes, offload modes, and library versions can change results. A locked software and hardware stack is more reproducible than a seed alone. Cross-device runs should be expected to preserve prompt intent and output validity, not pixels.

The showcased clips have the same limitation in a different form. They are creative examples, not release-validation renders. Their seeds, exact checkpoint revisions, and service preprocessing are not fully disclosed, and the repository explicitly says exact reproduction with the released package is not guaranteed. They are useful evidence of what the model is designed to explore, not a benchmark or acceptance corpus.

The samples show the intended creative surface

The four images below come from the repository's own showcase set. The product film combines a held-out product appearance with a clean studio, beat-driven cuts, macro shots, and sound design. The record-shop dance separates character, environment, motion, and camera references. The table-tennis rally uses a long prompt to choreograph overhead, crane, whip-pan, orbit, and dolly movement. The sitcom scene tests several shots, dialogue, a prop action, audience laughter, and a music cue on one timeline.

Compute is the constraint that survives the short sampler

Six evaluations reduce the number of sampling calls, not the model's footprint. Downloading the components for both tasks requires roughly 210 GB of base files plus about 5.5 GB for the adapters, with extra room for dependencies, download staging, and outputs. One request runs on one GPU. Exposing eight GPUs does not shard or accelerate that request; the idle GPUs can run independent processes.

Pika's representative H100 80GB tests measured about 66 to 69 GiB allocated during default 480p, 5-second sampling. Several 768p, 5-second and endpoint tasks also passed in default mode. A 768p, 15-second request exceeded H100 memory by default. With transformer block offload it allocated about 29 to 32 GiB for sampling while observed host memory, including page cache, approached 94 GiB. Those sampling figures exclude separate text-encoder and VAE stages, so they are not minimum system requirements.

Configuration	Evidence in the release	Deployment reading
480p, 5 seconds, text	H100 80GB default, about 66 to 69 GiB sampling allocation	A reasonable first qualification target
768p, 15 seconds	H100 80GB with CPU offload, about 29 to 32 GiB sampling allocation	Runnable, but leave headroom near 94 GiB host memory
Image, video, and audio references	Passed at 480p for 5 seconds	Does not qualify every maximum-size reference combination
H200 and B300	Not qualified for this package	Architectural similarity is not a verification result

Reference durations add another constraint. Every video or standalone audio reference must be between 2 and 15 seconds, and the total video duration and total standalone-audio duration must each stay at or below 15 seconds. Even a valid file count can exhaust memory. The fact that the preflight accepts a combination of references does not mean every GPU can complete it.

Minimal path to a first render

The package requires Python 3.12, a supported NVIDIA driver, and a compiler toolchain. Apply the pinned Diffusers patches before installing the editable package without dependencies. Skipping the patches can leave a different attention implementation than the one used for release validation. The first run downloads the pinned base and selected adapter. Local snapshot overrides are supported, but they are treated as trusted inputs and the command does not rehash every base shard.

python -m pip install -r requirements.txt
python tools/apply_diffusers_patches.py
python -m pip install --no-deps -e .

export MODEL_REPO=Pika-Labs/Pika-H3-Flex
export MODEL_REVISION=0ddf55ad7067ca1e76812253bd7b4600df87f759

pika-h3-pro \
  --task t2va \
  --prompt-file examples/prompts/t2va.txt \
  --model-repo "$MODEL_REPO" \
  --revision "$MODEL_REVISION" \
  --resolution 480p \
  --aspect 16:9 \
  --duration 5 \
  --seed 42 \
  --output outputs/sample.mp4

Start with 480p, five seconds, one GPU, and no extra references. Verify model download, conditioning, both decoders, MP4 encoding, and the sidecar metadata before increasing resolution, duration, or reference count. If memory is the limit, --cpu-offload can move transformer blocks between host and GPU, but it increases host-memory pressure and transfer time. It does not offload the text encoder or VAEs, and the reported sampling peak is not a complete minimum GPU requirement.

Where it fits, and where it does not

Pika H3 Flex fits professional ideation where a creator needs to test camera choreography, product films, multi-shot narrative, character-consistent animation, motion transfer, or synchronized dialogue and ambience. Ref2VA is especially useful when a production needs an explicit division of labor between subject, action, environment, camera, footage, and sound, instead of dropping every asset into an unstructured prompt.

It is not a validated general world model, robot-control model, or precision video editor. The package does not expose frame masks, keypoint constraints, physical guarantees, or deterministic identity locking. Reference preservation remains a generative condition. The maximum reference count is an accepted input bound, not proof that every full-size combination has passed on a given GPU.

Voice likeness, copyrighted footage, public distribution of real people, content provenance, and the territorial model license all require separate review. The useful conclusion is narrower and stronger: Pika H3 Flex makes the control roles of prompts, endpoint frames, and multimodal references explicit in a reproducible inference package, while keeping the hard questions around compute and licensing visible.