Building a Stage-Ready 3D Anime Music Video with GPT, Muse, and Opus 5.5

"Zero dollars" means no extra asset purchases

The most useful contribution in Hailey's article is not another AI-generated dance clip. It is a method for turning generative video from a slot machine into a production pipeline that can be divided into roles and revised in pieces. GPT handles character design, the script, storyboards, and choreography direction. Muse generates green-screen dance footage or a complete music video. Claude Opus 5.5 writes the code for keying, a Three.js stage, music analysis, lighting, particles, camera motion, and compositing.

The "zero dollars" claim needs a precise reading. Hailey means that she did not buy additional footage, music, or generation credits. She used tools and subscriptions she already had. It does not mean that anyone can reproduce the result without cost. Muse, Claude Opus 5.5, GPT, and similar tools still require subscriptions, credits, or API budgets, and repeated video generation can become expensive. What disappears is the physical studio, actors, 3D art assets, and conventional post-production crew, not all software and compute costs.

The real question is not which model can generate an entire music video in one click. It is how to let different models do what they are good at, then connect their outputs to an editable timeline.

That is why the result combines the organic motion of generated video with the stable camera work of a conventional 3D scene. A video model owns the performer. Code owns the geometry, materials, lights, and camera. The former is good at producing expressive but uncertain content. The latter is good at executing precise, repeatable rules. They are not compressed into one prompt. They meet on a timeline.

Split the assets before discussing one-click generation

If an entire music video is handed to a single video model, one prompt must simultaneously control character consistency, choreography, camera, lighting, scenery, song structure, and editing. If one part fails, the creator may regenerate the whole clip and lose the parts that already worked. Hailey starts by splitting the finished video into three replaceable asset classes.

Asset	Primary tool	Output	How to revise it
Character and script	GPT	Identity, personality, costume, storyboard, and motion intent	Change the text specification and regenerate only the affected asset
Motion and sound	Muse, Grok Bot, Suno, and others	Green-screen dance, a complete generated music video, or music	Replace a clip without rebuilding the entire stage
Stage and compositing	Claude Opus 5.5 and Three.js	Keying, prism geometry, lights, particles, camera, and synchronization	Change code or parameters and reproduce the result

Character design is more than a line such as "silver-haired cyber idol." The specification for NOVA includes her role, age, personality, visual keywords, front and side views, facial expressions, eye details, hair color, costume materials, accessories, and dance-style references. The purpose is not to produce one attractive character sheet. It is to give every later dance and storyboard generation a stable visual constraint.

In engineering terms, this resembles defining an interface. The character sheet defines input constraints. The storyboard defines timing constraints. Music and motion events define the synchronization interface. Once those interfaces are stable, the underlying generation model can change and the stage code can keep evolving. The expensive part is not one generation. It is the ability to redo only a small piece when the brief changes.

Route one: generate dance first, then make the music follow

In the first route, Hailey obtains a green-screen dance clip, analyzes its motion and the music, and drives the stage from what the body is already doing. This route suits a project where the dance is the first asset worth protecting. The performer carries the creative idea. Music, lighting, and camera organize themselves around motion that has already happened.

The performer clip is treated as a container of motion rather than a locked final image. The code reads video frames, recognizes body pose, motion phases, and timing, and turns the movement into events. Hailey reports that one version detected 38 motion events and found 20 synchronization points between music and motion. An event can be a raised hand, a turn, a drop, a formation change, or a held pose. A synchronization point is the instant when the pose reaches a key position and the music, light, or camera should change with it.

The panel shows both audio and motion events and distinguishes states such as pickup, hold, and waiting. A note at the bottom explains that the audio events are analyzed from the WAV rather than read from a score. The program uses HPSS and band filtering to find transients, with a synchronization tolerance of about 100 milliseconds. That detail matters. The system is not "understanding music" in a musical sense. It detects kicks, snares, cymbals, energy changes, and beat positions, then maps those measurable signals to visual parameters.

An early version used a 157 BPM track and visualized its kick, snare, and beat clock. A later interface shows about 122 BPM and divides the motion timeline into break, drop, and end regions. The numbers do not need to be forced into one value because Hailey changed the music and footage across iterations. The stable idea is the shared timing coordinate: once audio events and motion events occupy the same axis, stage parameters can become functions of that axis.

Synchronization signal	Visual effect it can drive	Why it works
Kick and low-frequency energy	Floor pulse, small camera shake, volumetric-light flashes	Low impacts support short, clearly visible visual responses
Snare and high-frequency transients	Particle bursts, prism glints, slices of light	Sharp sounds pair naturally with bright, granular visual elements
Motion phase and key poses	Shot changes, scene color, formation lighting	The visual center follows the body, so synchronization is not just mechanical flashing
Section boundaries	Stage structure, background material, camera distance	A break, drop, and ending need different visual states, not merely larger parameters

The advantage is that music and image can feel causally connected rather than merely hitting the same beat. A hand rises before light sweeps across it. A pose freezes and the camera briefly settles. When the chorus begins, stage geometry and particle density change together. Code turns those relationships into reusable rules instead of requiring frame-by-frame hand animation.

Route two: generate music first, then direct dance from a storyboard

The second route reverses the order. Music is fixed first. GPT then designs the story and shot list around the song, and Muse generates dance according to each shot. Hailey shows a version of roughly 55 seconds with 15 shots. The storyboard specifies each segment to the second, including performers, framing, action, and camera movement.

The value of this table is that it turns "generate a cool girl-group music video" into executable segments. From 0 to 3 seconds, the stage is dark and the three silhouettes approach through rising light. From 3 to 6 seconds, the edit cuts to a NOVA close-up. From 6 to 9 seconds, all three appear together. RIN and LUNA receive solo shots before the video moves into formation changes, chorus choreography, member interaction, and a final held composition. Every shot has a time range and a purpose, so the result no longer depends entirely on one long prompt.

Character roles are written into the plan. NOVA is the center, lead vocalist, and owner of the opening and chorus. RIN is the main dancer and rapper, responsible for explosive movement and rhythmic accents. LUNA is a supporting vocalist who supplies a sweeter expression and member interaction. This prevents all three characters from performing the same step while failing to give the audience a reason to distinguish them. In a multi-character video, narrative function is more useful than stacking adjectives such as "more detailed, more spectacular."

Music-first production gives the project a clear structure. GPT can assign different shot density and stage states to intro, verse, pre-chorus, chorus, bridge, and ending. Muse receives explicit sections, beats, and emotional intent instead of guessing where a full song should become more intense. The trade-off is that exact alignment between music and a specific body movement still needs review, especially when cuts and center-of-mass changes do not naturally coincide.

A code stage turns stochastic generation into an editable system

The most transferable part of the project is the Three.js stage written with Claude Opus 5.5. The performers remain two-dimensional video planes, but the stage, prisms, floor, lights, particles, and camera are real 3D objects. Code keys the green screen, separates the performers from the background, and places them in a scene whose perspective is shared by the camera.

This differs fundamentally from asking a video model to generate the performers and stage together. A video model changes details on every sample. The stage cannot be reused reliably, and changing one light color may resample the entire dance. A code stage is deterministic. The same scene parameters produce the same geometry. Lights can be changed independently. Camera motion can be changed independently. Particle density and trigger rules can be versioned. If the performer is not good, replace the performer without rebuilding the stage. If the stage is not good, modify the code without throwing away the motion.

// Conceptual outline, not the original source code
const timeline = buildTimeline({
  audioEvents,
  motionEvents,
  syncNodes,
});

for (const node of syncNodes) {
  timeline.at(node.time, () => {
    stage.setEnergy(node.audioEnergy);
    lights.pulse(node.beatStrength);
    particles.emit(node.motionLabel);
    camera.applyShot(node.shot);
  });
}

This pseudocode summarizes the layering shown in the article. Music analysis, motion analysis, synchronization nodes, and visual execution remain separate. A real implementation must also handle video decoding, green-screen thresholds, spill suppression, transparent edges, video texture updates, Three.js materials, and post-processing. Code does not replace taste. It executes relationships that the creator has already decided should be stable.

Green-screen keying also has clear limits. First, the performer remains a 2D video plane rather than a true 3D human. When the camera moves around the side, the person does not reveal a new viewing angle like a physical model. Second, foot contact, ground shadow, perspective change, and occlusion often require manual correction, or the performer appears to float above the stage. Third, hair, translucent costumes, and fast motion produce edge spill, and a crude threshold removes detail along with the background.

The article therefore demonstrates a useful compromise rather than a complete solution to video compositing. A video model supplies body motion. Code supplies a controllable space. This compromise is effective for short music videos, concept films, and social content. Projects that require true camera orbits, precise physical interaction, or sustained close-ups still need real 3D characters, motion capture, or a more elaborate compositing pipeline.

Three model combinations, three different bottlenecks

Hailey does not present one model stack as the only answer. She compares three routes. That comparison is more useful than a generic tool recommendation because each route fails for a different reason.

Combination	Strengths	Main problem	Best use case
Grok Bot + Muse	Natural character integration, simple workflow, lower cost	Weaker resolution, beat alignment, and fine stage control	Fast, coherent short videos where turnaround matters more than directability
Muse + Opus	Fully controllable and reusable stage, free adjustment of light and camera	Harder performer-to-3D compositing and manual green-screen cleanup	Music videos that need a distinctive stage, brand consistency, or many revisions
Grok Bot + Muse + Opus	Music, performer, and stage can be revised separately, raising the ceiling	Longest pipeline and the most complex asset handoff and synchronization	Full projects with time for several rounds of iteration, rather than same-day output

The first combination gives more control to the video model. It can produce a coherent clip that blends performer, environment, and music, but it is hard to change one element without changing the rest. The second gives the stage back to code, at the price of working with 2D green-screen footage. The third has the most room to scale, but the author must maintain text specifications, video clips, audio events, motion events, and code versions at the same time.

This is where claims that "AI makes creation simple" are often incomplete. The tools reduce the operational cost of modeling, rendering, and editing. They do not eliminate project management. More assets require clearer naming, timelines, and rollback paths. A mature workflow knows which steps can tolerate randomness and which must execute deterministically.

Music first or dance first?

Neither route is universally better. The order depends on which asset the creator wants to protect first. If there is already a dance worth keeping, music and stage should follow the body. The performer has the strongest presence, but the music must be adapted to the movement, and synchronization becomes more expensive. If there is already a complete song, shots and dance should follow the song structure. The progression is clearer, but each shot must match the song's duration and beats, and dance controllability becomes the main risk.

Decision point	Dance first	Music first
Starting asset	A green-screen motion clip worth keeping	A complete song or a clear section structure
Synchronization method	Detect motion events, then locate musical nodes	Split the song, then generate action for 15 shots
Main advantage	Natural body motion, with the stage built around its peaks	Clear structure and consistent control of shot density and emotion
Main risk	Music may lose coherence while being forced to follow animation	Dance may feel mechanical or fragmented while hitting the cuts

A more practical approach is to generate a 15 to 30 second proof before committing to a full song. The proof exposes the expensive problems: character consistency, foot sliding, green-screen edges, camera-induced proportion changes, and whether the beat map is too dense. If those problems are not solved in 15 seconds, extending the video to 55 seconds only multiplies them.

Complete generated music videos and the street-style variant

Hailey also shows Muse generating a complete music video directly, and a three-member version in which Suno supplies music and Muse generates images for that music. They demonstrate that one idea can be produced at different levels of automation. Direct generation treats the model as the whole director. The code-compositing route treats it as an asset supplier.

A 54 second K-pop version and a 30 second street version show that prompt writing is not only a matter of describing a musical genre. Character relationships, stage space, cutting rhythm, costume materials, and dance style all change the result. The street version moves the scene to graffiti walls and a wet floor and shifts the costumes toward looser streetwear. The character specification, scene description, and movement style must change together. Replacing one background word is not enough.

Why the failed version is worth keeping

The article also shows an early attempt built from SVG or keyframes. The beat clock, character, and interface are present, but the body does not move like a dancer. Keeping that failure is valuable because it shows that drawing the beat and making a body dance are different problems.

This is where video generation remains difficult to replace. Code is good at time, structure, and repetition. A video model is good at producing continuous body poses learned from large collections of human movement. Put them at the correct layers and the project can gain both physical expressiveness and stage stability. Force code to animate every joint and the project turns into motion capture and character rigging. Force one video model to own stage and camera and the project loses editability.

What this means for robot learning

This is not a robotics article, and music-video compositing should not be presented as evidence of robot capability. The structure still resonates with some robot-learning problems: detect events from a demonstration, then let a deterministic system execute them. The motion events in the video are analogous to phases, keyframes, and contact states extracted from a human demonstration. The Three.js stage and synchronization logic are analogous to a simulator, controller, and scheduler.

A robot can adopt a similar division of labor. A learned model can interpret vision and language and propose actions. A deterministic controller can enforce joint limits, collision constraints, timing, and safety. A real system cannot optimize only for visual plausibility. It must satisfy physical feasibility, closed-loop feedback, and recovery. The transferable idea is the architecture, not the generated video itself.

Three principles are especially reusable. First, compress a high-dimensional demonstration into an event sequence to simplify temporal alignment. Second, require a generative model to emit intermediate assets with timing and semantic labels instead of one opaque final output. Third, place a deterministic execution layer after those assets so results can be repeated, compared, and rolled back. For a robot, that execution layer must be far stricter because errors occur in the physical world.

Variables to control when reproducing the workflow

Do not begin by generating a complete song. Reduce the variables first: one character, one fixed stage, 8 to 15 seconds of music, one clear movement section, and one camera. Confirm that the character, green-screen edge, and floor shadow work before adding a second performer or more cuts.

1. Lock the character sheet. Costume, hair, palette, and facial features must remain identifiable across shots so every generation does not create a different person.
2. Lock the video specification. Use one frame rate, aspect ratio, and performer scale. Mixing landscape, portrait, and incompatible perspectives quickly breaks stage compositing.
3. Build the timeline first. Put song sections, motion events, and shot boundaries in one table before adding lights and particles.
4. Protect readability. Wide, medium, and close shots need distinct functions. If every second flashes and explodes, the ending has no climax.
5. Keep rollback paths. Store performers, audio, stage code, and compositing projects separately so any layer can be replaced.
6. Count the whole budget. Generation attempts, storage, transcoding, edge cleanup, and compositing review are all costs. Zero extra asset purchases does not mean zero labor.

The examples also show that the best improvement usually comes from changing an asset boundary rather than making a prompt longer. If the dancer is wrong, regenerate the dance instead of adding adjectives to the whole stage. If the stage lacks depth, adjust the code's space, materials, and lighting instead of regenerating the performer. The ability to revise only the broken layer is what makes this pipeline more mature than a single generation.

Conclusion: from slot-machine generation to a production system

Hailey's tutorial describes a clear progression. First, ask a model to generate a complete video. Next, separate character, motion, music, and stage. Then use code to stabilize the parts that should be deterministic. Each step makes a single generation look less magical, but it makes the overall process more like a real creative system.

GPT is not the final director, Muse is not a complete post-production department, and Claude Opus 5.5 does not supply automatic taste. They handle text planning, visual generation, and program implementation. The value comes from the interfaces. The character sheet constrains the video model. Motion events feed the stage code. Music analysis supplies temporal evidence. Three.js organizes the result into shots.

The method still depends on human judgment. Someone must decide which dance is worth keeping, which cut has force, which composite looks like a sticker, and which light change is merely noise. AI lowers the cost of experimentation while raising the demand for asset selection and system organization. The important shift is not finding a better one-click prompt. It is learning to manage model outputs like a small production pipeline.

Sources

Original article: Hailey (@HaileyAILabs), "Zero-dollar challenge: use AI and code to make a 3D anime music video with spectacular stage effects, full tutorial," X Article, published November 1, 2026.

Related demonstrations and process notes: the early 157 BPM rhythm stage, the Muse + Opus result summary, code keying and the dynamic stage, Grok Bot music + Muse complete music video, Suno + Muse + GPT girl group, the 54 second K-pop version, and the 30 second street-style version.