Audio & Video

Boundaries and failure modes, ordered from the ones specific to this channel down to the ones each member domain carries on its own. One, the easiest thing to misread at channel level: the union ladder is not a ranking. It pulls assets from both member domains into one list under a single ordering rule (featured, then maturity, then newest metric date) purely so a reader can see at a glance what we have reviewed on this line. An Elo of 1516 on video and a 100ms latency on speech are not commensurable, and appearing higher does not mean stronger. For a ranking, enter a member segment, or open /sota/video and /sota/audio and read each ruler separately. Two, missing evaluation keeps the whole channel conservative. Of the 12 boards we sync, there are text-to-video and image-to-video boards and no speech board, no TTS naturalness board and no ASR WER board; on music there is only third-party academic work such as WildSongBench, narrow in coverage and inconsistent in sampling budget across vendors (the 6.9632 for YuE2 is best-of-8 while the 6.8721 for Suno v5 is not the same budget, so the two numbers cannot be compared directly). Every water-level judgement on the audio side therefore carries vendor-claim; confirmed is reserved for verifiable public facts (GitHub star counts, licence, price, self-reported latency), and we run no blind listening test and recompute no WER. Three, alignment error in joint audio-video generation accumulates. A lip-sync miss is tolerable at 5 seconds and unmistakable across a 30 second narrative, and no third party publishes a protocol for measuring it, so vendor demos are always hand-picked samples. Read every "video with sound in one pass" claim as vendor-claim. Four, duration and physics remain the hard wall on the video side. Single-pass generation generally stops at 5 to 10 seconds; anything longer is built by relaying first and last frames or by stitching segments, and style drift and character inconsistency at the seams have to be picked out by hand. Fluids, cloth, multi-person interaction and hand-to-object contact are the frequent failure points, because what the model learns is the texture statistics of looking physical, not conservation laws. Five, most boundaries on the audio side are not technical. A few seconds of sample is enough to clone a voice, while consent chains, watermarking and abuse detection are all immature; open-weight TTS can be withdrawn over abuse (there is a 2025-09 precedent), so "open and usable today" is not "available long term". We fold compliance status into the confidence judgement. On music, training-data copyright litigation is unresolved, so check the licence before any commercial delivery. Six, latency and real-time behaviour are barely measured in evaluations. First-packet latency for high-quality TTS typically runs from a few hundred milliseconds to seconds, and genuinely usable real-time conversation needs streaming synthesis plus barge-in, so the most natural model on a board may be unusable in a call. Long-text consistency (paragraph prosody, how numbers and proper nouns are read, ambiguous readings, mixed Chinese and English text) is the most common production failure, while evaluation sets usually read only short sentences. Seven, cost stacks along two lines, so do not budget one side only. 720p video runs about $0.10 per second, roughly $3 for a 30 second clip, and three to ten re-rolls is normal in commercial delivery; speech is billed per second or per character, where bulk TTS unit prices sit far below video but the cost of a real-time channel lives in concurrency rather than duration; music generation is close to the video magnitude, with a single track taking tens of seconds to a few minutes. A finished narrated piece with a score must be budgeted as video re-rolls plus multiple voice takes plus multiple music takes, not as three unit prices added together.