Multimodal

The failure modes in this domain are the quietest and the most expensive. One, detail extraction is unreliable: small text inside an image, numbers in a table, the state of a dense interface - models produce fluent answers with wrong digits and do not express hesitation; document and chart QA score very well on benchmarks while production still requires a human check on any critical number, so every "read a value off an image" conclusion is defaulted down one grade. Two, resolution and tiling strategy are hidden variables: the same image processed differently (whole, tiled, or downsampled) gives very different results and vendors generally do not disclose it, which strips cross-model comparison of meaning. Three, temporal alignment across long video and many images is still weak: a model can describe a single frame but cannot reliably say what happened first, and video understanding starts to forget after tens of seconds. Four, hallucination decouples from visual grounding: models fill in objects that are not in the image using training priors, especially when the image is blurry or the question is leading, and this is the shared upstream problem between this domain and safety. Five, the benchmarks are narrow: MMMU and ChartQA use clean screenshots and well-formed charts, whereas the real world is tilted phone photos, compressed scans and mixed-layout PDFs, and out-of-distribution performance is consistently well below what the board shows. Cost and latency: visual input is billed far more heavily than text (one high-resolution image commonly costs the equivalent of hundreds to over a thousand tokens), multi-turn image conversation grows cost an order of magnitude faster than pure text, and on latency the time to first token is dominated by image encoding, so preprocessing a large image is often slower than generating the answer.