Two AI Agents, One Song, $100: An Autonomous Music-Video Face-Off

Researchers built an open-source agentic harness (github.com/hershalb/music-video-arena) that hands a model a song, a hard dollar budget, and a toolset, then steps back. Each agent researches available generation models, calls them via FAL or Replicate, inspects its own footage, and edits with local ffmpeg to assemble a final cut. They pitted Claude Fable 5 against GPT-5.6 Sol on the same brief — Bruno Mars and Mark Ronson’s “Uptown Funk,” plus a short description and a timestamped lyric transcript — running each at $25 and $100 for four runs total. Every plan, tool call, charge, and error was logged.

All four runs completed unattended and produced full-length videos with the original track muxed in. The budget caps only generation spend, not the LLM tokens: at $100, Sol spent $36.57 on footage versus Fable’s $48.60, and Fable’s token bill ran $17–$25 per run (30–40% of its total) against Sol’s $3–$4, making Fable the pricier pick overall at $73.65 despite finishing faster. Tool choices diverged too — three runs went pure text-to-video, Sol at $25 built an image-to-video pipeline and did the most inventive editing (text overlays, animated stills), and Sol at $100 mixed three video models in one run. Neither model touched Replicate.

The more interesting result is where frontier agents still fall short. All four struggled with character and story consistency, took lyrics absurdly literally (“make a dragon wanna retire” yields an actual dragon), and matched cuts to the beat while the motion inside clips ignored tempo. Most tellingly, none seriously reviewed their own output: once clips existed, the models concatenated and shipped rather than re-cutting or probing whether the footage was any good, and $100 of headroom went largely unused. The gap isn’t raw capability so much as the judgment to iterate — exactly the kind of long-horizon, subjective task where autonomous agents remain weak.