From Lyrics and a Song to a Finished Lyric Video, Without Touching an Editor
Making a lyric video is not hard. It is just tedious. You line up words with vocals by ear, find or make an image for every scene, keep the visuals from contradicting each other, add motion so it does not look like a slideshow, and then re-render the whole thing because one line landed half a second late.
So I built a tool that does it: feed it a lyrics text file and an audio track, and it hands back a finished video. It is open source, end to end. Browse the source on GitHub.
Here is a video it made autonomously in 50 minutes. Doing the same thing by hand would have cost me a few days. (And can you guess how I made the input lyrics and song?)
The story comes first
The first thing the pipeline does is not timing or images. It reads the lyrics and works out what the song is about — the storyline, the characters, the arc.
That ordering matters more than it sounds. The obvious build generates an image per line, independently. What you get is a slideshow of unrelated pictures that happen to share a song: the protagonist changes appearance every four seconds, the season flips, the palette lurches. Technically correct, visually incoherent.
Deriving the storyline once, up front, gives every later stage a shared reference. Each scene prompt is written against that story rather than against an isolated line of text, so the visuals hold together across three minutes instead of resetting at every cut.
What it does, in order
Once the story exists, the rest is a pipeline:
- Times every lyric to the actual vocals, down to the syllable — against the real audio, not an estimate from line lengths.
- Writes an AI image prompt per scene, each one grounded in the same storyline.
- Generates a unique visual for every line.
- Adds zoom-ins and zoom-outs plus cross-dissolves, so the result reads as motion rather than a slideshow.
- Fades the lyric overlays in and out in sync with the vocals.
- Fades the song out at the end.
- Renders the video with ffmpeg, the open-source engine underneath the whole thing.
None of these steps is exotic on its own. The value is that they are one command, and that each stage receives clean, structured input from the one before it.
Two ways to run it
You can run it fully hands-off: lyrics in, video out, no supervision.
Or you can run it stage by stage — inspect the storyline, check the timings, look at the generated images, adjust anything, then continue. The pipeline is the same either way; the only difference is whether it stops to let you look.
The mode I actually use
My favorite way to work is neither of those, exactly. I run it end to end, watch the result, and then fix the one thing that is off — swap an image I do not like, nudge a lyric’s timing by a beat — and rerun.
The tool knows what changed, and only rebuilds what is downstream of it. Replacing a single image does not re-time the vocals or regenerate the other forty scenes. A two-second fix does not cost a full re-render.
This is the difference between a generator and a tool. A generator gives you a take and you accept it or start over. Because every stage writes its output as inspectable state, an edit is a small, cheap correction rather than a reroll — which means you keep iterating instead of settling for the first version that was close enough.
What I take from it
The interesting part was not the image generation, which is the flashy bit and by now the easy bit. It was the two decisions around it: establishing the story before generating anything so the visuals share a world, and keeping intermediate state so a small fix stays a small fix.
AI produced the art and the prompts. ffmpeg produced the video. The pipeline’s job was to make sure they were all working from the same script.