Four-second clips can't make a movie

It’s been a busy month for video generation. Stability released Stable Video Diffusion last week, open weights that turn an image into a couple of seconds of video. Runway added Motion Brush to Gen-2, so you can paint a region of an image and tell it which way to move. And today Pika announced Pika 1.0 with a slick launch video and a lot of funding. The individual clips from all of them are often gorgeous.

Since I wrote about Make-A-Video last September, resolution and visual quality have improved enormously. Physics is still shaky but better. I predicted identity consistency within a clip would come next, and it mostly has, for short clips.

I spent a weekend trying to make a one-minute short with these tools: a simple scene, two characters having a conversation at a table, then one of them leaving. I’ve done some amateur filmmaking, and I know the basics of how scenes are cut together. It didn’t work, and the reasons are instructive.

Every clip is generated independently, so nothing carries over. The character’s face changes a little between shots, sometimes a lot. The jacket changes color. The cup on the table moves or disappears.

The 180-degree rule breaks. In a conversation, you keep the camera on one side of the line between the two characters, so one always faces left and the other always faces right. Cross the line and the audience gets disoriented. The generator has no idea there’s a line, so in one shot she’s looking left, and in the reverse shot he’s also looking left, and it reads as if they’re looking at the same thing instead of at each other.

Eyelines don’t match. If she looks up at him because he’s standing, his shot needs to look down at her. The generator doesn’t know where the other character is.

Screen direction breaks. If he exits frame right, he should enter the next shot from frame left. The generator makes him enter from wherever it likes.

None of these are visual quality problems. They’re the grammar of film, the rules that let separate shots feel like one continuous space and time. Editors and cinematographers learn them because audiences feel it immediately when they’re broken, even if they can’t say why.

Longer clips won’t fix these problems on their own. I want a system that keeps track of the characters, props, positions and camera layout, then uses that shared scene to generate each shot. That probably means an explicit 3D or 2.5D scene layout, and character references that carry across shots, orchestrated by something that understands coverage and editing. I haven’t seen anyone working on that layer yet, and I think whoever does will matter more than whoever makes the prettiest four seconds.