The AI video generation has come a long way in a short time, from 2-second clips with significant morphing and distortion to shots that look nearly indistinguishable from filmed footage. Runway is the latest player in the space to release its next-generation model.
Gen-3 was first revealed two weeks ago and after some initial testing by creative partners, it is now available to everyone, at least the text-to-video version. Text-to-image is coming soon.
Each generation produces a photorealistic 10-11 second clip with surprisingly accurate movement, including rendering of human actions that reflect the scenario and environment.
My initial tests show that it’s as good as Sora at some tasks, though it’s better than OpenAI’s video model because it’s publicly available. It’s also better than Luma Labs’ Dream Machine at understanding motion, but without an image-to-video model it falls short on consistency.
What is it like to work with Gen-3?
View on
I’ve been playing with it since launch and have created over a dozen clips to effectively refine the prompting process. “Less is more” and “be descriptive” are my main takeaways, though Runway produces a handy guide to prompting Gen-3.
Try to get the prompts right from the start, as each generation with Gen-3 costs between $1 and $2.40 per 10 second generation. The cheapest option is to upgrade credits, which cost $10 per 1000. In contrast, Luma Labs’ basic plan costs 20c per generation.
As for actually using the video generator, it works exactly like Gen-2. You give it your prompt and wait for it to create the video. You can also use lip-sync, which is now integrated into the same interface as video creation and animates across the entire video.
I came up with five prompts that worked particularly well and have shared them below. Until the launch of image-to-video, you have to be very descriptive if you want a certain look, but Runway’s Gen-3 image generation is impressive. You also only get 500 characters for a prompt.
1. Cyber City Race
This was one of the last prompts I created and built from refinement. It’s relatively short, but because of the specific description of both movement and style, Runway interpreted it exactly as I expected.
Prompt: “Hyperspeed POV: Racing through a neon-lit cyberpunk city, data streams and holograms blur as we zoom into a digital realm of swirling code.”
2. Diver
The first part of this had some weird motion blur over the eyes and the extended fingers that corrected themselves. Otherwise it was an impressive and realistic interpretation. The problem with the motion blur was the part of the prompt that suggested sunlight was shining through. The prompt was too complex.
Prompt: “Slow motion tracking shot: A diver explores a vibrant coral reef full of colorful fish. Shafts of sunlight pierce the crystal clear water, creating a dreamy atmosphere. The camera pans past the diver as they encounter a curious sea turtle.”
3. A street scene
This is not only one of my favorite videos from Runway Gen-3 Alpha, but of anything I’ve made with AI video tools in the past year or so. It didn’t quite follow the prompt, but it captures the sky changing throughout the day.
Prompt: “Hyperspeed timelapse: The camera rises from street level to a rooftop, showing the transformation of a city from day to night. Neon signs flicker to life, traffic turns into streams of light, and skyscrapers glow against the darkening sky. The final frame reveals a breathtaking cityscape under a starry night.”
4. The bear
I overwrote this prompt. It was supposed to show the bear getting more and more animated towards the end, but I was asking him to do too much in 10 seconds.
The prompt: “Slow motion close-up to wide angle: A worn, vintage teddy bear sits motionless on a child’s bed in a dimly lit room. Golden sunlight gradually filters through lace curtains, softly illuminating the bear. As the warm light plays on his fur, the bear’s glassy eyes suddenly blink. The camera pulls back as the teddy bear slowly sits up, his movements becoming more fluid and lifelike.”
I refined the assignment to: “Slow motion close-up to wide angle: an old teddy bear on a child’s bed comes to life as golden sunlight streams through the lace curtains. The camera pans back to reveal the bear sitting up and beginning to move.”
This gave a better movement, which was in reverse order of the original movement. However, there were some artifacts on the bear’s face and the bear still could not sit up.
5. The old farmer
This was the first prompt I tried with Runway Gen-3 Alpha. It’s too complex and descriptive because I was trying to replicate something I would make using image-to-video in Luma Labs Dream Machine. It wasn’t the same, but it was very well done.
Prompt: “Sun-beaten farmer, 70, surveys scorched fields. Leathery skin, silver beard, eyes squinting under dusty hat. Worn shirt, patched overalls. Calloused hands grip fence post. Golden light illuminates anxious lines, determination. Camera zooms in on steely gaze. Barren land stretches out, distant ruins loom. Makeshift irrigation, fortified fences visible. Old man reaches into hat, reveals hidden technology. Device flickers, hope dawns.”
More from Tom’s Guide
Back to MacBook Air
