Aleksandra Konoplia/Moment via Getty Images
Follow ZDNET: Add us as a preferred source on Google.
ZDNET’s key takeaways
- Ray3 has multimodal reasoning abilities, according to Luma AI.
- AI developers are pushing video models onto creative industries.
- Some tools are positioned as automated creative partners.
Just a few years ago, AI-generated video clips were a laughing stock on the internet — anyone remember the nightmarish video of AI-generated Will Smith wolfing down spaghetti? The technology has come a long way since then: Today, tech startups are competing to deliver generative AI tools which, at least in their vision of the future, aim to rival the quality of Hollywood production studios — at a tiny fraction of the cost.
Also: This new AI video editor is an all-in-one production service for filmmakers – how to try it
In the latest development in that competition, AI startup Luma AI announced its new video-generating model, Ray3, on Thursday. Its other product, Luma Dream Machine, lets users create videos from just their photos.
The model is available now through Dream Machine. It’s also accessible to paying customers of Adobe’s Firefly and Creative Cloud Pro, who can generate unlimited videos through the model until Oct. 1.
A “reasoning” AI video model
You’ve heard of so-called reasoning models like OpenAI’s o3, which are thought to consider a query, especially complex ones, for more time than standard generative AI models in order to return a more helpful and thorough answer. But thus far, those models haven’t had video-generating capabilities.
Also: AI’s not ‘reasoning’ at all – how this team debunked the industry hype
Ray3, according to Luma AI, stands apart from competing tools from companies like Google, Runway, Meta, and OpenAI primarily due to its ability to “reason” while generating video clips.
This is a vaguely defined and ontologically debatable term that’s thrown around a lot these days in how AI systems are marketed, a bit like “understanding,” “creativity,” and “agency.” In simple terms, it refers to a model’s ability to break problems down into multiple steps, reflect on the quality of its outputs, and iteratively improve upon them over time.
Rather than just generating video from a text prompt, Ray3 breaks the production process down into multiple steps, just as a creative team would. It has multimodal reasoning capabilities, meaning it can generate text along with visual assets to help users sketch out the concepts for the final video.
Filmmakers could, for example, prompt the system to annotate images or suggest camera angles for sequences of shots. The model is also the first of its kind, according to Luma AI, to be able to deliver video outputs in 4K high dynamic range, which means it offers a much broader visual spectrum of light and shadow.
Also: Will AI damage human creativity? Most Americans say yes
“The result is videos that feel more coherent, with characters that look consistent, scenes that unfold naturally over time, and physics that behave as they should,” Luma AI wrote in a press release.
Luma AI has not publicly disclosed any limits on the length of videos that Ray3 can generate, and the company did not immediately respond to ZDNET’s request for comment on this subject.
Automated creative partners
Luma AI is positioning Ray3 as an automated creative partner for filmmakers, video game designers, and advertisers.
A new “draft mode,” for example, enables Ray3 to quickly generate a variety of test clips, each with subtle variations, providing creative teams with a range of options and saving them time on the ideation process.
“This lets creators enter a state of flow, experimenting freely without worrying about time or compute costs,” Luma AI wrote.
Also: AI models know when they’re being tested – and change their behavior, research shows
Other AI developers have been selling their own tools on the premise that they can serve as automated creative partners, onto which humans can offload time-intensive and routine tasks, saving them money in the process. Earlier this week, for example, Amazon unveiled an AI agent that can help brands with virtually every step of the process of creating a short video ad.
