The Best AI Text To Video Generators

ChatGPT heralded an explosion of AI models able to generate text from a single prompt.

MidJourney showed us how realistic AI generated images could get.

And now we're seeing more and more AI text to video models creating videos from a single text prompt that would have taken a CGI graphics team months to develop.

VEED

So the first text to video AI tool on the list that I'm going to start with is VEED. Now their text to video generator is a little bit different from some of the others coming up, but I wanted to include it as you may find it much more practical for your purposes.

Rather than generating video from scratch using video generation models, VEED's AI text to video generator creates an AI voiceover and script and and generates compelling subtitles and then it sources appropriate stock audio and background music and footage to build out a video in full from your text prompt.

You can then edit the video in Veed's easy to use cloud based editing system and the cool thing here is that subtitles and the cuts optimize the video for engagement and it's great if you want to try out some quick short form video content for socials.

It's also really fun to play around with the different prompts and see what you get back.

RunwayML

Now, one of the earliest and most impressive AI text to video tools was RunwayML, which has been leading the charge when it comes to text to video AI. Its original Gen 1 model is available on iOS, and Gen 2 is able to realistically and consistently synthesize new videos. And it's really exciting to see what lies ahead for AI video generation from the Runway team.

Runway itself is essentially a suite of AI video tools, featuring everything from training custom models, to slowing down and removing the background of existing videos.

Gen 2 in particular is absolutely incredible, with live video masking, and generative video from text.

One of its latest features allows us to control specific areas of the video using the motion brush tool and it gives amazing control over how the images then animate.

You can kind of think of generative video systems like Gen 2 as very early and limited forms of general world creation models.

In order for Gen 2 to create realistic, short videos it's developed some understanding of physics and motion however it's still very limited in its capabilities struggling with complex camera or object motions amongst other things and creating humans that look realistic a little bit like some of the ai texture image tools but no doubt with future models this is only going to get better and more impressive so runway ml is definitely worth checking out.

OpenAI Sora

Sora is OpenAI's AI model that can create realistic and imaginative scenes from text instructions.

Sora is absolutely incredible and is able to generate complex scenes with multiple characters, specific types of motions, and accurate details of the subject and the background. The model itself understands not only what the user has asked for in their prompt, but also how those things exist in the physical world.

There are a bunch of amazing examples that OpenAI have put out, and although Sora is still in the research phase and isn't publicly available just yet, some of the tests we've seen and the videos generated are just amazing. absolutely mind blowing and if you compare this to the time and the effort it might take a team of computer generated videographers to create something similar for a film or for a short form advert it's absolutely mind blowing what's ahead.

Stable Diffusion Video

Stability AI is best known for stable diffusion and their image generation tools but they recently brought out stable video diffusion Which is freely available, including model code and weights for research and other non commercial purposes. The video duration lasts around two to five seconds. Their frame rate goes up to about 30 frames per second.

And the processing time is under two minutes, which is just super fast compared to some of the other tools on this list. Stability AI has put a lot of time and effort into some of their video generation capabilities, but most importantly, their focus on allowing things to be open access and community sourced enables their models to update and have plug ins added to them in a really quick and optimized way. We're already seeing a lot of Stability's AI image tools integrated into other AI tools and they're then expanded with custom models and I'd expect to see this with the AI video system as well. The actual quality of the video compared to Sora and RunwayML is pretty comparable but it does seem to be a lot quicker on some of the tests that I ran.

Pika Labs

Now the final AI text to video generator on this list is PikaLabs, and it's got some crazy features. Pika is freely accessible via their Discord server, and it works a little bit like Mid Journey. You can upload a 2D image and have it animate, or you can generate a video entirely from text. The short video clips and animations are really cool, and these can be clipped together in video editing software to then build out your own short film.

Pika have also launched their web app, which allows for AI editing of your videos and a more accessible interface, which is just really cool. Now, like all AI models, these are really early versions of what will be possible and video is really hard because the AI needs to generate consistent maps of environments, navigate and interact in those environments, while also building realistic models of human behavior as humans appear in those environments and scenes quite regularly.

We're just scratching the surface of AI text to video. And as we move towards AGI and AI models that can do text, image, and video, we're going to see all of these companies competing and optimizing their text to video AI models to make them as good as possible. And it's going to be really exciting seeing some of the short form and long form video content and films that are created for the world By creative people utilizing some of the AI tools.