So I’ve had this moment before when I first met Open AI Chat GPT…
They just announced SORA
And… HOLY COW…
https://twitter.com/OpenAI/status/1758192957386342435
This model can create full hyper realistic video… from a Text Prompt
4 Likes
There’s also some Stable Diffusion extensions that do stuff like this. You can even make a prompt, define key frames or rough compositions it should follow, and it’ll do it. All running locally on your GPU ** in a few minutes.
It’s neat and the obvious next step after all the still image generators. Though if you have a very specific vision of what you want, you will be frustrated. Ditto if you’re a stickler for details. No guarantees provided that the end product will have a proper number of fingers, toes or teeth. Still, it’s all better than it was even 6 months ago. I also believe video is better at some of this, because it can use previous frames to help with consistency, similar to DLSS and other ML-based super sampling tech is used in games.
** = RTX 3080/4070 or better, with a min 12GB of VRAM. Don’t bother with Radeon cards unless you hate yourself.
Related note: My kids are demanding I take some photos on them this weekend that I can bake into a fine-tuning model (a “LORA” in ML parlance) so they can create images of themselves being cowboys, race car drivers and astronauts.
3 Likes
Yeah one of the videos above showing a texas “cowboy” town. Has quite a few half horses walking around.
1 Like
First thing I noticed is that sometimes people’s knees flex forward while walking. Also look at the first few seconds, there’s people in the middle of the frame who are…like…tiny and moving weirdly. The scale is off and they’re floaty. And the woman in the black coat enters the frame while walking on a roof, before effortlessly stepping onto the path.
The sheer length and consistency of these are impressive though. But while the individual elements are coherent, the whole composition is nonsensical. On top of that, I suspect these were highly cherry picked out of hundreds of test runs. It’s still busted in a lot of ways.
TRUE BELIEVERS on the social medias are claiming Hollywood will be a ghost town in 5 years, replaced by internet randos dumping their scripts into a cloud instance. 
1 Like
I bet someone eventually comes up with an entertaining enough series from their home using these tools.