For several of these examples the details change if something goes out of frame and returns. A second castle is added in the dragon one. The people walking away from the camera change in the train platform one. The Hong Kong moped rider doesn't have a helmet and then he has one.
It's very good at maintaining consistency for things that stay in frame, but I don't know that this points to it developing a strong "world simulation" like they claim.
It's like its simulation model is confined to the renderable area. At the cost of increased computation, it would not be hard to generalize the technique to rendering a sphere centered around the POV to keep some of those artifacts at bay; however, the fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object. For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.
> While it can maintain coherent environments over short sessions, visual details and object permanence can still shift or drift over time—especially as a simulation runs longer, after multiple complex prompt edits, or when navigating throughout the scene. Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are active areas of ongoing research.
If it was actually not hard, they would have made their demo with it. I don't know the technical details, but I expect a product reveal like this to show the thing at its absolute best.
I am not sure you need something so complex. If objects on scene could be tagged with their location, and the camera movement tracked to identify when it brings the same objects back into scene, then the model can use its positional information to validate if the next rendered scene contains all the objects that should be expected.
Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.
Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?
> For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.
I suspect it is, because the 2D -> 3D models I've seen are terrible, but more importantly the 2D -> 3d models are terrible. Noteworthy because I suspect the number of people who have 3D scans laying around is tiny.
What you're describing is vaguely how video games work, to my understanding. There may be interesting parallels if that becomes a technique (like how distant objects are sometimes still rendered, but at a lower resolution/polygon count).
> fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object.
they got this working decently well for video over a year ago, maybe closer to two years ago at this point, time goes fast. so i assume it'll translate to progress in Genie too
I don’t understand why these world models aren’t designed to be backed by an updatable world state. It could probably just lay out the world onetime and reference it as needed vs trying to invent from wholecloth via diffusion. Object persistence!
> It's very good at maintaining consistency for things that stay in frame
I noticed several inconsistencies in things that stayed in frame. See the wall on the bridge in the Van Gogh example as they walk past it to see the other side.
I bet it's configurable and just a matter of how much resource you want to allocate. If it's truly a world model then it seems to me that it's precise benefit is that it can keep track and order in things and they just choose to accept some noise/chaos/randomness/? for the demos when things move out frame? Does that sound sensible? I'm just guessing but don't feel it's a hard limit.
Reading the comments almost everyone seems not impressed at all and points at obvious errors in the samples. Meanwhile it's been just roughly over 3 years since we've seen an AI generated abomination of Will Smith "eating" (or whatever you want to call that act) spaghetti. Yes, AI gets things wrong. You could say the same about LLMs still making obvious errors here and there. To me, personally, this seems super impressive and it's clearly another leap forward. Not perfect, yes. But an improvement.
I'm by no means anti-anything but I think what the comments are getting at is that these "world models" do not have a world model. It's a fundamental qualitative gap that no amount of compute has been able to close, just mask by everything else getting better.
This doesn't show up as much where the verifiable reward is "code compiles and tests pass" but you can still absolutely see it in things like movies, prose, or slide decks. Even frontier models are pretty bad at keeping track of narrative threads even as the technical quality drastically improves.
This begs the question of whether the gap can be closed just with more compute (presumably yes, or at least errors can be minimized beyond human capacity to see them) and is that going to be worth it with current tech (possibly no).
does your model of models admit the possibility of partial models?
the average human cannot vividly recollect a walk through a familiar neighborhood, much less invoke a coherent fantasy of going on such a walk. do they contain a world model?
Must we, every time the limitations of AI are discussed, always imagine some simpleton human to defend it? It's trite and rarely adds anything to the conversation. The point is to judge the model on its merits. If this is a good world model, then it's a good world model. It doesn't matter whether or not a human can recollect every detail about a familiar neighborhood. It's irrelevant
My problem with it is the shift in focus. The original Genie was pitched as a simulator of the world that an agent can interact with to learn so generally that it could be deployed into the real world. This new version looks like a good, controllable video generator. The fact that you're comparing it to the will smith video, rather than a simulator, is indicative of the fact they've strayed from that original goal. I think that's a shame
People are emotionally driven. When emotions drive us to do something we will (unconsciously) find plausible justifications for it without realizing we're doing it.
Anti-AI crowd is anti AI. Maybe they were (mostly) happy with status quo and don't like change. Maybe scared for the future. Or any other reason. Went from "AI can't even tell what 2+2 is", to "AI can't even tell whether these simply connected smooth 4-manifolds are merely homeomorphic or actually diffeomorphic after stabilization" without batting an eye. The goalposts will keep being moved and the dismissal will continue.
It's a huge leap from 3 years ago, but not a huge leap from other current AI models. People who are deep in the AI rabbit hole have seen impressive generative video that has fewer problems (or possibly just better chosen examples).
I agree. This is nothing short of magic -- or perhaps 95% of magic. Imagine yourself 5 years ago, and someone from today going back in time and telling you "hey, check this out, in just 5 years you'll be able to write this short text and a computer will generate this movie from it"...
It looks like they had a hard time with the magic wand, peacock-to-cat example. Despite using a very long prompt that repeatedly stresses that the video should look photorealistic and NOT like a video game, the output video looks like a video game. The model is clearly capable of generating photorealistic humans and animals, so I wonder why it struggled to do so with that particular prompt.
I might not understand what "world model" means, but from my understanding world model is the model trained on real world data, so theoretically having all laws of physics baked in the weights. So perfect world model should not be able to make the video for given prompt, as it is impossible in real world and the only possible way it can appear is in a setting of video game or movie. So this is expected, doesn't mean that is the reason behind this particular generation.
I have no idea if that's the case, but it would be funny if it didn't support negative prompting and they got someone to write the prompt who wasn't aware of that.
Edit: Okay, I read the prompts and they use negative phrasing a lot, so I guess it does work, but boy, are these prompts laborious. Requirements are repeated ad nauseam. This model seems to be quite tone deaf.
When people talk about "photorealism" it's to describe something not real (like video game graphics and artwork) that looks almost like a real photo/video. So you don't say that a real photo is photorealistic, and if you use that word when prompting models you end up making it not realistic.
I think they wanted the output to appear as if it were captured in real life rather than a game engine, but failed to do so which ironically made it one of the less visually realistic examples
Is it surprising? Looking at the demo videos I can absolutely see why it didn't get as much attention. Not that it's not impressive, it is, but these are in different leagues.
> movies are going to become video games and video games are going to become something unimaginably better
God what an annoying thing to say. Suggesting that movies will essentially become games, or that future video games will be unimaginably better (presumably because they are driven by generative world models) betrays such a surface level appreciation for what makes art and culture and all this humanity so beautiful as it is.
The entire point of movies is the deliberate choices of the filmmakers! Movies will not be improved by our ability to choose what the main character looks like or add a style filter. Super Mario 64 is great because it is an expression of the very hardware that constrains it. Inevitable future versions where anything can happen because someone hooked the ROM up to a generative world model won't be better for it.
I agree that this technology is absolutely world changing, and am cautiously eager to see what comes of it, but it just bums me out, perhaps needlessly, that the people at the helm can be so artless, unreflective. Whatever.
yeah some friends of mine in game dev are a bit doom and gloom about all this and I need to keep reminding them that the hard work is making the million decisions that communicate the vision, the code/assets were never really what people were buying your game for.
Even assuming an AI could do literally every technical part of game dev someone (for now) needs to sit there and go "nah this part doesn't feel right" and "that boss 2nd phase opener attack is too long" etc.
until AI can also answer those questions and intuit what is good and bad taste there's still stuff for game devs to do.
"minecraft in skyrim" is cool for about 10 seconds till you think about actually sitting down to play it.
They're getting surprisingly good at identifying why things don't seem right. I came back to an old game project that I had shelved due to the core game loop not feeling right and I decided to just ask Astra to analyze the gameplay and figure out what was wrong and it was able to provide a good direction to take it that also works well with the game mechanics that were already in place.
The new Terra 6.1 is pretty good at doing user testing as well via computer use, going through all the features and noting any bugs or inconsistencies.
It feels like we might be on a trajectory that takes us to a place where someone can just say "I'm bored, make me a new game" and then provide minimal feedback like "this isn't fun, make it better" and get an entertaining game without a lot of iteration. Of course we'll still have games that come from human ideas as well but vibe games seem like they're here to stay.
I'm not so sure, I've made an autobattler and even recently using Astra to come up with new mechanics or cards they're usually always bad as if it doesn't "get it" even if I feed it all the current cards in game and say "like this, these are good because of x and y."
When brainstorming, I usually get it to generate 20 ideas and then I come up with something entirely new based on bad stuff it responded with. I've never been able to actually use its ideas verbatim.
I'm not sure if this is a limitation of the tech or whether it will eventually have "taste" in such nebulous domains as "what is fun."
They still can't write funny jokes despite trillions of dollars of training because even though there's plenty of funny material and discussions on what funny is, I guess it's still kind of a human thing.
Yes, Opus 5.5 can build out quite a bit on its own that is super impressive when you first look at it. But spending just a couple minutes will make you realize dozens of things that are off.
Sam had OpenAI start the Ghibli style image-gen (movies that are specifically loved for their artisanal quality). The man has patternly failed to read the room around culture.
> I agree that this technology is absolutely world changing, and am cautiously eager to see what comes of it, but it just bums me out, perhaps needlessly, that the people at the helm can be so artless, unreflective. Whatever.
This is something your average person can play around with. The filmmakers have been fumbling the ball a lot lately (not all the time, though) and I can't wait to see what rises out of the average populous. Lots of crap expected, for sure, but there will be gems in there that I'm willing to bet will blow away what we've seen in the industry. Kind of how indie game companies are eating AAA's lunch lately.
Make something beautiful, with or without the new tech. You and the rest of us will be better for it. And the investors are damn desperate for something to really resonate with the people who experience it.
We've been living in a world of videogame-like FX movie slop for at least 20 years now, without the help of AI. Try to find insightful reflection of humanity and truth in a Marvel movie... God help you. It's as boring as, well, watching someone else play a video game.
It turns out, some people like that. A lot of people like it, presumably because it's fast and stimulating, and everyone's a sucker for wowing FX even as movie FX have gotten worse and more or less converged with video games.
What's lacking in these movies isn't so much plot, it's character and thoughtfulness. They tell us nothing about ourselves.
From an art history class, long ago, I remember the explanation of how classical Greek culture degraded into depictions if "pathos", visible in changes in the sculpture, and how that heralded more or less the decline of the civilization. It seems obvious to me that we've undergone a similar transition. And the thing is, it's not the technology itself, which could be used to produce meaningful art. It's the lack of thought on the part of the creators and the audience.
> Try to find insightful reflection of humanity and truth in a Marvel movie... God help you. It's as boring as, well, watching someone else play a video game.
It’s so true. The only time I watch one of these movies is when I’m stressed and exhausted and want something so dumb I’ll fall asleep in 20 minutes.
When it comes to these “worlds” that are generated, there’s no there there. I’ve tried various of these and after three minutes I close the browser. Walking around my neighborhood is a million times more interesting. Whereas the creator-constrained and -designed experiences (GTA comes to mind) actually deliver something engaging.
That would be a gotcha if anything else was making its way into theaters/ mass consciousness now. What are the highly anticipated movies? Spider-Man and a remake of some supposed epic from a while back.
This. There is amazing art everywhere right now but somehow it's all slop. Nothing but anecdotes. Meanwhile some friends are in a bedroom studio as we speak creating Aja.
That’s what led me back to books. Character, plot, ideas, coherence. I also love great prose.
You occasionally see thoughtful movies. Many of them are unsurprisingly based on books. The horror genre has become this weird eddy of creativity too with interesting characters and plots and some surprisingly good acting. Of course you’ve historically had innovation in that genre, often ignored because it’s horror and thus low brow by definition.
My take on horror being creative is that it’s like old school punk, pulp novels, zines, underground music. The key there is that these are genres and styles that give you permission to suck, and therefore permission to try stuff.
If it must be high art or it must gross nine figures, you can’t suck. So you can’t innovate.
>Try to find insightful reflection of humanity and truth in a Marvel movie...
I can name several, as well as something profound to be discovered in the execution of the MCU as a whole, but perhaps more salient is why so many people feel the need to tear down the straw man version of Marvel that they have in their minds.
Those are just five separate things. Putting arrows between them in the order that an immersion-seeker might rank them doesn’t mean that it’s actually one thing that became each of the five things, it just means your ability to appreciate those things is limited to “how much can this technology immerse me (or someone else whose money I want)”. This is annoying and artless and leads to claims of “no appreciation for the thing”.
They are all capable of art. Sam Altman saying how things will be in the future of two mediums he doesn’t work in says something very loudly. You just have to choose which thing it is.
Because radio is not the ancestor of video games. Even if you wanted to make the case, it will be painting -> photography -> silent movies... not radio. You can't bunch two different disciplines as related to each other like this even if both of them are art forms.
We still have podcasts but you think radio led to video games?
In the sense that it's another medium for telling a story to a wide audience it did. On the technical side, radio communication contributed to the development of networked communications and is still in use in electronic communications so it also made video games possible. You could also do a deep dive on how mass media in general was influenced by techniques first developed for radio plays e.g. the use and creation of sound effects, changes to scriptwriting, adapting the idea of serialized publications to episodic shows, etc.
This is the trend of available technology, not a ranking of any particular entry. A great radio show will not automatically make a great movie, for instance.
As an example, while Hitchhikers Guide to the Galaxy is available in at least 6 different forms of media, the movie is by far the worst.
The Orville show explores this idea, before LLMs became a thing. Simulators become the replacement for movies, but the simulations are still designed by a person (or alien).
The concept feels more like a video game than a movie in the show, but presumably you could program the simulation to happen around you instead of you moving through it.
Do you think that Super Mario Odyssey is better than Super Mario Bros for the NES? Is Breath of the Wild better than the Legend of Zelda? I mean -- take off your nostalgia-covered glasses and imagine if you handed them both to a kid who had seen neither which one they might be more interested in. Why do you think that is?
Is your contention better graphics are implicitly better? Does this mean movies in color are better than any movie in black and white? Also, are kids an amorphous blob? How should I know what a kid might like better?
I understand but I wouldn't dismiss how consumer choices already contribute to this
Younger people I know, and market to sometimes, literally avoid "plot based media" in favor of brainrot. Akin to what my generation used to call "sooo random"
People catering to it earn a lot. They don't identify as directors, but they are. They don't identify as artists either. They identify as influencers and brands.
You will never be the generation they engage with. I don't expect to base any of my choices in following trends looking for opportunities to be contingent on your affinity for movies and first/third person games.
Even when it comes to spending, they are a powerhouse because parents nationwide and worldwide give them their weekly allowances in platform credits, with no further accountability on how it's spent. There is a large audience already, trends that are full economies within a few months instead of needing to wait an entire 12 to 20 year generation for the audience to have independent purchasing power.
That's what is happening and why someone with even greater perception than me would say things that seem out of touch, but are closer to reality than you'd realize.
“Plot based media” you actually heard someone say this? Like it was a bad thing?
GenZ is culturally destitute. It’s shocking and sad. Every time I think no I must just be old and I don’t know, I go and dig and… very little. There’s just not much there except brain rot and blather and music that’s absolutely derivative of previous music at best. At worst it’s musical brainrot, like “phonk” inspired stuff.
I fear GenA will largely be in the same boat.
It’s scrolling and algorithms. People blast weapons companies and stuff like Palantir or Flock or whatever here, and for rational reasons, but infinite scroll plus engagement algorithms has done more harm to humanity at scale than any of those.
Some movies and some games (or parts of games) really are just for fun. Eg a lot of Bethesda games would improve if you could do a randomly generated storyline on your 4th playthrough, and WoW has 0 storyline to their dailies. It could be kind of fun like a fortune cookie to get a random story. I don't expect art, but it'd be funny to see if it made up a meme-able story.
An awful lot of mainstream movies do have fairly trite plots, with a predictable film style. That's not entirely hate, sometimes I really want to see Michael Bay blow up half of LA. I don't think the next Fast and Furious movie would suffer horribly from AI writing, it's never really been an "art" kind of series anyways.
And they will be so severely targeted to you that you won't be able to identify that they are designed to influence your purchasing or voting decisions. And they will be of production quality rivaling the highest budget films - rendered on your own hardware.
I submit that most people haven't been able to do that for some time now. Advertising wouldn't be such big business if it didn't work on a macro scale.
My 17yo niece shows me Nike ads on her phone, and when I ask her why she’s showing me ads, she gets upset and tells me they’re not ads — they’re web experiences.
What is obvious to you/us is not obvious to most..
I mean, sure, some advertisers might get more sophisticated. The garbagefest of internet advertising won't go anywhere, though. It'll still be as bleak and obnoxious as always, because someone will always accept a buck to let you barrage their visitors' eyes and ears.
The gameplay _becomes_ the ad. If you have a full sash of potions in every slot, they spell "C","O","K","E". The monster-bashing sounds sound like onomatopoetic "Apple!" "Goog!". The armor colors are the same as the NFL teams nearest your locale!
Sounds like the plot from the movie matrix. Only difference between the movie and our real timeline - the ai didn't need to enslave humans, we willingly chose enslavement by ai for ourselves by continuing our google subscription.
It also sounds like Ready Player One, where the real world is so shit, everyone spends all their time in VR, using rented equipment, making nearly everyone a slave to the corporations.
Ultra nerdy point: in the matrix it’s revealed that humans did in fact choose to join it. And the meta point was that most of them still would choose it over reality [1] every day.
[1] reality is also somewhat in question in the films universe
It's from Matrix Reloaded, when the architect explains past failures of the matrix and that they only succeeded when they figured out that they had to offer a choice, even if that choice was subconscious. That nearly everyone accepted the matrix, and the few that did not escaped to Zion. That's the whole point of Neo, to manage the outliers and keep the system going.
At least that's my recollection. I've watched the matrix a few times :)
You are correct. And it's not that they chose for the Matrix to exist in the first place - humanity fought a brutal war against machine rule and lost utterly, and signed away their bodies under duress ("Hand over your flesh, and a new world awaits you. We demand it." -Animatrix) - but, at least according to The Architect, everyone within is presented with a chance to leave, one way or another. Most choose to stay.
It's worth remembering that the Matrix is an allegory for systems of labeling and control (notably, gender), which make life (and specifically life with others) easier, though not necessarily better. The Matrix sustains the large human and machine populations amidst Earth's ecological collapse; people who choose Zion choose a harder life where they are free to be themselves. However, when Zion grows large enough to threaten the system, the system comes after its residents and wipes them out.
One person's freedom is tolerable, even necessary for the system to continue; everyone's freedom leads to everyone's doom. The movies have some weird-but-rational implications.
...Sorry, I lost track of my point, which is that we're already in The Matrix. The Matrix is society.
There was a guy named Cypher who betrayed the humans and cut a deal with the robots to live in the matrix as a rich guy. He fully accepts the steak he eats is not “real” but he enjoys it nonetheless
People already do this in many video games. The simulation isn't perfect, and maybe there's a tipping point where the simulation becomes real enough that it (maybe) supplants "reality". In a sense, this is disintermediation: instead of actually riding horse, I can ride a virtual dragon. Does that make it worse, or problematic? Again, we already have this, for driving race cars, fighting in wars, etc. etc. It's just a matter of fidelity and immersion?
Riding a real horse isn't just about the horse ride, right? It's about the trip to the horse farm. The small but real danger. If you're doing it on some sort of multi-day trip, it's about the memories you make with others, the romantic time spent with your partner at night, or if you're single maybe the hope of meeting a romantic partner. For some maybe it's even about doing things like cleaning the stables, which isn't intrinsically fun, but probably adds to the feeling of bonding with the horses.
(I've never ridden a horse. I'm surely wrong about some of those things)
There's also the "abundance issue" with simulations. Have you ever looked at your unplayed games on Steam, or a folder full of ROMs, and just had a hard time choosing or enjoying anything, because (a) there's so much there, nothing feels special and (b) even when you're enjoying one game, part of you is thinking that perhaps the next game is even better?
If things truly progress to Matrix-style simulation, I guess a lot of that is "solvable" by "just" manipulating/injecting memories. But I think that's how far a simulation would need to go in order to supplant real life experiences entirely.
Or, there's the even darker route, also explored by much scifi -- reality gets so shitty that nobody can really even aspire to an actual horse ride anyway, so simulations are all anybody has. A lot of people already live some kind of version of this anyway due to lack of funds.
This is the worst philosophy course I have ever surveyed. Explain further. Your contention is that, given money people didn’t “earn” they will immediately do drugs?
Yes, but the other three options are AI failing naturally, AI being stopped, or people being unable to participate in the economy without UBI and starving.
Right now we're fully on track for one of several dystopias.
Also, does anyone want movies to become games? I don't. I want movies to be movies. Good movies. With interesting stories, beautiful artistic cinematography, great characters, moving paintings as Lynch said it.
For me, this does nothing. I am attracted to film and gaming for the artistry and deliberate choices made by the authors. A synthetic approach may be stimulating but I find it ultimately void.
We’ve seen a lot of changes since 2016 including the rise of short form video, podcasts, audiobooks, music streaming, substack, even vinyl is up 3x (!)
Instagram Stories were radically different in 2016 than the reels of today. They were far more personalized (less algorithmic public content), slideshow-like updates. Ten second max video clips, with most being just images. Ephemeral as well, where they'd get removed in 24 hours.
And that’s not even close to the short form 90 second micro dramas that are already super popular in China and are ramping up elsewhere. And this category is now dominated by AI
It's still a radically different thing. It wasn't nearly as algorithmic. The interface and engagement was different. And the number of people consuming it all the time is totally different.
If they were really the same, why was there so much uproar over how annoying Shorts was when it was originally added?
How can you honestly think YouTube in 2016 was practically the same as Tik Tok? Why would these other social media companies spend so much trying to change to copy their same strategy if they were all already the same?
Short form content is radically different today than what it was in 2016. It's produced differently. It's presented differently. It's consumed differently. The audience is different. It's pervasiveness in society is different.
Many kids were, sometimes. Also, Vine was still pretty different from modern short form content, being capped at only a few seconds of length. Now most people consume tons of short form content, with many being 30s or more. It's pretty radically different in terms of what most people consume.
Like, movies sometimes have alternate endings. What if they were more choice driven, and replayable. Video games have remarkably dumb ideas that keep persisting, like you can encapsulate all that is “Health” as an integer. Instead games can be an aggregation of entity bound micro-simulation that create emergent scenarios, yet follow the laws of narrative collectively. Games that build themselves. Worlds within worlds kinda jazz.
Maybe you need a mirror. People are literally watching AI generated soap operas that have gone viral. Perhaps it's your lack of understanding that frustrates you
I for one would love to be able to experience Godzilla Minus Zero interactively. I get that it's different from movies. I remember seeing Avatar in 3-D and realizing that James Cameron was directing the audience's attention to a particular part of a scene by the way the 3-D focused, and it was a little off-putting at times. That's a small thing compared to what will be possible in a few years, but still -- being immersed in Avatar will be an amazing experience. I'd also love to see a version of Avatar written by '80s James Cameron -- he was a much better writer back then -- or maybe other, more skillful writers had greater influence on him? In any case, it's inarguable that Terminator, Aliens, The Abyss, and Terminator 2 are all better written than the Avatar films.
More and more off-topic, but this made me wonder: this interactivization would work much better for some works than others. Many comedies are scripted so tightly that I can't imagine any wider world around them (also, on the other side, Memento), while something like Twin Peaks would be perfectly expandable into a world (if you somehow had David Lynch in a datacenter, which sounds oxymoronic).
> I for one would love to be able to experience Godzilla Minus Zero interactively.
So would I. But I don't not want to experience it as a movie. It'd be something separate. Not an upgrade.
Which makes Sam's comment the inane blather of a pitchman. We don't need to bend over and try to steelman marketing dross. He doesn't care if that statement means nothing, why should we?
Oh, I see your point. I didn’t think of “movies becoming video games” as meaning there’s no such thing as a movie anymore. Kind of like the false argument about evolution, “if we can from monkeys why are there still monkeys?”
With VR coming and then going technology has shown itself to hold within it no more immersive capability than a good book. It is not about rendering firepower it is about hitting the right notes.
Llms wont be able to create good games. Consider for a sec that there's like 500 "Great" games ever made, maybe more but not much, in some genres that number is less than 10. There is no training data, and its largely a matter of taste and culture that cant be represented/distilled into vector embeddings.
While i generally agree, some of this tech could be a major force multiplier and lower the barriers to entry.
Even if you have a great game idea, it takes a ton of time and money to even get it off the ground, let alone get it coded correctly and maybe get the reach you want. A LOT of flash in the pan ideas probably fizzled out faster than they needed (most recently Among Us and Fall Guys come to mind) because they were built to ship, not to easily modify.
Tech like this could lower the barrier to smaller and more creative teams or even individuals and get some better competition in the area. Fighting games KNEW that rollback was the way looooooong before AAA bit the bullet and started making it standard, and they still fuck it up (and it probably would've taken longer if COVID didn't force the issue and punish those lagging).
That said 2 of the largest hurdles i've seen in everyone's "magically ideal" game are enemy AI and Plot.
Interesting and well designed enemy AI is still hard, and even games that did it well built the whole game around it (FEAR) and the system is vastly less impressive without that focus (Just Cause). I don't think we're anywhere near modern AI solving the issue of making a good AI system for a game due to design space problems, but it might.
Plot though...that's tricky. Everyone always says how cool it would be for everything to be like Mass Effect or more, with all choices mattering and everything interacting, but someone has to write that. AI is not turning out well though out plot points that don't feel like the equivalent of Bethesda mission creep.
Point of this whole ramble being that in arcadey/simple stuff I think we'll see a lot more great games (and would argue we already were with the growth of the indie scene over the last 15 years thanks to steam and other tools), but there's still going to be entire genres that are just miserable to develop.
I agree. Game design is what makes a good game, not the programming. And I'm actually excited about game designers and artists being able to take their independent ideas and prototype games in a day/week/month/year/whatever.
+1 on fighting games, an interesting trend in that genre right now is that many of the latest titles (realistically everything other than SF6) are having trouble with player retention, while AI-assisted netcode updates for old titles like SF4 and Tekken 6 seem like more compelling products
Or, the lower barrier to entry crowds out the committed indie devs willing to invest years of their life honing their unique artistic vision for a game that might end up being a hit on Steam (among many other failures). The perceived odds of getting some recognition or even a moderately profitable hit go from "slim but possible" to "hopeless".
Because now their competition outnumbers them 1000 to 1, the competition isn't devs or artists but anyone with a phone, who can be from any part of the world who may not even speak the same language. Many to most of these solo "devs" aren't interested in games at all, and just hoping to rack up sales, ads, hype a crypto shitcoin launch or whatever. These slop games all have flashy attention grabbing visuals, slick product pages, five minute promo videos, with the games boasting 500+ hours of gameplay... of which 99% are recycled borderline unplayable trash.
The indie market for lemons stifles indie devs who struggle to find any audience, with modest dividends going to the most deceptive, scammiest solo sellers churning out 10+ titles per day (mostly 1:1 copycats of GTA/AAA games, psuedo-porn games, gambling games, meme/news cycle one-off games, etc). Middlemen like Steam print more money than ever as purchase volume increases while the indie gaming market becomes closer to the mobile games industry today.
The vast majority of the force multiplier value from AI is captured by the handful of AAA conglomerates who follow the tech monopoly strategy of maintaining market share by constant acquisitions and collectively control the most valuable IP while constantly reducing headcount thanks to models like Genie.
But who knows. It just seems like wishful thinking whenever I hear talk about how AI will empower the little guy to take on Goliaths considering the huge amounts of capital driving AI research and products are coming from those very same giants.
Their "good games" is different from our "good games".
To us, games like BG3 or E33 are masterpiece, but let take a look the most profitable games, GTA V, that is their "masterpiece". R* Knows this and high chance GTA 6 will continue this trend, a passable story combined with top notch graphic and physic, which both AI is really good at already
Instead of better it's become unimaginably worse since games can no longer serve as a means of self expression because the second you express anything worthwhile 100 people copy it immediately ... without you. It's like Disney (whose whole business model is copying public domain stories and music and ... but taking out the source and everyone who ever had anything to do with it). Except there's no longer one highly selective company destroying the world's culture one piece at a time, but 10000000 idiots looking to make a quick buck, destroying culture and self-expression like a fire raging in the rainforest.
There will be no more "celeste" games where you join for the platforming but stay for the (politically offensive to all sides) story, because AIs won't discuss the story as it's politically incorrect. Celeste is fundamentally about depression and anxiety, due to being transgender, and is about offending both sides of the argument. First, the main character is transgender, with zero explanation or need for explanation how far that goes, which of course offends conservatives. Second the goal of the main character is to run away from any provided help and professionals (which the director did too) and even under the most extreme of circumstances only reluctantly accept help from a family member that then promptly dies making things "worse".
The Qwen and other Chinese models (and ...) will refuse to discuss being transgender at all, and any western models will shut down any discussion of running away from help hard (because otherwise they get held responsible for the consequences, as has already happened)
(oh and that running away from professional help in psychological issue is actually observed to be a positive thing in almost all cases, in studies. The main way to make psychological problems worse is to let them fester and running away from help is time and time again observed to be far more productive than going to professionals and expecting them to fix things. Because that's an extreme human bias: it can never be (allowed to be) the right reaction to NOT do something about a problem, especially when countless people are paid to "solve" the problem. In other words: if you take people who expressed intent to commit suicide and compare the groups who "get help" and the group that successfully shut down help, less people actually go through with it in the second group, and worse than that, the more intense the help, the better the odds the patient ends up killing themselves. Right to the point that doctors sometimes assist such patients because after 2 years of attempts people live in constant misery and have caused enough medical damage that they will have constant pain or worse the rest of their life. And yet people keep forcing help on patients. Usually in response to an actual suicide where no help was provided. Then more help gets built, and 5 years later we yet again observe ... more suicides. But tell someone with an actual problem to avoid help and you'll be skinned alive for that)
So AI could never make a game like that. Not because it actually can't, or isn't capable, but because there already is a strong political bias, in every model you might use to make it and neither side is going to back down on their policies. The Chinese Communist Party is not suddenly going to change it's mind about transgenderism. And the western models are not suddenly going to openly discuss running away from psychological help, not even as part of a story.
I get that AI anxiety is running high. But, this is doomerism running hard.
Needle in a Haystack Simulator is the new Threes!https://www.theverge.com/22914955/threes-2048-ketchapp-copyc... A very simple game to clone that gets 10,000 headlines because creators have always been anxious about cheap clones. And, anxiety drives the clicks that pay the bills.
Meanwhile, local models are already extremely capable and slowly shrinking their gap vs. frontier models over time. And, it has been long established that the censorship trained into those models can be disabled easily with open-source tools. De-censored models will happily dig into topics that push beyond the limits of sanity, let alone political pearl-clutching.
Besides that. If you want to write about some topic and AI doesn't want to help you, you still have your fingers. Type like a human and you haven't lost anything compared to when Celeste was made.
Hmm. Never heard of non-conservatives thinking Celeste was offensive. It seemed clear to me that a large part of her character arc was learning to accept help when it was offered.
It will lead nowhere. Graphics intensive video games as interactive environments benefit from efficient rendering done by hardware a fraction of the cost of what a neural network requires for the same output. This is a business dead end.
A big chunk of the right side of castle disappeared, including catapult, when camera turned back while flying on dragon. See 1:25 to 1:29. I was not expecting this to happen in a "world model".
The 80's Japanese train station clip also had the conductor speaking clear english in an accent I can't quite place. Slight British tinge to it but not really? Not sure. The ambient audio also makes no sense.
I've seen weird stuff in images that OpenAI sends me in it's promotional emails too. I would think that if you were trying to show off you'd make sure there's none of these obvious AI issues in what you choose to show off with. Maybe they don't really care? "Gemini choose 10 scenes that would be impressive for our announcement and add them to the post."
That entire clip looks like it's straight out of a complete bargain-bin videogame. The weird smoothness of the ride, the pivot in mid air, the wings apparently don't have collision with the poles they fly past.
Definitely wouldn't have been one of my picks to show off my fancy new model.
Oh yeah, don't get me wrong "bargain-bin videogame" is a wild step up from "accidental amorphous horrors that defy description". It just jumped out to me as looking unusually bad compared to the other samples they provided. The stained glass deer, for example, was awesome.
There's also a whole extra bonus castle that appears to the left when the camera looks left around 0:22. Which is amusing both because it wasn't there on the initial approach, and also because nobody puts two castles each with their own distinct curtain walls right next to each other like that.
The oddest thing about this, to me at least, is that they pick that as their showcase. Did they themselves not notice? Or did they think their potential customers wouldn’t? Not sure really which of the two is worse.
I am disappointed mainly because current video models are way more consistent these days. This being a new release, I was expecting more than few seconds of consistency in the scenes.
Cool demos, but does anybody else find it troubling that companies keep claiming this tech can train robots? It’s so obviously not representative of the real world.
> Cool demos, but does anybody else find it troubling that companies keep claiming this tech can train robots? It’s so obviously not representative of the real world.
A proprietary version of this technology is already training Waymo's robot cars. Waymo blogged about their DeepMind-provided world model a year ago.
> It’s so obviously not representative of the real world.
That entirely depends on the use-case and level of granularity needed, doesn't it?
Apologies for the incidindiary wording of my initial concluding question (which I immediately edited to be less flamey, and hopefully more cogent).
> Isn’t the Waymo version used for some small portion of the dataset to cover edge cases that they have no real data for?
The usecases in the blog post[1] paint a much more expansive picture of how Wayne used Genie 3: including enabling transfer learning to convert single-camera dashcam footage into simulated full-take sensor data (multi-camera and Lidar), which is extremely valuable.
> My basis is that I’m a real human and these videos look far more like bad video games than the real life I’ve experienced
1:1 fidelity isn't required for a lot of training tasks; for instance, computer vision can do a lot with 32x32 resolution images, which is trash quality to humans. I think the potential customers for this technology will constrain the model to fit their use cases.
And warfare of course, the stuff they won’t openly talk about but we might find out about months or years down the line when it’s used to destroy another school or hospital with people inside.
Subreddits where SolidGoldMagikarp was counting to a million in comments were used to train a machine that then writes web pages that more machines are trained on. The sheer volume of data with sufficient verisimilitude generated by these models may well work to train robots. It might not, but I think it might well work. The idea that things have to be high-fidelity high-quality to yield good results has clearly not held up in other domains.
This stands out to me too, it seems to be representative of physics the way a video game is representative of physics, likely less so given the non-determinism
When your training simulation fails to reproduce certain features of the real-world environment, your control model is likely to have trouble with them, so more realistic simulations are helpful, but even fairly unrealistic training simulations are pretty useful.
In general, training neural networks to control robots is going to require simulations, ideally differentiable simulations. Even if they aren't differentiable, though, you can SGD your simulated training runs across a data center in a way that you just can't with training runs on Roombas.
I think its a useful exercise to try and name a cultural artifact (picture, video, game, music, book, etc) that was improved, or made substantially possible, by this technology. Ideally something that has moved you or provoked a lasting response.
Conversely, try to think of examples where Generative AI has betrayed a lack of creativity or care from the creator, and thus left a sour or ambivalent or impression.
I find the former impossibly hard, and the latter depressingly easy. Do others feel the same?
There are tons of cool, creative, funny and wow stuff being done with AI. A third of my Instagram feed is AI creators and some of the stuff they do is mind blowing, but even when it's not super impressive, some of it is just so creative and well done. Saiyenfeld is amazing (DBZ Seinfeld mashup by grickusai). It's taking a medium that is familiar, putting a spin on it and doing it well. You can do things that you couldn't do otherwise. People with interesting concepts can finally get them out of their head into something really interesting. People are so doomer and depressed, but they purposely don't see the stuff that is just fucking fun. I spend all day with a 5090 and courses and I still can't make stuff as good, creative, cool or funny as these people who actually have those talents and they are just fuddling through with token services.
If you want to get inspired go to civitai and look through the top stuff and be wowed. Maybe they didn't draw it, but they figured out how to make it happen and when you look at enough generated stuff you can tell the average from the holy shit. It's easy to get inspired by what people can come up with and want to do something yourself, which is a positive impact I'd say.
By Genie specifically? Or by generative AI in general?
If you mean world models like Genie,I don't know. If you mean the latter, I have definitely used software that I have enjoyed that I do not believe would have been possible without generative AI.
Fluxer is the first Discord competitor that when you show to someone they can't say "well Discord does X better", and it was largely made using AI.
Herdr is designed for AI usage, but I actually really like it even for just normal terminal multiplexing. It was surely made with a lot of AI.
I have seen people make incredible things at my work that have actually solved multi-year long pain points that we've never had the resources to address before.
I have used AI to patch and revive a bunch of small services like chat bots or web services and such that I had made and had decayed into failure over the years.
Sure, it's easy to point at people using AI as a megaphone for their lack of care. But I don't think it's impossibly hard to find people doing good things with it either.
Software might be the exception, but I think some skepticism is still warranted.
When software is hard there's a kind of filter; a good piece of software means a human/team behind it with grit and care and user empathy. You can have some* confidence in the continued stability and maintenance of the software. (*relatively speaking, and with huge error bars).
Without knowing the details of Fluxer, if someone tells me about a vibecoded Discord clone, my immediate thoughts are like:
- how much thought and care went into this?
- how much work from Discord has been cannibalized?
- will the authors be around next week to keep maintaining and making improvements and thinking deeply about their users?
I have also seen a lot of fantastic demos and stuff with AI coding. I use AI coding all day. I just think there's a weird gap still, where code and images and text are all "cheap", but none of this has translated into anything I personally use or care about.
They're not really promising to do that. It's not even clear this model will be meaningfully commercialized. The key line is "Understanding and simulating the world is a crucial aspect of building AGI". This model will be used for research and robotics, and maybe the training of further models.
Without getting into a larger argument about whether this is the correct approach to building AGI, I think its reasonable to ask people to engage with how this technology gets used in practice, besides its nominal purpose of "building AGI".
The predominant way people engage with this technology right now is through slop content in their instagram feeds.
(Btw I say all this as someone that has worked on generative models for 10 years).
Throw a gifted storyteller at any medium and they will produce something worth paying attention to. AI generated content has not had that moment yet. I'm not sure it will ever, but it's worth thinking about.
It took the medium of film ~50 years to find Hitchock and ~75 to find Spielberg.
Looks cool, but it's very hard to get me excited about yet another proprietary model for which absolutely zero technical details are released (not even the number of parameters), which is safetymaxxed to the limit, and whose usage quota will (most likely) be so limited that it'll be effectively useless for anyone who doesn't work at Google or doesn't have infinite money.
Most cutting edge technologies through history have been 1) somewhat opaque and 2) narrow/exclusive in their deployment. Historically speaking, that's virtually synonymous with "cutting edge technology".
Whining that I, a random Internet user with limited budget, usecases that the researchers don't care about, and no connection to the researchers, don't get technical details (??) or access (??) on day one is just profoundly goofy shit.
Obviously being able to use these things right away would be more exciting. But to be unable to get excited by advancements we can't immediately use is... quite entitled.
The sole purpose of a press release is to generate positive interest. It is not entitled for the reader to expect the time they have invested in reading it to be worthwhile.
> erhmygord I can't even buy this car right this second for nearly nothing and use it to drive a thousand miles per hour through a kindergarten parking lot?!?!
It’s not an exciting advancement if it’s just the AI equivalent of google wave. It’s totally reasonable to be skeptical of marketing and want to try something for yourself.
Google wave was eventually released to the public but initially it required an invitation. So if you had a friend at google you could get in but if not you were out of luck. This killed early momentum and when they eventually opened up the product was dead on arrival.
> But to be unable to get excited by advancements we can't immediately use is... quite entitled.
Well, I didn't go knocking on Google's door asking them to give me access to their fancy models. It's them who came up to us saying "hey, look at the amazing model we have here; too bad we won't tell you anything about how it works nor even let you access it". Entitled you say? Then maybe don't dangle a carrot right before my face, and then complain that I would like to eat it?
> "Don't talk publicly about things I personally cannot use right now" is peak entitlement, yes.
I never said anything like this? My point wasn't that "I can't use it, therefore it's bad"; my point was that as-is there's pretty much nothing exciting about this. Just because this exists inside Google doesn't really mean or do anything; it doesn't meaningfully advance the state-of-art precisely because nothing about this is public.
Do you remember Sora[1]? Announced in pretty much exactly the same way -- grandiose webpage with fancy demos, zero technical details, limited availability and then... remind me what happened? Oh, right, they completely shut it down, and in the end it didn't "advance" squat. Might as well just never existed. So remind me why exactly I should be excited about this one here...?
It's just a lame gimmick, this is not 2024 anymore, people have started to get sick of this AI thingie, especially when it comes to video/image generation. That Simon guy with his pelican is the exception that can only be seen on tech forums like this one.
As to why US (and probably also some Chinese) companies have started pouring hundreds of billions into AI-image generation is beyond my comprehension, but on the other hand I'm no economic genius.
If I can’t realistically use it then I don’t care, especially when there’s no independent proof yet it works this well, given a cursory glance. If it trickles down into future developments I can use then cool, but it likely won’t because they want a moat. You don’t have to shield the poor defenseless multi-billion dollar megacorp from the genuine apathy they inspire.
If this is the best “frontier” they can achieve, then it’s no wonder they haven’t done anything groundbreaking in the space yet that produces practical results.
We should distill everything else we can into open weights.
These are exciting for robotics and industrial perhaps, but they have no consumer use case until the lid is blown off and consumers can do whatever they want with them.
So much societal progress is behind held up because the leading labs want to keep a stranglehold on the tech. Imagine if every PhD or hacker had access to the raw weights.
The general history of new technologies is that they're available for a narrow set of circumstances and then they proliferate. Can't believe that needs to be said on "hackernews"
Except that most technology was built in a way that didn't require all of the humanity knowledge and labor to be sequestered and then regurgitated back as slop that we need to pay as a subscription from what essentially is a rent seeking mafia.
I mean this doesn't seem very insane? Basic info about the model, the ability to actually use it. Google in particular is awful about getting access to their models from the UX side: I don't even bother trying them out unless it's via openrouter.
Yeah, I want that too, including the weights and training material. But I am still exited by what one could potentially do with it, by looking at the prompts and generated videos, even if it is cherry picked.
I’ve decide that the best (and maybe only) way for me to interact with google AI models, is on the search results
It’s where I get the most value, and my expectations are very low, no attachment if it goes away
Of course they are mining me for data anyway, but they were doing that before already (and I do have a Gmail account, although I’ve almost completely stopped using Chrome - btw, it’s crazy that you can’t add plugins/extensions to Chrome on Guest mode)
That touches on the core reason not to get excited about these Google releases: They release something cool and then don't iterate on it. Instead they go off building another cool demo instead.
Gah. Go iterate. Listen to customers, adapt, iterate some more etc etc. That's how products are made.
This is going to transform how we create games, movie content, and interact with virtual worlds, as well as those born from our imagination. You have control, like a simulator where you can close your eyes and immerse yourself in artificial worlds that feel incredibly real. It’s crazy how fast things are changing.
In the clip with a marble bust dropped on a watermelon, at this point https://imgur.com/a/IHLnkUs the bust should've toppled over, if the physics were properly modeled.
This is a reflection of a complaint I frequently make about so called "world models". It's more accurate to call them video game models. They don't simulate the world, they simulate video games. For that reason a common issue with them is that the physics are video game physics rather than real world physics.
I'm sure you're probably right about video games being used in training and, as a result, the models reflecting video game physics: No air resistance, no static/dynamic friction curve, etc.
However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine.
I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal.
Yes this is right. Something about the way it works under the hood means it wants: 1. bust falls 2. it crushes the watermelon 3. it teeters 4. it settles on the plinth. It is sort of interpolating between these states, which it feels are a realistic sequence of events, not actually attempting to model what would happen in a real "world".
Which is fine for a video game. Not at all fine if you want to train robots with it. Trying to combine both use cases into a single model is doomed to failure IMO.
I would not be so opinionated, modern coding agents derive from chatbot technology. Very different problem domain. They share language but language is universal so that doesn't say much. Then they figured out that the pretraining is a good starting point for RLVR. Who's to say a more cinematic imagination can't be a good starting point for a world model? It doesn't need to be the whole solution.
People are arguing if the marble bust should've toppled over or not, completely missing the fact that marble is brittle and the bust would have surely cracked
How often do things in the real world not behave exactly as you expected them to?
I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events.
We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff.
You're missing the forest for the trees. The less than amazing physics isn't the point.
When you have a conversation, you have an idea in your mind of how the conversation is going to go. You predict what the other person is going to say before they say it.
When you are driving, you predict what the other drivers are going to do, you predict what signs say, how long you have until the green. If you see two red lights attached to the back of a car flash on, you may predict that the car is about to slow down and you start slowing down before their speed has changed.
Forget video games and movies. I believe these models will be that inner predictive model for robots.
> How often do things in the real world not behave exactly as you expected them to?
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
Your expectation is the point, not the physics. I'm not arguing that their model has an amazing grasp of physics. I'm arguing that this can be used as a predictive internal model for a robot.
I agree but I feel like the exaggerated physics were intentional and fitting with the video theme. If they had wanted it to topple, they wouldn't have posted the video.
That's the thing about machine learning over time, and the big hole in the whole world model idea. Time and time again we make simulations and train models and we think that loop will work. It never does.
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
It's where (ex-)"AI luminaries" go to die. It's a way to make glacial, slow progress in the correct direction while never actually getting anywhere.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone (some model) who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
My point was that there is also the possibility of AI intelligence improving to the point that there isn't a difference anymore. That whatever beats the next set of benchmarks (which of course won't be the simulator or world model) is so smart it is both a better solution to the problem the world simulator poses and a better world simulator all in one and a better tool for creatives to build games or content.
This has already happened to other ML fields. For example, there's not much point anymore to ASR models (just feed audio into LLMs) and the space that TTS models occupy is fast disappearing. Oh and it's not just that Qwen 3.6 is a good ASR in addition to being good at coding mario. Doing things this way, not using an ASR model is fundamentally more expressive. A whole bunch of companies started up on ASR + TTS "towers" (lots of models, Pipecat style) as they're called, and they're dead companies walking. Their tech has been surpassed by just bog-standard generic LLMs in a way that they will never get the upper hand again.
A chain of models approach, a separate ASR, will never do this: "you said you agree but I can hear you're sarcastic, so I'll explain again". Also getting a conversational tower to switch languages when the user does that (or when it makes sense in a language course) is not quite impossible but very, very hard. And yet feeding audio directly into Qwen will almost magically solve these problems better than I will ever do it (lightly finetuned). It's fundamentally better in every way (except perhaps speed, of course. Although arguably such a tower needs an LLM too, it'll just have way less tokens)
This point has long been passed for image recognition. Feeding a photo or screenshot into a full LLM along with a request on what to do beats the crap out of any OCR, image recognition, ... model every time these days.
And now there's also hints of world models just being bypassed. Just giving a finetuned model the last 50 video frames (surprisingly, that fits in context, especially at reduced resolution), movement statistics, motor power, status, ... of a robot and asking what to do actually works (with heavy finetuning, but I think it'll start working without). Well ... AND then distilling that knowledge into a much smaller model because B200 racks are more than a bit to heavy for a logistics robot to carry around. But we'll get over that.
Its easy to see what works in hindsight, but in reality with this sort of research you have to find an approach that’s working and keep pushing it and see where it goes.
The results so far from genie have been stunning, and its not obvious to me that another approach is guaranteed to end up better for all use cases.
Gotta love the kraken destroying the city in the background of the Hong Kong lovers motorcycle scene despite nothing in the prompt about monsters. Filmmakers are still safe.
But remember, today is the worst the models will ever be. I agree they are still safe though... they will probably use AI to make even more films faster and more creative since it enables them to do more.
In the man walking through the dressing room scene, the camera starts behind the man. And you can see that the left half of his body is wearing a grey suit... but the right half is wearing a grey dress. :O
Isn't that a simpler explanation of reality than the zillions of particles all existing simultaneously in this massive universe?
Some sort of simple primal generative AI construct that just feeds a conscious experience for one. You. Nothing else is real. No one else is real.
Something like that would be an infinitely simpler, than running the physics and conscious experiences of this unimaginably large universe we think we're in.
Simplicity is only relevant to humans (and other animals that evolved into trying to forecast their survival). Most of the universe seems to prefer chaos. So no, from a universal perspective, there is no reason for reality to be simple or elegant.
Not that I have zero trust in anything AI related that Google demos, but the first video example on the right, the model completely ignores inputs towards the end.
With Anthropic going public, I expect a lot of model releases from the competition signalling to the market. "We are still around, please leave some money for us too"
Google is already public, has a larger more normie customer base, many products to integrate AI in with established enterprise accounts in the millions... For some reason theyre not valued at 50:1 revenue... I dont think all that many people are going to be that excited for Anthropic as a business, Its an escape hatch, using the public as exit liquidity while the founders can still make a 100bil.
If the checker clip is meant to be used to develop a robot to play checkers, then the physics is not remotely good enough. When the robot takes the first piece, the piece looks like it's moving up from the board, then suddenly doesn't. The last move also isn't valid checkers, unless I'm missing something -- the robot offers its piece to be eaten and the human doesn't take it.
A few years from now, Lydia in Skyrim will have a fully consistent personality with dynamic dialogue and full contextual memories of all our adventures together in Skyrim.
A visual makeover is great but what happens when millions of people fall in love with virtual characters who seem to deserve that love?
Question is if we even want that. Video games often are escapism, we want an illusion of reality but not actual reality. I mean there definitely is a market for that, but I don't think it's for everyone.
There was a comic book about comic books I read when I was young. I believe it was "Understanding Comics: The Invisible Art" - part of it dealt with what is gained and lost with realism.
In a nutshell, (with my memory of 30-40 years), the more and more realistic a scene becomes, the less the brain applies it to ourself. It becomes this definite place, definite person, rather than ourself or our surroundings projected into other characters.
All these scenes, that I guess are amazing, they all are creepy to me.
That actually sounds implementable even now! Except the fully consistent personality part. And maybe that's a good thing. Falling in love with beings without physical presence sounds agonizing; probably better spent on real people.
I imagine you could implement the fully consistent personality by putting a recurring dream about the player, followed by waking up, in Lydia's context at startup.
I don't know if I'm missing the point, but Genie 4 almost looks worse than Genie 3 in terms of player control? It doesn't even seem to respond to inputs, it just looks like they took some AI generated videos and put the key widgets on top.
Astonishing to me that a company like Google would be willing to publish such a terrible looking product demo. Interactive, but with completely made-up inputs that are ignored?
I think you're exactly right. This was already a problem with Genie 3, but they've appear to have gone all in on video generation over controllability.
The original Genie was a great piece of work with a real sense of purpose - to enable agents to learn about the real world in the way Dreamer or Schmidhuber's paper sketched. But now most of the people that drove that mission have left Deepmind (Jack Parker-Holder, Tim Rocktaschel, Demis) and it's become this instead.
another POV is, what even are these experiences? whom are they for? it would take Google 10 minutes to hire all the best game developers for Android, and cost them like $2m, which is peanuts, and give them fancy Google Research titles. but instead, they're trying to get 100 google PMs, who don't give their kids screentime and who don't play any games the objective of making a product for Bart Simpsons. like these people don't realize that being good parents disqualifies them from making immersive first person shooter, "narrative" experiences.
> what even are these experiences? whom are they for?
One possible use for these is to facilitate lab -> this -> real world for robot models. Basically better than lab / algorithmically designed environments and cheaper / easier to train on than real life. Something that "looks" like real life and can scale, to apply RL for robotics to.
The more parsimonious explanation is that 100 PMs are even less qualified to make surgical robots, but at least they can download a video game and play it for 5 minutes. There's a great scene in You Can See Everything where Billy Evans is watching bio presentations on 1.5x speed, which obviously, makes him look like a huge moron - "knowing enough" bio is very bearish.
Anyway, let's take your theory to its logical conclusion: the product development between real and simulated soft tissue behavior is much bigger than Exponential Time being a hypothesis and being proven. There isn't some effective acceleraltruist path here. Dario Amodei was a biotech dropout. The thing you are betting on which Google is not doing anyway, maybe they will destroy the descriptive single cell sequencing paper mill, but they're not going to be able to simulate the fat inside a BMI 30 abuela undergoing a gallbladder removal.
Why hire crew or actors when you could have them be fully AI? Why film a commercial with fluffed, oil dusted, well-lit hamburgers when you could fully render something? Why actually have a vehicle capable of off-roading when you could just AI-render what it looks like off-roading (or vertically taking off, for eg)?
It's a dystopian business use case that I hope will get obliterated in the near future as the generations that believe in this stuff and are fascinated by it age out of the population.
I don't know. With each new release of these models, they compare side-by-side how they are better than the previous ones, but at the same time I feel less impressed by each new release. Somehow they manage to be a seemingly big step forward but at the same time I feel I have seen so much AI that I just look at the common failure modes and they are all still there.
It's easy for them to show demos of abstract things because they don't necessarily have to follow real-world rules, but anything modeled after real world falls short of expectations.
I would like to see how it breaks, don't spend all your focus on how nice it looks with specially curated shots. At least give a few examples of what happens when it breaks from running too long. They mention that it can fail under "Understanding current limitations" but don't show any examples. Especially since they mention the cases of safety and responsible use, it seems quite relevant to help prepare people by showing them some examples of how it can break and so you an learn to spot it.
Again, very impressive from a technical point. But language is so flawed to do anything creative. It just turns out all looking the same and kind of boring
"controlling" some kid-height character. the button-pressing seems almost frustrated with the lack of response, long holds with no (or opposite) change and sometimes mashing. are they supposed to be buttons a person pressed to direct it? or did it create those too?
The non-coherence of the world is a pretty solid tell. Any time the camera pans away and then back, the scene changes. The model doesn't remember what used to be there dozens of frames ago and so it invents something new.
Humans interact with their environment by subconsciously simulating it and predicting what will happen in the next few moments. This ability gives us the capacity to work more efficiently and reliably in any environment. These world models will serve a similar purpose to the AI and robotic systems of tomorrow. This is a foundational piece.
We don't have to fall off a cliff to learn to drive on a mountain road, we can imagine the consequences for cheaper. If we didn't have imagination life would be much more dangerous.
Podcasting is going through the thousand cheap ads with the obnoxiously inserted interstitial ads. What they have accomplished is me unsubscribing to the worst offending podcast and beginning the journey of ad blocking on podcast with the help of whisper and LLMs. Also now I hate Shopify with the heat of a 1000 burning suns due to their obnoxious ads and the insane rate they get played.
So yea that one expensive ad would probably have a positive impact compared to a tidal wave of slop. Quality > Quantity after all.
The authors could still design their prompts to steer players towards predefined developments and situations. Anyway, the possibilities of something like this are infinite, the only constraint will be the cost of the compute.
A nit, but given that this is from a major corporation, and LLMs are supposed to be em-dash-happy, it's surprising (disappointing?) that the announcement uses:
> audio-like music
That's a hyphen. What's "audio-like music" -- music that is almost sound but not quite? Music made from raw audio? :-)
Yeah, I think that sentence was supposed to be: Genie 4 is natively multimodal, enabling audio--like music, ambient noises, and interactive dialogue--to be simultaneously generated with the world.
Agreed that hyphens should be used to indicate compound words, but you're saying that "audio-like" is an intended compound word? Then it's an adjective, modifying "music"? What would that mean?
I think the disagreement here is not whether that’s how you should use hyphens but whether that’s what was intended at all. The term audio-like music makes no sense.
I think you're fundamentally missing the point being made here.
Yes, "-like" is commonly used as a suffix, in which case a hyphen is correct. But describing music as "audio-like" is clearly nonsense (music isn't "like" audio, it is audio), so that's not the sense in which it's being used here.
For clarity, the full sentence is:
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
Surprised that "do you really think we're going to make it?" scene made the cut of the first video, her expression and tone is totally out of sync with the scene and demonstrates the problem with almost every AI gen video I've come across when it comes to capturing humanity.
The painting->interactive scene looks astonishing though
And their most impactful application will be training smaller models for military drones to detect, target and kill humans. sigh...
Every time I'm about to be excited about AI the cynic in me slaps me back to reality. I imagine the endless possibilities of AI for disabled people, then realize that it's going to leave millions of people disabled instead.
It's hard for me to understand Google's redefinition of "world models" (basically: a txt2videogame modality for LLMs) as anything other than an attempt to bury Yann Lecun's definition of "world models", which includes an implicit bear case against LLMs and thus Google's stock price.
it blows my mind that people actually want this. I couldn't be less interested in it.
I read a book, watch a movie, and play a game, to get the point of view and creative decisions by the people who make them. I don't want shadowy projections of my own mind and whims creating the experience. I can conceptually understand why some people may _think_ they want a "genie" that just grants every wish, but my suspicion is that much like other slop deluging us, they will quickly see that just getting whatever you want doesn't make interesting or good
Google aren't the only ones with this tech. You can try out a 24/7 rolling set of video feeds for yourself at https://fal.live
An LLM generates four alternative prompts for what could happen next and there's a real time vote running for each option. It's trippy, like watching someone's dream.
Can someone ELI5 how this "Leads to AGI"? I can kind of understand some sort of "theory of mind" going on here, but how would having a complex world model enable, say, an AGI that could file my taxes for me?
AGI is a loaded phrase these days. But if we focus on how it will improve the quality of training data for LLMs, I can prob talk about that.
Right now LLMs only have text, images, and videos to train from. Anything it understands about physics is from an abstract level. It's like learning how to calculate throwing a baseball before you've ever even thrown a baseball. Imagine having to learn force = mass x acceleration before you've even picked up a ball.
What this world generation Google is doing appears to be something Nvidia has spoken about too. Letting LLMs/AI learn from a world physics simulation. Meaning LLMs having the opportunity to "experience" physics, the same way you intuitively know how to throw a ball after attempting a few times yourself.
If an LLM can better understand how our real world works the quality of its output improves -- theoretically.
As far as your taxes go, you can create an intelligent workflow right now to do that for you. Don't need AGI for that.
So world models capture inherent complexity of physical reality to help teach AI about hidden laws which probably helps train both LLMs and multimodal models used for autonomous vehicles and robots? And AMD acquiring World Labs is a move to outdo Nvidia’s work in this space. And Google probably sees this as a gateway to a new platform of autonomous vehicles and humanoids that they could rent out as a CSP? Competing with Nvidia, Tesla and complementing Waymo. Am I getting that right?
That all sounds right to me, but apparently they can also be used for simply generating videos, or "unlocking the creation of new interactive experiences"... which both seem like competing goals with everything you just mentioned.
If only there was some way we could create some sort of arrangement of 3D models and place a virtual camera within the arrangement. We could introduce some physics simulation and make the position of the virtual camera move and render what it sees to the user.
If such a system were possible, it seems like it would be far better at consistency and realism, since it could be fine tuned and hand crafted to best represent the desired world. It could be used to make interactive experiences, games, perhaps even feature films!
Honestly MiniMax H3 looks better than this at most of the things I've been able to generate locally. These look super AAA game like - movement is uncanny and feels like it was trained on GTA3 or something like that - is this just me?
Why do we need this? No thank you. We don't need flying cars and more vendor lock in. We need to serve the current google within constraints. That should be driving direction. Being a better ecosystem member. Another fascist product from google.
I am again so disappointed with the discourse about AI in these threads. It rarely if ever focuses on anything remotely technical. People just trot out their agenda and their opinions (much like what I am doing now). We are building civilization level tools and the most technical analysis I see is that the blog post announcing it was written by an LLM (it wasn't) or that an ogre riding a unicorn into the clouds of Venus was missing a claw (wth?).
Yah, meta is discouraged here but why are we as a community putting up with this level of meta disruption? I guess I'm supposed to adapt to the new community but there's nothing there there.
I will stick with the more technical posts here and avoid anything to do with a new AI model and the complete and shocking thoughtlessness of the commenters (most of them).
This is part of analyzing model output against others and known capabilities. The Orge riding a bike is just a QA test. Maybe you can work on civilization level testing tools.
I didn't see any comments related to that - now I understand. Your reply does not seem to be in good faith, then you pick on the poster not the message.
Why are they plowing money into this garbage? I don't understand it. I guess it is because Google is lagging behind in the AI race but the roi of this bs must be close to zero. Clowns.
So if those genie 4 characters have all individual chain of thought reasoning and persistent retrospection / recollection, how do these human characters know they are not living in a simulation made by biological humans? Not sure you understand what i'm getting at. Some ethical questions do arise.
It could be even weirder. We might be playing video games that are essentially built from the ground up by and for the individual gamer, such that the concept of a "game", let alone the same game, doesn't even make sense. Same for what we have called movies and TV for the last 100 years.
You can already go and generate a book to read or a song to listen to, no one does this, I don't know why you expect it to be different with video games. People want to consume art, they don't want to make it.
You can't just remove the societal/cultural aspects of art without neutering its emotional appeal and impact. Ripping all humanity of out movies/video games is not going to result in higher quality art.
Hi, I always loved watching Pixar movies, partly because I knew that what took them days to render (in their literal data center) would eventually be done in real-time on my desktop.
In the same way, seeing that something is possible gives hope to the people who are making those open models that you want.
What the fuck are the little wasd/arrow keys supposed to mean? They have no relationship to the movement of the character or the camera. It's just flashing buttons.
But to what end? How does this help humanity? How does this help us encounter truth and beauty? Whether you look at this from a utilitarian angle or from a transcendental one, it is a collosal waste of time, energy, and money.
What is this for? I cannot understand these models. They are very much polluting the internet with nonsense and use capacity that people and businesses need for actual work.
If these are the cause of why I have to pay 2-4x I used to for a computer, then Google should go fuck themselves.
The volume of content on YouTube is enormous, I’m kind of surprised they offer auto captions at all given that they have little to no competition in the space.
Ok but like, surely the Deepmind team could sneeze and get flawless captions for this video. Why rely on the default YouTube captions instead of just attaching correct ones? It seems silly.
Yeah if they cared they could create their own captions for sure. They probably didn’t even think about it. Maybe some intern will do it after reading this thread
They're not proof-reading the blog posts either. The second sentence:
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
(Emphasis mine). Like, yeah, I'd sure hope the music is similar to audio.
I wonder if this was a de-LLMese pass gone wrong, and the original was "enabling audio -- like music, ambient noises, and interactive dialogue -- to be simultaneously generated with the world.
Looks like we are inching closer to on the fly open world game creation soon. While the updates excite me as a senior data scientist but as a human they all just look scary. Just me?
As I said before, World models will become simulation engines of AAA games where there is no need to release the actual game artifact and even if the game exists as a binary artifact, it will never be released and will be available on their hosted cloud back-end, then simulated and streamed back to you.
So, Stadia + World Models = New generation of DRM for games.
Why? Why would a non-deterministic engine be better for a game when it can be instead structured and made deterministic and then more effectively distributed to consumer hardware?
Game industry is still struggling to find a way to incorporate regular LLMs without suffering catastrophic edge cases. A long way to go before these can do more than fancy tech demo, I think.
> We'll be riding the huge wave of indie titles that are incoming.
The problem will be separating the wheat (people with good stories and concepts enabled by Ai) from the chaff (people with no taste slopping out trash with Ai)
None of this makes sense. Think of the economics involved here: is it cheaper to run tens/hundreds of thousands of GPU clusters to simulate a unique game world for millions of players, or would it be cheaper to just write the god damn code once, and then host it for download, or to allow them to be streamed?
Stadia was pretty good, be it OnLive was similar quality in like 2010. It even had the best version of Cyberpunk 2077 when that launched. But the model was free version of the service for a limited resolution AND you have to buy games there OR a subscription based option and you still have to buy games there.
That made no sense and wasn't a viable business model.
I am sure the are other typical Google fumbles that would have meant they were not the victor; but I am sure an equivalent service would be mainstream today if AI had not dramatically changed the economics.
[also Google bought DeepMind in 2014; 'way beforehand' is and odd way to look at it]
I disagree, dropping below 60 FPS will be noticed and turn players away, you are not getting that with good input handling even today, it has nothing to do with GPUs and everything to do with networking
> beforehand...
is about the craze, the ChatGPT moment and beyond, when GPUs demand went exponential, not when various types of Ai algorithms were available
sure... despite all the other people who used it and are saying otherwise, and the fact that google totally fumbled in the Ai race as well as built custom silicon for Ai...
... history is helpful ... google didn't - and still doesn't - understand gamers, same for all the other Ai hucksters, gamers are among the most anti-Ai
And nobody will play that trash, and the world won't change.
Also, lmao, they're already trying to nickel and dime you and sell you microtransactions that took 5 minutes to make for 10 bucks. If you think they're going to pay the costs for running a world model for every single person, lol.
No it doesn't, not even close. You are looking at a video represented as pixels changing on a flat screen. None of it actually exists. You exist, the computer or phone in front of you is composed of mass and energy, and absolutely none of it is algorithmic. World models, as complex as they are, are still just pushing numbers around on paper.
Even if we can show that it is technically possible to model a universe like ours to the degree that our current perception of reality suggests exists, there are three other big factors that would need to be weighed:
Is it feasible? What Dyson sphere level of power would be needed to maintain such a simulation? Is that level of energy available to whomever is running the simulation?
Is there a positive ROI? What does this advanced civilization get out of running such an expensive simulation?
Is it moral? The simulation involves sentient creatures who are capable of deeply contemplating their own existence, capable of feeling rapturous pleasure and indescribable pain and loss, and ostensibly capable of making moral choices that range from heroic to depraved. Doesn't there come a point where the beings in a simulation are so close to the beings in "base reality" that the same moral principles apply? If you press a button that kicks off a simulation where humans can torture and murder each other, behaving essentially identically to humans in "base reality," aren't you in some sense facilitating those deaths?
> What Dyson sphere level of power would be needed to maintain such a simulation
Why would you assume energy is relevant to whoever turned on our simulation? If I created a simulation where the only source of energy was cotton candy, my creations might ask "but where would our creator even get enough cotton candy to run a simulation like this", not realize that it's absurd to think that the creator of a simulation has any of the same constraints that exist inside the simulation.
You run it to see if your military tactics will defeat the enemy and then you keep running it until you find a strategy that guarantees victory. Who cares if some sim lives are churned through the simulation, our lives are at stake!
Reductionist but I feel like that’s how it would go.
How does that argument work? I've seen more than a few advances, yet I don't see how any of it provides evidence that we are simulations. This might be one of those things that comes down to the definition of "simulation" I guess.
Not sure if you're joking, but the only such evidence I could think of would be something like finding some way to influence the hardware of our simulation and seeing a glitch in our world
Who is asking for this? Do advertisers really want this? I feel dread about many of the AI products being sold. They're mostly tools to help funnel money from the users to the bosses.
For several of these examples the details change if something goes out of frame and returns. A second castle is added in the dragon one. The people walking away from the camera change in the train platform one. The Hong Kong moped rider doesn't have a helmet and then he has one.
It's very good at maintaining consistency for things that stay in frame, but I don't know that this points to it developing a strong "world simulation" like they claim.
It's like its simulation model is confined to the renderable area. At the cost of increased computation, it would not be hard to generalize the technique to rendering a sphere centered around the POV to keep some of those artifacts at bay; however, the fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object. For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.
> While it can maintain coherent environments over short sessions, visual details and object permanence can still shift or drift over time—especially as a simulation runs longer, after multiple complex prompt edits, or when navigating throughout the scene. Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are active areas of ongoing research.
If it was actually not hard, they would have made their demo with it. I don't know the technical details, but I expect a product reveal like this to show the thing at its absolute best.
I am not sure you need something so complex. If objects on scene could be tagged with their location, and the camera movement tracked to identify when it brings the same objects back into scene, then the model can use its positional information to validate if the next rendered scene contains all the objects that should be expected.
Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.
Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?
The trick is to render a larger frame in lower fidelity and then only render a smaller output frame in full fidelity.
Or render a low fidelity 3d model for coherence and then raytrace that onto a 2d frame that is then rendered at higher fidelity.
Shhhh!!! That’s the next release… you are spoiling the surprise!!
> For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.
I suspect it is, because the 2D -> 3D models I've seen are terrible, but more importantly the 2D -> 3d models are terrible. Noteworthy because I suspect the number of people who have 3D scans laying around is tiny.
What you're describing is vaguely how video games work, to my understanding. There may be interesting parallels if that becomes a technique (like how distant objects are sometimes still rendered, but at a lower resolution/polygon count).
> fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object.
they got this working decently well for video over a year ago, maybe closer to two years ago at this point, time goes fast. so i assume it'll translate to progress in Genie too
This sounds like bunch of gibberish you made up
Thank you hahahaha
I don’t understand why these world models aren’t designed to be backed by an updatable world state. It could probably just lay out the world onetime and reference it as needed vs trying to invent from wholecloth via diffusion. Object persistence!
> It's very good at maintaining consistency for things that stay in frame
I noticed several inconsistencies in things that stayed in frame. See the wall on the bridge in the Van Gogh example as they walk past it to see the other side.
I bet it's configurable and just a matter of how much resource you want to allocate. If it's truly a world model then it seems to me that it's precise benefit is that it can keep track and order in things and they just choose to accept some noise/chaos/randomness/? for the demos when things move out frame? Does that sound sensible? I'm just guessing but don't feel it's a hard limit.
And these are the examples they chose. I have to wonder if they add some with mistakes on purpose.
If it's anything like their other anything related to AI, we're seeing cherry picked "best possible" examples.
It just doesn't have object permanence yet!
Reading the comments almost everyone seems not impressed at all and points at obvious errors in the samples. Meanwhile it's been just roughly over 3 years since we've seen an AI generated abomination of Will Smith "eating" (or whatever you want to call that act) spaghetti. Yes, AI gets things wrong. You could say the same about LLMs still making obvious errors here and there. To me, personally, this seems super impressive and it's clearly another leap forward. Not perfect, yes. But an improvement.
I'm by no means anti-anything but I think what the comments are getting at is that these "world models" do not have a world model. It's a fundamental qualitative gap that no amount of compute has been able to close, just mask by everything else getting better.
This doesn't show up as much where the verifiable reward is "code compiles and tests pass" but you can still absolutely see it in things like movies, prose, or slide decks. Even frontier models are pretty bad at keeping track of narrative threads even as the technical quality drastically improves.
This begs the question of whether the gap can be closed just with more compute (presumably yes, or at least errors can be minimized beyond human capacity to see them) and is that going to be worth it with current tech (possibly no).
does your model of models admit the possibility of partial models?
the average human cannot vividly recollect a walk through a familiar neighborhood, much less invoke a coherent fantasy of going on such a walk. do they contain a world model?
Must we, every time the limitations of AI are discussed, always imagine some simpleton human to defend it? It's trite and rarely adds anything to the conversation. The point is to judge the model on its merits. If this is a good world model, then it's a good world model. It doesn't matter whether or not a human can recollect every detail about a familiar neighborhood. It's irrelevant
My problem with it is the shift in focus. The original Genie was pitched as a simulator of the world that an agent can interact with to learn so generally that it could be deployed into the real world. This new version looks like a good, controllable video generator. The fact that you're comparing it to the will smith video, rather than a simulator, is indicative of the fact they've strayed from that original goal. I think that's a shame
People are emotionally driven. When emotions drive us to do something we will (unconsciously) find plausible justifications for it without realizing we're doing it.
Anti-AI crowd is anti AI. Maybe they were (mostly) happy with status quo and don't like change. Maybe scared for the future. Or any other reason. Went from "AI can't even tell what 2+2 is", to "AI can't even tell whether these simply connected smooth 4-manifolds are merely homeomorphic or actually diffeomorphic after stabilization" without batting an eye. The goalposts will keep being moved and the dismissal will continue.
It's a huge leap from 3 years ago, but not a huge leap from other current AI models. People who are deep in the AI rabbit hole have seen impressive generative video that has fewer problems (or possibly just better chosen examples).
I agree. This is nothing short of magic -- or perhaps 95% of magic. Imagine yourself 5 years ago, and someone from today going back in time and telling you "hey, check this out, in just 5 years you'll be able to write this short text and a computer will generate this movie from it"...
Impressive... at populating our visual world with more slop? Sure.
Would this be impressive in terms of convincing you to buy something or teach you something or entertain you?
Genuinely asking.
> Genie 4 has advanced audio capabilities
> Prompt asks for saxophone
Looks inside:
> Sound of a clarinet
Maybe Genie 5 will get it.
I watched it on mute, and even so I figured it didn't sound great, given that the guy's fingers barely move at all.
Perfection doesnt exist - technology will never free us from the human condition & constant struggles...
A sultry saxophone riff would free me from the human condition & constant struggles. Come on Genie!
I ain't asking for perfection here.
King Curtis said otherwise.
It looks like they had a hard time with the magic wand, peacock-to-cat example. Despite using a very long prompt that repeatedly stresses that the video should look photorealistic and NOT like a video game, the output video looks like a video game. The model is clearly capable of generating photorealistic humans and animals, so I wonder why it struggled to do so with that particular prompt.
I might not understand what "world model" means, but from my understanding world model is the model trained on real world data, so theoretically having all laws of physics baked in the weights. So perfect world model should not be able to make the video for given prompt, as it is impossible in real world and the only possible way it can appear is in a setting of video game or movie. So this is expected, doesn't mean that is the reason behind this particular generation.
It's probably trained in a video game-like environment.
I have no idea if that's the case, but it would be funny if it didn't support negative prompting and they got someone to write the prompt who wasn't aware of that.
Edit: Okay, I read the prompts and they use negative phrasing a lot, so I guess it does work, but boy, are these prompts laborious. Requirements are repeated ad nauseam. This model seems to be quite tone deaf.
Probably too much of the training data that has Magic wands were from a video game.
When people talk about "photorealism" it's to describe something not real (like video game graphics and artwork) that looks almost like a real photo/video. So you don't say that a real photo is photorealistic, and if you use that word when prompting models you end up making it not realistic.
> Prompt: A REAL, ultra-photorealistic, high-end 35mm cinematic
What do they mean "Real" here ?
I think they wanted the output to appear as if it were captured in real life rather than a game engine, but failed to do so which ironically made it one of the less visually realistic examples
Surprising how Tencent's HY-World 2.0 open weights world model didn't get as much attention:
https://github.com/Tencent-Hunyuan/HY-World-2.0
Is it surprising? Looking at the demo videos I can absolutely see why it didn't get as much attention. Not that it's not impressive, it is, but these are in different leagues.
Still, it's local, and I can't wait to see what it can do now that I know it exists.
Reminds me of a post[1] by Sam Altman in 2024:
> movies are going to become video games and video games are going to become something unimaginably better
Sometimes I really wanna skip 10 years into the future and see where all this eventually leads.
[1]: https://x.com/sama/status/1776083954786836979
> movies are going to become video games and video games are going to become something unimaginably better
God what an annoying thing to say. Suggesting that movies will essentially become games, or that future video games will be unimaginably better (presumably because they are driven by generative world models) betrays such a surface level appreciation for what makes art and culture and all this humanity so beautiful as it is.
The entire point of movies is the deliberate choices of the filmmakers! Movies will not be improved by our ability to choose what the main character looks like or add a style filter. Super Mario 64 is great because it is an expression of the very hardware that constrains it. Inevitable future versions where anything can happen because someone hooked the ROM up to a generative world model won't be better for it.
I agree that this technology is absolutely world changing, and am cautiously eager to see what comes of it, but it just bums me out, perhaps needlessly, that the people at the helm can be so artless, unreflective. Whatever.
yeah some friends of mine in game dev are a bit doom and gloom about all this and I need to keep reminding them that the hard work is making the million decisions that communicate the vision, the code/assets were never really what people were buying your game for. Even assuming an AI could do literally every technical part of game dev someone (for now) needs to sit there and go "nah this part doesn't feel right" and "that boss 2nd phase opener attack is too long" etc. until AI can also answer those questions and intuit what is good and bad taste there's still stuff for game devs to do. "minecraft in skyrim" is cool for about 10 seconds till you think about actually sitting down to play it.
They're getting surprisingly good at identifying why things don't seem right. I came back to an old game project that I had shelved due to the core game loop not feeling right and I decided to just ask Astra to analyze the gameplay and figure out what was wrong and it was able to provide a good direction to take it that also works well with the game mechanics that were already in place.
The new Terra 6.1 is pretty good at doing user testing as well via computer use, going through all the features and noting any bugs or inconsistencies.
It feels like we might be on a trajectory that takes us to a place where someone can just say "I'm bored, make me a new game" and then provide minimal feedback like "this isn't fun, make it better" and get an entertaining game without a lot of iteration. Of course we'll still have games that come from human ideas as well but vibe games seem like they're here to stay.
I'm not so sure, I've made an autobattler and even recently using Astra to come up with new mechanics or cards they're usually always bad as if it doesn't "get it" even if I feed it all the current cards in game and say "like this, these are good because of x and y." When brainstorming, I usually get it to generate 20 ideas and then I come up with something entirely new based on bad stuff it responded with. I've never been able to actually use its ideas verbatim.
I'm not sure if this is a limitation of the tech or whether it will eventually have "taste" in such nebulous domains as "what is fun." They still can't write funny jokes despite trillions of dollars of training because even though there's plenty of funny material and discussions on what funny is, I guess it's still kind of a human thing.
> not feeling right
That came from you. And it’s a deep thing.
Yes, Opus 5.5 can build out quite a bit on its own that is super impressive when you first look at it. But spending just a couple minutes will make you realize dozens of things that are off.
Sam had OpenAI start the Ghibli style image-gen (movies that are specifically loved for their artisanal quality). The man has patternly failed to read the room around culture.
Why? Customers lave the Ghibli style image-gen.
You don't understand, the important room is the tiny one in back not that big arena with ticket sales in front. /s
the Ghibli stuff was awesome, and I agree with him on video games
> I agree that this technology is absolutely world changing, and am cautiously eager to see what comes of it, but it just bums me out, perhaps needlessly, that the people at the helm can be so artless, unreflective. Whatever.
This is something your average person can play around with. The filmmakers have been fumbling the ball a lot lately (not all the time, though) and I can't wait to see what rises out of the average populous. Lots of crap expected, for sure, but there will be gems in there that I'm willing to bet will blow away what we've seen in the industry. Kind of how indie game companies are eating AAA's lunch lately.
Make something beautiful, with or without the new tech. You and the rest of us will be better for it. And the investors are damn desperate for something to really resonate with the people who experience it.
This. Do instead of consume and we all get a little better off.
We've been living in a world of videogame-like FX movie slop for at least 20 years now, without the help of AI. Try to find insightful reflection of humanity and truth in a Marvel movie... God help you. It's as boring as, well, watching someone else play a video game.
It turns out, some people like that. A lot of people like it, presumably because it's fast and stimulating, and everyone's a sucker for wowing FX even as movie FX have gotten worse and more or less converged with video games.
What's lacking in these movies isn't so much plot, it's character and thoughtfulness. They tell us nothing about ourselves.
From an art history class, long ago, I remember the explanation of how classical Greek culture degraded into depictions if "pathos", visible in changes in the sculpture, and how that heralded more or less the decline of the civilization. It seems obvious to me that we've undergone a similar transition. And the thing is, it's not the technology itself, which could be used to produce meaningful art. It's the lack of thought on the part of the creators and the audience.
> Try to find insightful reflection of humanity and truth in a Marvel movie... God help you. It's as boring as, well, watching someone else play a video game.
It’s so true. The only time I watch one of these movies is when I’m stressed and exhausted and want something so dumb I’ll fall asleep in 20 minutes.
When it comes to these “worlds” that are generated, there’s no there there. I’ve tried various of these and after three minutes I close the browser. Walking around my neighborhood is a million times more interesting. Whereas the creator-constrained and -designed experiences (GTA comes to mind) actually deliver something engaging.
> Try to find insightful reflection of humanity and truth in a Marvel movie...
Why would you look for that in any standard action movie?
Not every movie needs to awaken your soul to some big revelation about the universe.
The industry is called "the entertainment industry" for a reason. People are able to be entertained without deep philosophical introspection.
That would be a gotcha if anything else was making its way into theaters/ mass consciousness now. What are the highly anticipated movies? Spider-Man and a remake of some supposed epic from a while back.
> And the thing is, it's not the technology itself, which could be used to produce meaningful art.
Exactly. I am not de-facto against the use of AI in art. But artful use of the medium is sparse at the moment.
And then look into all the good movies in Cannes every year. Or subgenres even in us such as horror. Very far away from slop...
This. There is amazing art everywhere right now but somehow it's all slop. Nothing but anecdotes. Meanwhile some friends are in a bedroom studio as we speak creating Aja.
That’s what led me back to books. Character, plot, ideas, coherence. I also love great prose.
You occasionally see thoughtful movies. Many of them are unsurprisingly based on books. The horror genre has become this weird eddy of creativity too with interesting characters and plots and some surprisingly good acting. Of course you’ve historically had innovation in that genre, often ignored because it’s horror and thus low brow by definition.
My take on horror being creative is that it’s like old school punk, pulp novels, zines, underground music. The key there is that these are genres and styles that give you permission to suck, and therefore permission to try stuff.
If it must be high art or it must gross nine figures, you can’t suck. So you can’t innovate.
>Try to find insightful reflection of humanity and truth in a Marvel movie...
I can name several, as well as something profound to be discovered in the execution of the MCU as a whole, but perhaps more salient is why so many people feel the need to tear down the straw man version of Marvel that they have in their minds.
...Oh, fine, I'll give you one:
How is it a strawman if I've seen plenty Marvel movies and report on my experiences?
Radio -> Silent Movies -> Talkies -> Movies -> Video Games -> ? Why is this annoying? We are moving toward greater immersion through technology.
Those are just five separate things. Putting arrows between them in the order that an immersion-seeker might rank them doesn’t mean that it’s actually one thing that became each of the five things, it just means your ability to appreciate those things is limited to “how much can this technology immerse me (or someone else whose money I want)”. This is annoying and artless and leads to claims of “no appreciation for the thing”.
They are all capable of art. Sam Altman saying how things will be in the future of two mediums he doesn’t work in says something very loudly. You just have to choose which thing it is.
Because radio is not the ancestor of video games. Even if you wanted to make the case, it will be painting -> photography -> silent movies... not radio. You can't bunch two different disciplines as related to each other like this even if both of them are art forms.
We still have podcasts but you think radio led to video games?
In the sense that it's another medium for telling a story to a wide audience it did. On the technical side, radio communication contributed to the development of networked communications and is still in use in electronic communications so it also made video games possible. You could also do a deep dive on how mass media in general was influenced by techniques first developed for radio plays e.g. the use and creation of sound effects, changes to scriptwriting, adapting the idea of serialized publications to episodic shows, etc.
This is the trend of available technology, not a ranking of any particular entry. A great radio show will not automatically make a great movie, for instance.
As an example, while Hitchhikers Guide to the Galaxy is available in at least 6 different forms of media, the movie is by far the worst.
Except that movies don’t replace video games. They’re entirely different mediums.
The Orville show explores this idea, before LLMs became a thing. Simulators become the replacement for movies, but the simulations are still designed by a person (or alien).
The concept feels more like a video game than a movie in the show, but presumably you could program the simulation to happen around you instead of you moving through it.
there was a TV commercial in the 1970s that suggested this would happen in the future. Knowing Seth MacFarlane, that's where he got the idea.
Do you think that Super Mario Odyssey is better than Super Mario Bros for the NES? Is Breath of the Wild better than the Legend of Zelda? I mean -- take off your nostalgia-covered glasses and imagine if you handed them both to a kid who had seen neither which one they might be more interested in. Why do you think that is?
Is your contention better graphics are implicitly better? Does this mean movies in color are better than any movie in black and white? Also, are kids an amorphous blob? How should I know what a kid might like better?
Maybe just imagine what one might think? How would you feel if you hadn't eaten breakfast?
Odyssey is better than SMB, but OOT-WW-TP are all better than BOTW (I consider it and TOTK to be the worst games in the series).
I understand but I wouldn't dismiss how consumer choices already contribute to this
Younger people I know, and market to sometimes, literally avoid "plot based media" in favor of brainrot. Akin to what my generation used to call "sooo random"
People catering to it earn a lot. They don't identify as directors, but they are. They don't identify as artists either. They identify as influencers and brands.
You will never be the generation they engage with. I don't expect to base any of my choices in following trends looking for opportunities to be contingent on your affinity for movies and first/third person games.
Even when it comes to spending, they are a powerhouse because parents nationwide and worldwide give them their weekly allowances in platform credits, with no further accountability on how it's spent. There is a large audience already, trends that are full economies within a few months instead of needing to wait an entire 12 to 20 year generation for the audience to have independent purchasing power.
That's what is happening and why someone with even greater perception than me would say things that seem out of touch, but are closer to reality than you'd realize.
“Plot based media” you actually heard someone say this? Like it was a bad thing?
GenZ is culturally destitute. It’s shocking and sad. Every time I think no I must just be old and I don’t know, I go and dig and… very little. There’s just not much there except brain rot and blather and music that’s absolutely derivative of previous music at best. At worst it’s musical brainrot, like “phonk” inspired stuff.
I fear GenA will largely be in the same boat.
It’s scrolling and algorithms. People blast weapons companies and stuff like Palantir or Flock or whatever here, and for rational reasons, but infinite scroll plus engagement algorithms has done more harm to humanity at scale than any of those.
> “Plot based media” you actually heard someone say this? Like it was a bad thing?
yes, and its Gen Alpha that says it. they're just young though, but they are in aggregate an independent spending powerhouse
That’s less alarming since they are young.
I still feel like the cultural output of GenZ or lack of it is worrying. Compared to every previous generation by their age, I just don’t see a lot.
I will wait though. Maybe they will get over whatever it is.
> GenZ is culturally destitute
And video killed the radio star. Kids these days, man.
Some movies and some games (or parts of games) really are just for fun. Eg a lot of Bethesda games would improve if you could do a randomly generated storyline on your 4th playthrough, and WoW has 0 storyline to their dailies. It could be kind of fun like a fortune cookie to get a random story. I don't expect art, but it'd be funny to see if it made up a meme-able story.
An awful lot of mainstream movies do have fairly trite plots, with a predictable film style. That's not entirely hate, sometimes I really want to see Michael Bay blow up half of LA. I don't think the next Fast and Furious movie would suffer horribly from AI writing, it's never really been an "art" kind of series anyways.
That reminds me, has anyone made Ow! My Balls! using AI video gen?
> Sometimes I really wanna skip 10 years into the future and see where all this eventually leads.
Sounds glib, but honestly ... more Ads :/
Every ad since the dawn of time was designed to influence your purchasing decisions
But you could identify them as ads and understand their purpose.
That will soon likely no longer be the case.
I submit that most people haven't been able to do that for some time now. Advertising wouldn't be such big business if it didn't work on a macro scale.
Nobody is saying advertisement doesn’t work.
Nowadays with so much "viral hacking" and "guerilla marketing", you have to be mindful about all content you consume
Perhaps only half of "ads" are actually marked as such
So, it won't change that much from today, the difference won't be as big as the difference between 2000s ads and today
How would we know if we can't identify them and they don't tell us? The ads are among us...
This was already lost with product placements in movies/TV shows.
Product placement is the most obvious form of "subversive" advertising you could have chosen.
My 17yo niece shows me Nike ads on her phone, and when I ask her why she’s showing me ads, she gets upset and tells me they’re not ads — they’re web experiences.
What is obvious to you/us is not obvious to most..
I mean, sure, some advertisers might get more sophisticated. The garbagefest of internet advertising won't go anywhere, though. It'll still be as bleak and obnoxious as always, because someone will always accept a buck to let you barrage their visitors' eyes and ears.
“The ads will get so good you won’t realize they’re ads and you’ll end up buying all this stuff without awareness of why you’re doing it”
Do you actually think this is going to happen *to you* or are you just speculating about all the idiot dupes out there?
The gameplay _becomes_ the ad. If you have a full sash of potions in every slot, they spell "C","O","K","E". The monster-bashing sounds sound like onomatopoetic "Apple!" "Goog!". The armor colors are the same as the NFL teams nearest your locale!
It will be a nightmare! haha
Dont be silly, it will be on the companies hardware. We're pushed to a rent to "use" model in so many things now.
Subtle product placements tailored specifically for you will be injected into movies and TV
That’s optimistic, assuming we are still valuable as consumers.
Such a boring knee jerk reaction to things.
In what sense, the commenter .. or the reality of Ads?
So, people will be able to completely skip living in the real world, and instead spend their entire time plugged into a simulation, like a vegetable.
Sounds wonderful. May as well skip this and just spend my life hooked up to an IV drip of drugs.
Sounds like the plot from the movie matrix. Only difference between the movie and our real timeline - the ai didn't need to enslave humans, we willingly chose enslavement by ai for ourselves by continuing our google subscription.
It also sounds like Ready Player One, where the real world is so shit, everyone spends all their time in VR, using rented equipment, making nearly everyone a slave to the corporations.
The people building AI must be deeply unhappy, and won't be satisfied until they impose their misery on everyone.
Ultra nerdy point: in the matrix it’s revealed that humans did in fact choose to join it. And the meta point was that most of them still would choose it over reality [1] every day.
[1] reality is also somewhat in question in the films universe
Is this from the original trilogy?
It's from Matrix Reloaded, when the architect explains past failures of the matrix and that they only succeeded when they figured out that they had to offer a choice, even if that choice was subconscious. That nearly everyone accepted the matrix, and the few that did not escaped to Zion. That's the whole point of Neo, to manage the outliers and keep the system going.
At least that's my recollection. I've watched the matrix a few times :)
You are correct. And it's not that they chose for the Matrix to exist in the first place - humanity fought a brutal war against machine rule and lost utterly, and signed away their bodies under duress ("Hand over your flesh, and a new world awaits you. We demand it." -Animatrix) - but, at least according to The Architect, everyone within is presented with a chance to leave, one way or another. Most choose to stay.
It's worth remembering that the Matrix is an allegory for systems of labeling and control (notably, gender), which make life (and specifically life with others) easier, though not necessarily better. The Matrix sustains the large human and machine populations amidst Earth's ecological collapse; people who choose Zion choose a harder life where they are free to be themselves. However, when Zion grows large enough to threaten the system, the system comes after its residents and wipes them out.
One person's freedom is tolerable, even necessary for the system to continue; everyone's freedom leads to everyone's doom. The movies have some weird-but-rational implications.
...Sorry, I lost track of my point, which is that we're already in The Matrix. The Matrix is society.
There was a guy named Cypher who betrayed the humans and cut a deal with the robots to live in the matrix as a rich guy. He fully accepts the steak he eats is not “real” but he enjoys it nonetheless
Nothing in the original trilogy specified or suggested that humans as a group willingly joined the Matrix to begin with.
One individual human (Cypher) did choose to be plugged back in, though it didn't work out for him.
People already do this in many video games. The simulation isn't perfect, and maybe there's a tipping point where the simulation becomes real enough that it (maybe) supplants "reality". In a sense, this is disintermediation: instead of actually riding horse, I can ride a virtual dragon. Does that make it worse, or problematic? Again, we already have this, for driving race cars, fighting in wars, etc. etc. It's just a matter of fidelity and immersion?
Riding a real horse isn't just about the horse ride, right? It's about the trip to the horse farm. The small but real danger. If you're doing it on some sort of multi-day trip, it's about the memories you make with others, the romantic time spent with your partner at night, or if you're single maybe the hope of meeting a romantic partner. For some maybe it's even about doing things like cleaning the stables, which isn't intrinsically fun, but probably adds to the feeling of bonding with the horses.
(I've never ridden a horse. I'm surely wrong about some of those things)
There's also the "abundance issue" with simulations. Have you ever looked at your unplayed games on Steam, or a folder full of ROMs, and just had a hard time choosing or enjoying anything, because (a) there's so much there, nothing feels special and (b) even when you're enjoying one game, part of you is thinking that perhaps the next game is even better?
If things truly progress to Matrix-style simulation, I guess a lot of that is "solvable" by "just" manipulating/injecting memories. But I think that's how far a simulation would need to go in order to supplant real life experiences entirely.
Or, there's the even darker route, also explored by much scifi -- reality gets so shitty that nobody can really even aspire to an actual horse ride anyway, so simulations are all anybody has. A lot of people already live some kind of version of this anyway due to lack of funds.
Sorry for no coherent point. Just rambling!
People do it in all forms of “cinematic world building” including amusement parks, vacations, games books movies etc.
Reality is very much blurry
The Bliss IV is an upcharge subscription, though.
This is why UBI bad.
This is the worst philosophy course I have ever surveyed. Explain further. Your contention is that, given money people didn’t “earn” they will immediately do drugs?
Yes, but the other three options are AI failing naturally, AI being stopped, or people being unable to participate in the economy without UBI and starving.
Right now we're fully on track for one of several dystopias.
For some... that is sadly a better existence in cases.
Video games are already movies that are video games.
The ending cutscene of Metal Gear Solid 4 alone is a full feature-length film.
(And it’s annoying as hell if you actually want to play a game rather than watch a movie.)
There’s no end to games with near or actually 0 story/narrative.
Games can have a story/narrative without extensive scripted cutscenes! These things are almost entirely orthogonal.
It will lead to garbage.
Also, does anyone want movies to become games? I don't. I want movies to be movies. Good movies. With interesting stories, beautiful artistic cinematography, great characters, moving paintings as Lynch said it.
If I want games, I'll play games.
People still love oral story telling (standup). Adding more immersion doesn't necessarily enhance the experience.
For me, this does nothing. I am attracted to film and gaming for the artistry and deliberate choices made by the authors. A synthetic approach may be stimulating but I find it ultimately void.
10yrs isn't far enough. 2016 wasn't that different from today relative to media.
We’ve seen a lot of changes since 2016 including the rise of short form video, podcasts, audiobooks, music streaming, substack, even vinyl is up 3x (!)
Practically nobody consumed short form TikTok-like media in 2016. Now it's a highly dominant type of media.
Untrue. Insta stories were pervasive.
Instagram Stories were radically different in 2016 than the reels of today. They were far more personalized (less algorithmic public content), slideshow-like updates. Ten second max video clips, with most being just images. Ephemeral as well, where they'd get removed in 24 hours.
It's not the same at all.
And that’s not even close to the short form 90 second micro dramas that are already super popular in China and are ramping up elsewhere. And this category is now dominated by AI
BS, People already binge watching youtube back in the day. It's just the content is not as toxic as it is recently
Regular YouTube isn't the same as YouTube shorts. It's a vastly different product.
Short, <5 min video has already been a thing on youtube before tiktok. The most memorable one to me is "cat meme" and "failarmy"-style one.
It's still a radically different thing. It wasn't nearly as algorithmic. The interface and engagement was different. And the number of people consuming it all the time is totally different.
If they were really the same, why was there so much uproar over how annoying Shorts was when it was originally added?
How can you honestly think YouTube in 2016 was practically the same as Tik Tok? Why would these other social media companies spend so much trying to change to copy their same strategy if they were all already the same?
Short form content is radically different today than what it was in 2016. It's produced differently. It's presented differently. It's consumed differently. The audience is different. It's pervasiveness in society is different.
All the kids were on Vine back then, weren't they?
Many kids were, sometimes. Also, Vine was still pretty different from modern short form content, being capped at only a few seconds of length. Now most people consume tons of short form content, with many being 30s or more. It's pretty radically different in terms of what most people consume.
Like, movies sometimes have alternate endings. What if they were more choice driven, and replayable. Video games have remarkably dumb ideas that keep persisting, like you can encapsulate all that is “Health” as an integer. Instead games can be an aggregation of entity bound micro-simulation that create emergent scenarios, yet follow the laws of narrative collectively. Games that build themselves. Worlds within worlds kinda jazz.
> movies are going to become video games and video games are going to become something unimaginably better
This is the kind of detached from reality, devoid of humanity statement only a tech bro could come up with.
It boggles the mind how these people can be so dumb. He literally does not understand why people enjoy movies.
Maybe you need a mirror. People are literally watching AI generated soap operas that have gone viral. Perhaps it's your lack of understanding that frustrates you
Do you know these people and have spoken to them?
Yes, I've seen them watch it in real life too.
I for one would love to be able to experience Godzilla Minus Zero interactively. I get that it's different from movies. I remember seeing Avatar in 3-D and realizing that James Cameron was directing the audience's attention to a particular part of a scene by the way the 3-D focused, and it was a little off-putting at times. That's a small thing compared to what will be possible in a few years, but still -- being immersed in Avatar will be an amazing experience. I'd also love to see a version of Avatar written by '80s James Cameron -- he was a much better writer back then -- or maybe other, more skillful writers had greater influence on him? In any case, it's inarguable that Terminator, Aliens, The Abyss, and Terminator 2 are all better written than the Avatar films.
More and more off-topic, but this made me wonder: this interactivization would work much better for some works than others. Many comedies are scripted so tightly that I can't imagine any wider world around them (also, on the other side, Memento), while something like Twin Peaks would be perfectly expandable into a world (if you somehow had David Lynch in a datacenter, which sounds oxymoronic).
The problem with an interactive Twin Peaks would be finding your way out of it :-)
> I for one would love to be able to experience Godzilla Minus Zero interactively.
So would I. But I don't not want to experience it as a movie. It'd be something separate. Not an upgrade.
Which makes Sam's comment the inane blather of a pitchman. We don't need to bend over and try to steelman marketing dross. He doesn't care if that statement means nothing, why should we?
Oh, I see your point. I didn’t think of “movies becoming video games” as meaning there’s no such thing as a movie anymore. Kind of like the false argument about evolution, “if we can from monkeys why are there still monkeys?”
Movies come from books, but people still read.
But I get the point.
> Sometimes I really wanna skip 10 years into the future and see where all this eventually leads.
Either the desert of the real, or something unimaginable to Mk 1 humans.
Great, I guess I can say bye bye to movies
With VR coming and then going technology has shown itself to hold within it no more immersive capability than a good book. It is not about rendering firepower it is about hitting the right notes.
Type of thing he would say.
Llms wont be able to create good games. Consider for a sec that there's like 500 "Great" games ever made, maybe more but not much, in some genres that number is less than 10. There is no training data, and its largely a matter of taste and culture that cant be represented/distilled into vector embeddings.
While i generally agree, some of this tech could be a major force multiplier and lower the barriers to entry.
Even if you have a great game idea, it takes a ton of time and money to even get it off the ground, let alone get it coded correctly and maybe get the reach you want. A LOT of flash in the pan ideas probably fizzled out faster than they needed (most recently Among Us and Fall Guys come to mind) because they were built to ship, not to easily modify.
Tech like this could lower the barrier to smaller and more creative teams or even individuals and get some better competition in the area. Fighting games KNEW that rollback was the way looooooong before AAA bit the bullet and started making it standard, and they still fuck it up (and it probably would've taken longer if COVID didn't force the issue and punish those lagging).
That said 2 of the largest hurdles i've seen in everyone's "magically ideal" game are enemy AI and Plot.
Interesting and well designed enemy AI is still hard, and even games that did it well built the whole game around it (FEAR) and the system is vastly less impressive without that focus (Just Cause). I don't think we're anywhere near modern AI solving the issue of making a good AI system for a game due to design space problems, but it might.
Plot though...that's tricky. Everyone always says how cool it would be for everything to be like Mass Effect or more, with all choices mattering and everything interacting, but someone has to write that. AI is not turning out well though out plot points that don't feel like the equivalent of Bethesda mission creep.
Point of this whole ramble being that in arcadey/simple stuff I think we'll see a lot more great games (and would argue we already were with the growth of the indie scene over the last 15 years thanks to steam and other tools), but there's still going to be entire genres that are just miserable to develop.
I agree. Game design is what makes a good game, not the programming. And I'm actually excited about game designers and artists being able to take their independent ideas and prototype games in a day/week/month/year/whatever.
+1 on fighting games, an interesting trend in that genre right now is that many of the latest titles (realistically everything other than SF6) are having trouble with player retention, while AI-assisted netcode updates for old titles like SF4 and Tekken 6 seem like more compelling products
Or, the lower barrier to entry crowds out the committed indie devs willing to invest years of their life honing their unique artistic vision for a game that might end up being a hit on Steam (among many other failures). The perceived odds of getting some recognition or even a moderately profitable hit go from "slim but possible" to "hopeless".
Because now their competition outnumbers them 1000 to 1, the competition isn't devs or artists but anyone with a phone, who can be from any part of the world who may not even speak the same language. Many to most of these solo "devs" aren't interested in games at all, and just hoping to rack up sales, ads, hype a crypto shitcoin launch or whatever. These slop games all have flashy attention grabbing visuals, slick product pages, five minute promo videos, with the games boasting 500+ hours of gameplay... of which 99% are recycled borderline unplayable trash.
The indie market for lemons stifles indie devs who struggle to find any audience, with modest dividends going to the most deceptive, scammiest solo sellers churning out 10+ titles per day (mostly 1:1 copycats of GTA/AAA games, psuedo-porn games, gambling games, meme/news cycle one-off games, etc). Middlemen like Steam print more money than ever as purchase volume increases while the indie gaming market becomes closer to the mobile games industry today.
The vast majority of the force multiplier value from AI is captured by the handful of AAA conglomerates who follow the tech monopoly strategy of maintaining market share by constant acquisitions and collectively control the most valuable IP while constantly reducing headcount thanks to models like Genie.
But who knows. It just seems like wishful thinking whenever I hear talk about how AI will empower the little guy to take on Goliaths considering the huge amounts of capital driving AI research and products are coming from those very same giants.
Their "good games" is different from our "good games".
To us, games like BG3 or E33 are masterpiece, but let take a look the most profitable games, GTA V, that is their "masterpiece". R* Knows this and high chance GTA 6 will continue this trend, a passable story combined with top notch graphic and physic, which both AI is really good at already
What makes a good game has changed over time. Pacman is, honestly, boring as shit now.
Mappy, weirdly enough, holds up well. Bubble Bobble, still good. Mortal Kombat 1, meh.
Some of my favorite classic games, almost unplayable now. Tastes have changed dramatically.
Movies are the same. Very few movies hold up after multiple decades. Some do. But even some great ones don't.
I don’t really agree for movies, basically my entire top 50 is pre-2010, many of my favorites much older.
Why wait? It's become very clear already: https://kotaku.com/the-proliferation-of-copycats-on-steam-ma...
Instead of better it's become unimaginably worse since games can no longer serve as a means of self expression because the second you express anything worthwhile 100 people copy it immediately ... without you. It's like Disney (whose whole business model is copying public domain stories and music and ... but taking out the source and everyone who ever had anything to do with it). Except there's no longer one highly selective company destroying the world's culture one piece at a time, but 10000000 idiots looking to make a quick buck, destroying culture and self-expression like a fire raging in the rainforest.
There will be no more "celeste" games where you join for the platforming but stay for the (politically offensive to all sides) story, because AIs won't discuss the story as it's politically incorrect. Celeste is fundamentally about depression and anxiety, due to being transgender, and is about offending both sides of the argument. First, the main character is transgender, with zero explanation or need for explanation how far that goes, which of course offends conservatives. Second the goal of the main character is to run away from any provided help and professionals (which the director did too) and even under the most extreme of circumstances only reluctantly accept help from a family member that then promptly dies making things "worse".
The Qwen and other Chinese models (and ...) will refuse to discuss being transgender at all, and any western models will shut down any discussion of running away from help hard (because otherwise they get held responsible for the consequences, as has already happened)
(oh and that running away from professional help in psychological issue is actually observed to be a positive thing in almost all cases, in studies. The main way to make psychological problems worse is to let them fester and running away from help is time and time again observed to be far more productive than going to professionals and expecting them to fix things. Because that's an extreme human bias: it can never be (allowed to be) the right reaction to NOT do something about a problem, especially when countless people are paid to "solve" the problem. In other words: if you take people who expressed intent to commit suicide and compare the groups who "get help" and the group that successfully shut down help, less people actually go through with it in the second group, and worse than that, the more intense the help, the better the odds the patient ends up killing themselves. Right to the point that doctors sometimes assist such patients because after 2 years of attempts people live in constant misery and have caused enough medical damage that they will have constant pain or worse the rest of their life. And yet people keep forcing help on patients. Usually in response to an actual suicide where no help was provided. Then more help gets built, and 5 years later we yet again observe ... more suicides. But tell someone with an actual problem to avoid help and you'll be skinned alive for that)
So AI could never make a game like that. Not because it actually can't, or isn't capable, but because there already is a strong political bias, in every model you might use to make it and neither side is going to back down on their policies. The Chinese Communist Party is not suddenly going to change it's mind about transgenderism. And the western models are not suddenly going to openly discuss running away from psychological help, not even as part of a story.
So really, we've just lost these games.
I get that AI anxiety is running high. But, this is doomerism running hard.
Needle in a Haystack Simulator is the new Threes! https://www.theverge.com/22914955/threes-2048-ketchapp-copyc... A very simple game to clone that gets 10,000 headlines because creators have always been anxious about cheap clones. And, anxiety drives the clicks that pay the bills.
Meanwhile, local models are already extremely capable and slowly shrinking their gap vs. frontier models over time. And, it has been long established that the censorship trained into those models can be disabled easily with open-source tools. De-censored models will happily dig into topics that push beyond the limits of sanity, let alone political pearl-clutching.
Besides that. If you want to write about some topic and AI doesn't want to help you, you still have your fingers. Type like a human and you haven't lost anything compared to when Celeste was made.
100%
Needle in a haystack isn't a game. It's a way to stimulate your brain for 30 seconds at best. It's not a game.
Man I really liked Threes.
Hmm. Never heard of non-conservatives thinking Celeste was offensive. It seemed clear to me that a large part of her character arc was learning to accept help when it was offered.
It will lead nowhere. Graphics intensive video games as interactive environments benefit from efficient rendering done by hardware a fraction of the cost of what a neural network requires for the same output. This is a business dead end.
A big chunk of the right side of castle disappeared, including catapult, when camera turned back while flying on dragon. See 1:25 to 1:29. I was not expecting this to happen in a "world model".
The 80's Japanese train station clip also had the conductor speaking clear english in an accent I can't quite place. Slight British tinge to it but not really? Not sure. The ambient audio also makes no sense.
I've seen weird stuff in images that OpenAI sends me in it's promotional emails too. I would think that if you were trying to show off you'd make sure there's none of these obvious AI issues in what you choose to show off with. Maybe they don't really care? "Gemini choose 10 scenes that would be impressive for our announcement and add them to the post."
The accent is received pronunciation, like Tom Hiddleston. It sounds like Tom's voice. Perfectly normal British accent.
That entire clip looks like it's straight out of a complete bargain-bin videogame. The weird smoothness of the ride, the pivot in mid air, the wings apparently don't have collision with the poles they fly past.
Definitely wouldn't have been one of my picks to show off my fancy new model.
Getting good quality dragon POV flight data for SFT must be really hard.
the goal posts have definitely moved since the original will smith spaghetti video huh?
Oh yeah, don't get me wrong "bargain-bin videogame" is a wild step up from "accidental amorphous horrors that defy description". It just jumped out to me as looking unusually bad compared to the other samples they provided. The stained glass deer, for example, was awesome.
The labs are the ones moving them. We're all just measuring against their own, exponentially growing, claims.
There's also a whole extra bonus castle that appears to the left when the camera looks left around 0:22. Which is amusing both because it wasn't there on the initial approach, and also because nobody puts two castles each with their own distinct curtain walls right next to each other like that.
"Storm the castle!"
"Which one!?"
The oddest thing about this, to me at least, is that they pick that as their showcase. Did they themselves not notice? Or did they think their potential customers wouldn’t? Not sure really which of the two is worse.
Neither one: I think this is the best output they could get.
customers? You can buy Genie? It's still very much in tech preview, research phase.
In the original dragon clip, at 0:29 we can see a square tower in the right side of the screen
As he passes by the tower, it becomes round, by 0:40
It is always incredible to me how quickly we go from "this entire technology is pure science fiction" to "nitpicking over details"
It's not, it's just that hacker news is a hyperbolic bikeshedding chamber.
I am disappointed mainly because current video models are way more consistent these days. This being a new release, I was expecting more than few seconds of consistency in the scenes.
I was under the impression he has already fly past the catapult.
Cool demos, but does anybody else find it troubling that companies keep claiming this tech can train robots? It’s so obviously not representative of the real world.
> Cool demos, but does anybody else find it troubling that companies keep claiming this tech can train robots? It’s so obviously not representative of the real world.
A proprietary version of this technology is already training Waymo's robot cars. Waymo blogged about their DeepMind-provided world model a year ago.
> It’s so obviously not representative of the real world.
That entirely depends on the use-case and level of granularity needed, doesn't it?
Isn’t the Waymo version used for some small portion of the dataset to cover edge cases that they have no real data for?
This blog is suggesting that we could train robots using only a world model.
> Do you have a basis for this idea, or it's a gut-feel?
My basis is that I’m a real human and these videos look far more like bad video games than the real life I’ve experienced.
Apologies for the incidindiary wording of my initial concluding question (which I immediately edited to be less flamey, and hopefully more cogent).
> Isn’t the Waymo version used for some small portion of the dataset to cover edge cases that they have no real data for?
The usecases in the blog post[1] paint a much more expansive picture of how Wayne used Genie 3: including enabling transfer learning to convert single-camera dashcam footage into simulated full-take sensor data (multi-camera and Lidar), which is extremely valuable.
> My basis is that I’m a real human and these videos look far more like bad video games than the real life I’ve experienced
1:1 fidelity isn't required for a lot of training tasks; for instance, computer vision can do a lot with 32x32 resolution images, which is trash quality to humans. I think the potential customers for this technology will constrain the model to fit their use cases.
1. https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-f...
Train robots feels like the academia justification excuse when we really know the value of this will be entertainment.
And warfare of course, the stuff they won’t openly talk about but we might find out about months or years down the line when it’s used to destroy another school or hospital with people inside.
Subreddits where SolidGoldMagikarp was counting to a million in comments were used to train a machine that then writes web pages that more machines are trained on. The sheer volume of data with sufficient verisimilitude generated by these models may well work to train robots. It might not, but I think it might well work. The idea that things have to be high-fidelity high-quality to yield good results has clearly not held up in other domains.
Has anyone ever tracked down SolidGoldMagikarp and interviewed them? It's such a uniquely bizarre claim to fame.
Elon revealed to the world that you can just lie and people will give you money, so now they all lie.
Elizabeth shows us that it has to be the right lie though.
This stands out to me too, it seems to be representative of physics the way a video game is representative of physics, likely less so given the non-determinism
Gemini Robotics is a thing: https://deepmind.google/models/gemini-robotics/
They clearly know how to train robots. Unless Genie is a misdirection...
I guess they want military contracts or something.
You can get useful levels of robot training out of Super Mario Bros 2; see John Carmack's talk from last year: https://www.youtube.com/watch?v=4epAfU1FCuQ
When your training simulation fails to reproduce certain features of the real-world environment, your control model is likely to have trouble with them, so more realistic simulations are helpful, but even fairly unrealistic training simulations are pretty useful.
In general, training neural networks to control robots is going to require simulations, ideally differentiable simulations. Even if they aren't differentiable, though, you can SGD your simulated training runs across a data center in a way that you just can't with training runs on Roombas.
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue
What on Earth is audio-like music? Do they mean music-like audio?
Audio like: music, ambient-noises etc
I'm blessed to see that even google let their most important marketing prosa be generated by ai models without any proofreading from fleshy humans.
That isn't a mistake AI would make though -- it's something a human would accidentally do when replacing em dashes with hyphens
Which they would do if the text was LLM-generated and they wanted to disguise that fact. Or have a vibe-coded script to do the same.
I'm sorry but this comment is idiotic on so many levels
It could have been a human trying to justify their existence. "no no, I wrote this ad copy myself: look, no em-dashes!".
Quite the reverse honestly, I found it really funny that they didn't bother to have an AI proofread it before sending out
That was probably meant to be an em-dash, or have spaces around the hyphen.
probably meant "audio like music, ambient noises, and interactive dialogue."
or just "music-like audio"?
Have you listened to "music" created by non-specialized models that use WebAudio to create?
It seems like a good descriptor to me! ;)
As opposed to video-like music...
I think its a useful exercise to try and name a cultural artifact (picture, video, game, music, book, etc) that was improved, or made substantially possible, by this technology. Ideally something that has moved you or provoked a lasting response.
Conversely, try to think of examples where Generative AI has betrayed a lack of creativity or care from the creator, and thus left a sour or ambivalent or impression.
I find the former impossibly hard, and the latter depressingly easy. Do others feel the same?
There are tons of cool, creative, funny and wow stuff being done with AI. A third of my Instagram feed is AI creators and some of the stuff they do is mind blowing, but even when it's not super impressive, some of it is just so creative and well done. Saiyenfeld is amazing (DBZ Seinfeld mashup by grickusai). It's taking a medium that is familiar, putting a spin on it and doing it well. You can do things that you couldn't do otherwise. People with interesting concepts can finally get them out of their head into something really interesting. People are so doomer and depressed, but they purposely don't see the stuff that is just fucking fun. I spend all day with a 5090 and courses and I still can't make stuff as good, creative, cool or funny as these people who actually have those talents and they are just fuddling through with token services.
If you want to get inspired go to civitai and look through the top stuff and be wowed. Maybe they didn't draw it, but they figured out how to make it happen and when you look at enough generated stuff you can tell the average from the holy shit. It's easy to get inspired by what people can come up with and want to do something yourself, which is a positive impact I'd say.
I'm glad you feel this way.
I can't personally relate, and I think its kind of telling that I'm being down-voted for simply asking people to engage with this thought experiment.
I agree, I don't understand why this technology has so much energy put into it. Just because we can doesn't mean we should?
By Genie specifically? Or by generative AI in general?
If you mean world models like Genie,I don't know. If you mean the latter, I have definitely used software that I have enjoyed that I do not believe would have been possible without generative AI.
Fluxer is the first Discord competitor that when you show to someone they can't say "well Discord does X better", and it was largely made using AI.
Herdr is designed for AI usage, but I actually really like it even for just normal terminal multiplexing. It was surely made with a lot of AI.
I have seen people make incredible things at my work that have actually solved multi-year long pain points that we've never had the resources to address before.
I have used AI to patch and revive a bunch of small services like chat bots or web services and such that I had made and had decayed into failure over the years.
Sure, it's easy to point at people using AI as a megaphone for their lack of care. But I don't think it's impossibly hard to find people doing good things with it either.
Software might be the exception, but I think some skepticism is still warranted.
When software is hard there's a kind of filter; a good piece of software means a human/team behind it with grit and care and user empathy. You can have some* confidence in the continued stability and maintenance of the software. (*relatively speaking, and with huge error bars).
Without knowing the details of Fluxer, if someone tells me about a vibecoded Discord clone, my immediate thoughts are like: - how much thought and care went into this? - how much work from Discord has been cannibalized? - will the authors be around next week to keep maintaining and making improvements and thinking deeply about their users?
I have also seen a lot of fantastic demos and stuff with AI coding. I use AI coding all day. I just think there's a weird gap still, where code and images and text are all "cheap", but none of this has translated into anything I personally use or care about.
They're not really promising to do that. It's not even clear this model will be meaningfully commercialized. The key line is "Understanding and simulating the world is a crucial aspect of building AGI". This model will be used for research and robotics, and maybe the training of further models.
Without getting into a larger argument about whether this is the correct approach to building AGI, I think its reasonable to ask people to engage with how this technology gets used in practice, besides its nominal purpose of "building AGI".
The predominant way people engage with this technology right now is through slop content in their instagram feeds.
(Btw I say all this as someone that has worked on generative models for 10 years).
Unironically, Tung Tung Tung Sahur and some funny music video about p(doom). But yeah that’s basically it.
Throw a gifted storyteller at any medium and they will produce something worth paying attention to. AI generated content has not had that moment yet. I'm not sure it will ever, but it's worth thinking about.
It took the medium of film ~50 years to find Hitchock and ~75 to find Spielberg.
I have thought about it.
I'm encouraging others to do the same and be honest with themselves about the direction we're headed.
Looks cool, but it's very hard to get me excited about yet another proprietary model for which absolutely zero technical details are released (not even the number of parameters), which is safetymaxxed to the limit, and whose usage quota will (most likely) be so limited that it'll be effectively useless for anyone who doesn't work at Google or doesn't have infinite money.
> "which is safetymaxxed to the limit,"
I feel refusal'd just *looking* at this page.
What an insane set of requirements for a frontier technology and your excitement by it.
i think he's bummed out that a cool thing has shipped but: 1. he doesn't get any information about how it works 2. he doesn't get to use it
sure, it's still a cool accomplishment but i can understand being a little bit frustrated / not as excited for such an announcement.
Most cutting edge technologies through history have been 1) somewhat opaque and 2) narrow/exclusive in their deployment. Historically speaking, that's virtually synonymous with "cutting edge technology".
Whining that I, a random Internet user with limited budget, usecases that the researchers don't care about, and no connection to the researchers, don't get technical details (??) or access (??) on day one is just profoundly goofy shit.
Obviously being able to use these things right away would be more exciting. But to be unable to get excited by advancements we can't immediately use is... quite entitled.
Most cutting edge technologies through history did not have near-peer widely-deployed open competitors. This technology does.
The sole purpose of a press release is to generate positive interest. It is not entitled for the reader to expect the time they have invested in reading it to be worthwhile.
Press release announcing a new car:
> erhmygord I can't even buy this car right this second for nearly nothing and use it to drive a thousand miles per hour through a kindergarten parking lot?!?!
If you mean that this press release amounts to the AI equivalent of the latest Tesla Roadster 'release'... then I guess you're kinda right
This.
And I wish I had thought to grab that nick years ago…
When tweetles battle beetles… AND…
It’s not an exciting advancement if it’s just the AI equivalent of google wave. It’s totally reasonable to be skeptical of marketing and want to try something for yourself.
Absolutely no need to slander Google Wave like that.
Oh, but there is, good sir. There is.
.. what was wrong with google wave? I kinda miss it
Fake hacker thinking Google Wave wasn't released to the public lol
Google wave was eventually released to the public but initially it required an invitation. So if you had a friend at google you could get in but if not you were out of luck. This killed early momentum and when they eventually opened up the product was dead on arrival.
> But to be unable to get excited by advancements we can't immediately use is... quite entitled.
Well, I didn't go knocking on Google's door asking them to give me access to their fancy models. It's them who came up to us saying "hey, look at the amazing model we have here; too bad we won't tell you anything about how it works nor even let you access it". Entitled you say? Then maybe don't dangle a carrot right before my face, and then complain that I would like to eat it?
Then don't click on the link homie
"Don't talk publicly about things I personally cannot use right now" is peak entitlement, yes.
> "Don't talk publicly about things I personally cannot use right now" is peak entitlement, yes.
I never said anything like this? My point wasn't that "I can't use it, therefore it's bad"; my point was that as-is there's pretty much nothing exciting about this. Just because this exists inside Google doesn't really mean or do anything; it doesn't meaningfully advance the state-of-art precisely because nothing about this is public.
Do you remember Sora[1]? Announced in pretty much exactly the same way -- grandiose webpage with fancy demos, zero technical details, limited availability and then... remind me what happened? Oh, right, they completely shut it down, and in the end it didn't "advance" squat. Might as well just never existed. So remind me why exactly I should be excited about this one here...?
[1] -- https://openai.com/index/sora/
In order to claim there is any 'advancement' here requires a baseline of openness which is not met here.
> is just profoundly goofy shit
Why not expect it? It's what most model releases do.
Yeah, I read it as: " Cool! Who cares? " Maybe because that's how I felt
You must have a very narrow conception of sanity.
The future is coming by appointment only.
I think this might be what Gibson meant by "... it's just not evenly distributed"
Not insane at all, completely sensical.
> frontier technology
It's just a lame gimmick, this is not 2024 anymore, people have started to get sick of this AI thingie, especially when it comes to video/image generation. That Simon guy with his pelican is the exception that can only be seen on tech forums like this one.
As to why US (and probably also some Chinese) companies have started pouring hundreds of billions into AI-image generation is beyond my comprehension, but on the other hand I'm no economic genius.
No it’s not insane.
If I can’t realistically use it then I don’t care, especially when there’s no independent proof yet it works this well, given a cursory glance. If it trickles down into future developments I can use then cool, but it likely won’t because they want a moat. You don’t have to shield the poor defenseless multi-billion dollar megacorp from the genuine apathy they inspire.
If this is the best “frontier” they can achieve, then it’s no wonder they haven’t done anything groundbreaking in the space yet that produces practical results.
It's another closed world model.
Only open weights matter.
We should distill everything else we can into open weights.
These are exciting for robotics and industrial perhaps, but they have no consumer use case until the lid is blown off and consumers can do whatever they want with them.
So much societal progress is behind held up because the leading labs want to keep a stranglehold on the tech. Imagine if every PhD or hacker had access to the raw weights.
The general history of new technologies is that they're available for a narrow set of circumstances and then they proliferate. Can't believe that needs to be said on "hackernews"
Default internet culture is "I am entitled to everything digital, in full, for free".
Which stems from the fact that most people have never created or contributed anything meaningful.
The internet has become a privacy and rent seeking hell hole.
It used to be a lot different.
Look at companies trying to build more things that pull value into their private cloud offerings.
Except that most technology was built in a way that didn't require all of the humanity knowledge and labor to be sequestered and then regurgitated back as slop that we need to pay as a subscription from what essentially is a rent seeking mafia.
The only one pretending this is something that needs to be said, for whatever reason, is you.
amen
I mean this doesn't seem very insane? Basic info about the model, the ability to actually use it. Google in particular is awful about getting access to their models from the UX side: I don't even bother trying them out unless it's via openrouter.
Yeah, I want that too, including the weights and training material. But I am still exited by what one could potentially do with it, by looking at the prompts and generated videos, even if it is cherry picked.
I’ve decide that the best (and maybe only) way for me to interact with google AI models, is on the search results
It’s where I get the most value, and my expectations are very low, no attachment if it goes away
Of course they are mining me for data anyway, but they were doing that before already (and I do have a Gmail account, although I’ve almost completely stopped using Chrome - btw, it’s crazy that you can’t add plugins/extensions to Chrome on Guest mode)
I don't get how Google is so horrible at product. Let me play with this. Otherwise it's DOA.
That touches on the core reason not to get excited about these Google releases: They release something cool and then don't iterate on it. Instead they go off building another cool demo instead.
Gah. Go iterate. Listen to customers, adapt, iterate some more etc etc. That's how products are made.
Some companies are so default-alive due to essentially building a money printer that risk tolerance has been tuned down severely.
So Google is the new Microsoft but with better software engineering quality. And two decades younger. And they're still slowing down.
(Maybe I should reallocate some of my GOOG stock into ANT after all.)
I'm not entirely convinced the new models aren't just a rollback to a clean training data state after the current model has been released for a while.
This is going to transform how we create games, movie content, and interact with virtual worlds, as well as those born from our imagination. You have control, like a simulator where you can close your eyes and immerse yourself in artificial worlds that feel incredibly real. It’s crazy how fast things are changing.
If AR Headsets are supported by Genie 4, it would be mind blowing. But latency would be a bottleneck to get there.
Alert me when these "world models" can make a video of a skateboarding line that fools a skateboarder.
And no, this didn't: https://www.youtube.com/watch?v=V2K_OLiZ_RM
I watched all of that video, it was really good.
In the clip with a marble bust dropped on a watermelon, at this point https://imgur.com/a/IHLnkUs the bust should've toppled over, if the physics were properly modeled.
This is a reflection of a complaint I frequently make about so called "world models". It's more accurate to call them video game models. They don't simulate the world, they simulate video games. For that reason a common issue with them is that the physics are video game physics rather than real world physics.
I'm sure you're probably right about video games being used in training and, as a result, the models reflecting video game physics: No air resistance, no static/dynamic friction curve, etc.
However, I don't think that's the problem in the example of the marble bust not toppling. It would have toppled in a video game engine as well; they model gravity and inertia just fine.
I think what's happening is the model isn't being a world model at all, it's being a language model. It's trying to make the "scripted" thing happen, and it probably "feels" like the script should end in the bust on the pedestal.
Yes this is right. Something about the way it works under the hood means it wants: 1. bust falls 2. it crushes the watermelon 3. it teeters 4. it settles on the plinth. It is sort of interpolating between these states, which it feels are a realistic sequence of events, not actually attempting to model what would happen in a real "world".
It also failed to properly model the physics of knights riding wyverns into battle.
Which is fine for a video game. Not at all fine if you want to train robots with it. Trying to combine both use cases into a single model is doomed to failure IMO.
I would not be so opinionated, modern coding agents derive from chatbot technology. Very different problem domain. They share language but language is universal so that doesn't say much. Then they figured out that the pretraining is a good starting point for RLVR. Who's to say a more cinematic imagination can't be a good starting point for a world model? It doesn't need to be the whole solution.
If they don’t get ahead of this our robots are going to be sitting ducks against knights riding wyverns smh
People are arguing if the marble bust should've toppled over or not, completely missing the fact that marble is brittle and the bust would have surely cracked
How often do things in the real world not behave exactly as you expected them to?
I feeling I get from this is that this could be the mental model for a robot in the real world, continuously corrected by actual events.
We all have a world model in our heads of what we think is going to happen, we adjust quickly (or not) based on what does. This is pretty incredible stuff.
Busts rarely hang in midair at a 45° angle.
As an aside, why fall into the role of being an apologist for google-hype and Big Tech?
You're missing the forest for the trees. The less than amazing physics isn't the point.
When you have a conversation, you have an idea in your mind of how the conversation is going to go. You predict what the other person is going to say before they say it.
When you are driving, you predict what the other drivers are going to do, you predict what signs say, how long you have until the green. If you see two red lights attached to the back of a car flash on, you may predict that the car is about to slow down and you start slowing down before their speed has changed.
Forget video games and movies. I believe these models will be that inner predictive model for robots.
> How often do things in the real world not behave exactly as you expected them to?
They wouldn't be called "Laws of Physics" if obeying them was optional. If your center of gravity is outside your base, you _will_ topple over. That's just how gravity works.
Your expectation is the point, not the physics. I'm not arguing that their model has an amazing grasp of physics. I'm arguing that this can be used as a predictive internal model for a robot.
I agree but I feel like the exaggerated physics were intentional and fitting with the video theme. If they had wanted it to topple, they wouldn't have posted the video.
They were demonstrating steering.
So what, the world model failed to model physics. The thing they said it did elsewhere. It's a valid critique
That's the thing about machine learning over time, and the big hole in the whole world model idea. Time and time again we make simulations and train models and we think that loop will work. It never does.
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
It's where (ex-)"AI luminaries" go to die. It's a way to make glacial, slow progress in the correct direction while never actually getting anywhere.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone (some model) who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
It depends if the goal is a more accurate simulation, or if the goal is a tool for creatives to build games or content.
If it's the latter, I don't see a 'hole'.
My point was that there is also the possibility of AI intelligence improving to the point that there isn't a difference anymore. That whatever beats the next set of benchmarks (which of course won't be the simulator or world model) is so smart it is both a better solution to the problem the world simulator poses and a better world simulator all in one and a better tool for creatives to build games or content.
This has already happened to other ML fields. For example, there's not much point anymore to ASR models (just feed audio into LLMs) and the space that TTS models occupy is fast disappearing. Oh and it's not just that Qwen 3.6 is a good ASR in addition to being good at coding mario. Doing things this way, not using an ASR model is fundamentally more expressive. A whole bunch of companies started up on ASR + TTS "towers" (lots of models, Pipecat style) as they're called, and they're dead companies walking. Their tech has been surpassed by just bog-standard generic LLMs in a way that they will never get the upper hand again.
A chain of models approach, a separate ASR, will never do this: "you said you agree but I can hear you're sarcastic, so I'll explain again". Also getting a conversational tower to switch languages when the user does that (or when it makes sense in a language course) is not quite impossible but very, very hard. And yet feeding audio directly into Qwen will almost magically solve these problems better than I will ever do it (lightly finetuned). It's fundamentally better in every way (except perhaps speed, of course. Although arguably such a tower needs an LLM too, it'll just have way less tokens)
This point has long been passed for image recognition. Feeding a photo or screenshot into a full LLM along with a request on what to do beats the crap out of any OCR, image recognition, ... model every time these days.
And now there's also hints of world models just being bypassed. Just giving a finetuned model the last 50 video frames (surprisingly, that fits in context, especially at reduced resolution), movement statistics, motor power, status, ... of a robot and asking what to do actually works (with heavy finetuning, but I think it'll start working without). Well ... AND then distilling that knowledge into a much smaller model because B200 racks are more than a bit to heavy for a logistics robot to carry around. But we'll get over that.
Its easy to see what works in hindsight, but in reality with this sort of research you have to find an approach that’s working and keep pushing it and see where it goes.
The results so far from genie have been stunning, and its not obvious to me that another approach is guaranteed to end up better for all use cases.
yes. I feel Google's video physics has always been bad. slow, aritifical, unnatural.
Real time AV testing would be one of the better use case before AVs are released on the roads.
Gotta love the kraken destroying the city in the background of the Hong Kong lovers motorcycle scene despite nothing in the prompt about monsters. Filmmakers are still safe.
It prompts "gigantic alien tentacles that destroy the world."
I suspect you're reading the base prompt, but the video also demonstrates adding further prompts to change what is happening as the scene progresses.
I thought that was pretty cool.
But remember, today is the worst the models will ever be. I agree they are still safe though... they will probably use AI to make even more films faster and more creative since it enables them to do more.
It was the complete lack of fear or urgency on the part of the characters for me.
It was their eyeballs not drying out or squinting due to the wind for me.
In the man walking through the dressing room scene, the camera starts behind the man. And you can see that the left half of his body is wearing a grey suit... but the right half is wearing a grey dress. :O
That stained glass landscape video is pretty cool. I wonder if it would be possible to implement that effect in a real-time game engine.
What if we're all in Genie 10 right now?
Isn't that a simpler explanation of reality than the zillions of particles all existing simultaneously in this massive universe?
Some sort of simple primal generative AI construct that just feeds a conscious experience for one. You. Nothing else is real. No one else is real.
Something like that would be an infinitely simpler, than running the physics and conscious experiences of this unimaginably large universe we think we're in.
Simplicity is only relevant to humans (and other animals that evolved into trying to forecast their survival). Most of the universe seems to prefer chaos. So no, from a universal perspective, there is no reason for reality to be simple or elegant.
Really? I thought the universe boiled down to a few fundamental forces as it is.
explains how fringe details don't make sense, like no one knowing what oracle does exactly
Maybe Oracle is running Genie 10
Yeah 2020 was the upload point when we were all corralled during covid. And trees were the data centers of the dinosaurs. I mean, what if?
Not that I have zero trust in anything AI related that Google demos, but the first video example on the right, the model completely ignores inputs towards the end.
With Anthropic going public, I expect a lot of model releases from the competition signalling to the market. "We are still around, please leave some money for us too"
Google is already public, has a larger more normie customer base, many products to integrate AI in with established enterprise accounts in the millions... For some reason theyre not valued at 50:1 revenue... I dont think all that many people are going to be that excited for Anthropic as a business, Its an escape hatch, using the public as exit liquidity while the founders can still make a 100bil.
The public is the final "greater fool".
If the checker clip is meant to be used to develop a robot to play checkers, then the physics is not remotely good enough. When the robot takes the first piece, the piece looks like it's moving up from the board, then suddenly doesn't. The last move also isn't valid checkers, unless I'm missing something -- the robot offers its piece to be eaten and the human doesn't take it.
guys they are building digital cages so they can rob you blind while you are in your nice digital world, sorry we will ignore this stuff into oblivion
A few years from now, Lydia in Skyrim will have a fully consistent personality with dynamic dialogue and full contextual memories of all our adventures together in Skyrim.
A visual makeover is great but what happens when millions of people fall in love with virtual characters who seem to deserve that love?
Question is if we even want that. Video games often are escapism, we want an illusion of reality but not actual reality. I mean there definitely is a market for that, but I don't think it's for everyone.
There was a comic book about comic books I read when I was young. I believe it was "Understanding Comics: The Invisible Art" - part of it dealt with what is gained and lost with realism.
In a nutshell, (with my memory of 30-40 years), the more and more realistic a scene becomes, the less the brain applies it to ourself. It becomes this definite place, definite person, rather than ourself or our surroundings projected into other characters.
All these scenes, that I guess are amazing, they all are creepy to me.
That actually sounds implementable even now! Except the fully consistent personality part. And maybe that's a good thing. Falling in love with beings without physical presence sounds agonizing; probably better spent on real people.
I imagine you could implement the fully consistent personality by putting a recurring dream about the player, followed by waking up, in Lydia's context at startup.
skyrim already has mods for AI chat with all of your followers. I haven't tried it but plenty of people use it, (in VR as well, of course).
man it really shows some of you have ai psychosis, nobody in the gaming world wants that, that's some wizard level virginity low even for gamers
The prompts build impressive clips.
I am more interested in how to make consistent "participants" and "environment" with multiple clips so they look like one longer clip.
Will that require same dev/coding harness and spec-kit-like engineering flow?
Like create a spec on how a movie will look like, feed contexts, create tasks so that multiple clips look congruent?
I don't know if I'm missing the point, but Genie 4 almost looks worse than Genie 3 in terms of player control? It doesn't even seem to respond to inputs, it just looks like they took some AI generated videos and put the key widgets on top.
Astonishing to me that a company like Google would be willing to publish such a terrible looking product demo. Interactive, but with completely made-up inputs that are ignored?
I think you're exactly right. This was already a problem with Genie 3, but they've appear to have gone all in on video generation over controllability.
The original Genie was a great piece of work with a real sense of purpose - to enable agents to learn about the real world in the way Dreamer or Schmidhuber's paper sketched. But now most of the people that drove that mission have left Deepmind (Jack Parker-Holder, Tim Rocktaschel, Demis) and it's become this instead.
I wonder if that's because the target audience is people directing/producing (commercial) videos, rather than games.
another POV is, what even are these experiences? whom are they for? it would take Google 10 minutes to hire all the best game developers for Android, and cost them like $2m, which is peanuts, and give them fancy Google Research titles. but instead, they're trying to get 100 google PMs, who don't give their kids screentime and who don't play any games the objective of making a product for Bart Simpsons. like these people don't realize that being good parents disqualifies them from making immersive first person shooter, "narrative" experiences.
> what even are these experiences? whom are they for?
One possible use for these is to facilitate lab -> this -> real world for robot models. Basically better than lab / algorithmically designed environments and cheaper / easier to train on than real life. Something that "looks" like real life and can scale, to apply RL for robotics to.
The more parsimonious explanation is that 100 PMs are even less qualified to make surgical robots, but at least they can download a video game and play it for 5 minutes. There's a great scene in You Can See Everything where Billy Evans is watching bio presentations on 1.5x speed, which obviously, makes him look like a huge moron - "knowing enough" bio is very bearish.
Anyway, let's take your theory to its logical conclusion: the product development between real and simulated soft tissue behavior is much bigger than Exponential Time being a hypothesis and being proven. There isn't some effective acceleraltruist path here. Dario Amodei was a biotech dropout. The thing you are betting on which Google is not doing anyway, maybe they will destroy the descriptive single cell sequencing paper mill, but they're not going to be able to simulate the fat inside a BMI 30 abuela undergoing a gallbladder removal.
Sorry, I thought you were asking about the tech applications. I have no idea about the other stuff, can't comment.
Just as JPod by Douglas Coupland predicted.
When Google's epitaph is eventually written, PMs will figure prominently as the catalyst for the rapidly accelerating decline.
Why hire crew or actors when you could have them be fully AI? Why film a commercial with fluffed, oil dusted, well-lit hamburgers when you could fully render something? Why actually have a vehicle capable of off-roading when you could just AI-render what it looks like off-roading (or vertically taking off, for eg)?
It's a dystopian business use case that I hope will get obliterated in the near future as the generations that believe in this stuff and are fascinated by it age out of the population.
The point is simulation for autonomous vehicles and robots.
The models need to watch more Gallagher… cuz that watermelon smash was like a 2/10.
Can't wait for the hyper-realistic customer service bots and deepfake propaganda videos.
I don't know. With each new release of these models, they compare side-by-side how they are better than the previous ones, but at the same time I feel less impressed by each new release. Somehow they manage to be a seemingly big step forward but at the same time I feel I have seen so much AI that I just look at the common failure modes and they are all still there.
It's easy for them to show demos of abstract things because they don't necessarily have to follow real-world rules, but anything modeled after real world falls short of expectations.
I would like to see how it breaks, don't spend all your focus on how nice it looks with specially curated shots. At least give a few examples of what happens when it breaks from running too long. They mention that it can fail under "Understanding current limitations" but don't show any examples. Especially since they mention the cases of safety and responsible use, it seems quite relevant to help prepare people by showing them some examples of how it can break and so you an learn to spot it.
Their posted videos show plenty of failures. Read the rest of the comment section.
Again, very impressive from a technical point. But language is so flawed to do anything creative. It just turns out all looking the same and kind of boring
In the first comparison of Genie 3 to Genie 4 it seems that:
- in Genie 4 the character's suit stopped making sense in the back where it now has a weird fold?
- the controls don't control the character even anymore as now its controlling some kid-height character in first person instead?
- Everything looks like Unreal Engine.
This does not look good.
"controlling" some kid-height character. the button-pressing seems almost frustrated with the lack of response, long holds with no (or opposite) change and sometimes mashing. are they supposed to be buttons a person pressed to direct it? or did it create those too?
I found it perplexing how the dragons shrank in size, or perhaps the castle turret in the background didn't shrink fast enough.
Either way, the sense of believability was rapidly (and smoothly) lost.
I highly suspect that Genie is being made primarily for internal use, and access is more to gather feedback/training data and a bit of PR.
> Google AI Ultra exclusive
> AI Ultra (and thus Genie) are not available for Workspace companies
Who is this for?
Q: Is there a easy way for average people to recognize these live super-deep fakes?
I know I could run the content through a fingerprint detector. However, I'm a professional software engineer...not a normal user.
What's the ethical impact of this technology??
>Is there a easy way for average people to recognize these live super-deep fakes?
Almost all the AI gen videos I have seen have extremely weird camera movement when panning around, it usually just looks way too smooth/uncanny.
The non-coherence of the world is a pretty solid tell. Any time the camera pans away and then back, the scene changes. The model doesn't remember what used to be there dozens of frames ago and so it invents something new.
What is the point of this vs cheap robotic arm simulations?
The only place I see this applicable is for visual artists prototyping
Humans interact with their environment by subconsciously simulating it and predicting what will happen in the next few moments. This ability gives us the capacity to work more efficiently and reliably in any environment. These world models will serve a similar purpose to the AI and robotic systems of tomorrow. This is a foundational piece.
We don't have to fall off a cliff to learn to drive on a mountain road, we can imagine the consequences for cheaper. If we didn't have imagination life would be much more dangerous.
You could produce a thousand cheap ads for your product or one expensive real ad.
Podcasting is going through the thousand cheap ads with the obnoxiously inserted interstitial ads. What they have accomplished is me unsubscribing to the worst offending podcast and beginning the journey of ad blocking on podcast with the help of whisper and LLMs. Also now I hate Shopify with the heat of a 1000 burning suns due to their obnoxious ads and the insane rate they get played.
So yea that one expensive ad would probably have a positive impact compared to a tidal wave of slop. Quality > Quantity after all.
You could produce immersive VR content specified only by prompts and in which any action is possible.
Wish fulfilment gets extremely boring, extremely fast.
Video games are interesting because of very specific constraints the developers impose on the player.
The authors could still design their prompts to steer players towards predefined developments and situations. Anyway, the possibilities of something like this are infinite, the only constraint will be the cost of the compute.
The future of movies will be promising.
[1]: https://www.minid.net/2026/8/2/the-actor-was-the-expensive-p...
Thank you for that article, it was a great laugh.
You are a shell of a human being.
We’ve come a long way from simulating a Unix file system in a GPT-3 prompt haven’t we?
Amazing!
Question I have is when will they be able to render something more then 30 seconds.
And how much will it cost? There has to be a HUGE amount of computing power behind this. As a video game developer I live in fear and anticipation.
> "A REAL, ultra-photorealistic, high-end 35mm cinematic first-person perspective..."
Real nice prompting there, Google. It also still looks like a video game rendering.
ironically, the word "realistic" appears next to game footage, not next to real one.
Not ironic at all. You describe a rendered image as photorealistic, you just call a photo a photo.
That's the irony. The over prompting ends up doing the opposite thing it is intended to do.
>coming soon to Google AI Ultra subscribers in Project Genie
Would be fun to muck around with world models. Are there any that I can try for under £200 p/m without being part of a special project?
Techincally, isn't the less expensive sub for Gemini Ultra £75 p/m?
I guess they used Genie 4 to make a video about people using Genie 4 in different contexts. This may have some implications.
>audio-like music
My favorite kind of music.
A nit, but given that this is from a major corporation, and LLMs are supposed to be em-dash-happy, it's surprising (disappointing?) that the announcement uses:
> audio-like music
That's a hyphen. What's "audio-like music" -- music that is almost sound but not quite? Music made from raw audio? :-)
Yeah, I think that sentence was supposed to be: Genie 4 is natively multimodal, enabling audio--like music, ambient noises, and interactive dialogue--to be simultaneously generated with the world.
That's a proper use of a hyphen though, an emdash would be wrong.
But it's not, unless you know what "audio-like music" is?
I don't think so, and in fact an em dash would be much less confusing.
audio, e.g. music
audio — like music
Oh maybe I did just misread the intent of the sentence? I read it as "something like audio" but yeah that is confusing since that would be all music.
That's a suffix word though, not a pause.
Hyphens should be used to link compound/suffix words, not em dashes.
Agreed that hyphens should be used to indicate compound words, but you're saying that "audio-like" is an intended compound word? Then it's an adjective, modifying "music"? What would that mean?
That was my initial reading and I may have just misread it as them intending it to be compound rather than an aside modifier.
Well okay, not a "compound" per se, but it's a prefix/suffix, which similarly should use hyphens.
I think the disagreement here is not whether that’s how you should use hyphens but whether that’s what was intended at all. The term audio-like music makes no sense.
I think you're fundamentally missing the point being made here.
Yes, "-like" is commonly used as a suffix, in which case a hyphen is correct. But describing music as "audio-like" is clearly nonsense (music isn't "like" audio, it is audio), so that's not the sense in which it's being used here.
For clarity, the full sentence is:
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
Why does everyone hate colons?
audio: music, ...
No one ever got em-dash cancer…
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
Looks like a hiccup, which created an odd and ambiguous sentence.
There used to be editors who would read press releases to catch this.
Surprised that "do you really think we're going to make it?" scene made the cut of the first video, her expression and tone is totally out of sync with the scene and demonstrates the problem with almost every AI gen video I've come across when it comes to capturing humanity.
The painting->interactive scene looks astonishing though
The playable element is cool PR but the news is that endless synthetic training data for embodied AI models is basically here or close.
And their most impactful application will be training smaller models for military drones to detect, target and kill humans. sigh...
Every time I'm about to be excited about AI the cynic in me slaps me back to reality. I imagine the endless possibilities of AI for disabled people, then realize that it's going to leave millions of people disabled instead.
It will probably do both
It's hard for me to understand Google's redefinition of "world models" (basically: a txt2videogame modality for LLMs) as anything other than an attempt to bury Yann Lecun's definition of "world models", which includes an implicit bear case against LLMs and thus Google's stock price.
> … hard for me to understand Google's redefinition of "world models"
I’ve also wondered what they mean by “world model”, whereas I interpret them as being about predicting outcomes - something an LLM cannot reliably do.
Is this new model an LLM or a predictive outcome model (as you say a la Yann Lecun), or some hybrid?
Neuralink integration and I'm out of here.
it blows my mind that people actually want this. I couldn't be less interested in it.
I read a book, watch a movie, and play a game, to get the point of view and creative decisions by the people who make them. I don't want shadowy projections of my own mind and whims creating the experience. I can conceptually understand why some people may _think_ they want a "genie" that just grants every wish, but my suspicion is that much like other slop deluging us, they will quickly see that just getting whatever you want doesn't make interesting or good
Google aren't the only ones with this tech. You can try out a 24/7 rolling set of video feeds for yourself at https://fal.live
An LLM generates four alternative prompts for what could happen next and there's a real time vote running for each option. It's trippy, like watching someone's dream.
Showing off an anglo-bias with the Japanese train station in the 1980s, with a man speaking english.
Can someone ELI5 how this "Leads to AGI"? I can kind of understand some sort of "theory of mind" going on here, but how would having a complex world model enable, say, an AGI that could file my taxes for me?
AGI is a loaded phrase these days. But if we focus on how it will improve the quality of training data for LLMs, I can prob talk about that.
Right now LLMs only have text, images, and videos to train from. Anything it understands about physics is from an abstract level. It's like learning how to calculate throwing a baseball before you've ever even thrown a baseball. Imagine having to learn force = mass x acceleration before you've even picked up a ball.
What this world generation Google is doing appears to be something Nvidia has spoken about too. Letting LLMs/AI learn from a world physics simulation. Meaning LLMs having the opportunity to "experience" physics, the same way you intuitively know how to throw a ball after attempting a few times yourself.
If an LLM can better understand how our real world works the quality of its output improves -- theoretically.
As far as your taxes go, you can create an intelligent workflow right now to do that for you. Don't need AGI for that.
Infinite Tsukuyomi is coming soon…
It looks sooo boring and generic that it's hard for me to imagine anyone thought https://www.youtube.com/watch?v=sFfPkUJd_Jk is a good showcase.
what good in the world could possibly come from this being available? did anyone working on this ever think about that?
The creation of shareholder value. Much has been thought about it.
So world models capture inherent complexity of physical reality to help teach AI about hidden laws which probably helps train both LLMs and multimodal models used for autonomous vehicles and robots? And AMD acquiring World Labs is a move to outdo Nvidia’s work in this space. And Google probably sees this as a gateway to a new platform of autonomous vehicles and humanoids that they could rent out as a CSP? Competing with Nvidia, Tesla and complementing Waymo. Am I getting that right?
That all sounds right to me, but apparently they can also be used for simply generating videos, or "unlocking the creation of new interactive experiences"... which both seem like competing goals with everything you just mentioned.
all the things you've dreamt of - your vivid dreams - now you can replay them.
for all the a.i misery with hardware prices i.e ram.
such a wonderful world we live in now.
again only available in trusted research preview.
If only there was some way we could create some sort of arrangement of 3D models and place a virtual camera within the arrangement. We could introduce some physics simulation and make the position of the virtual camera move and render what it sees to the user.
If such a system were possible, it seems like it would be far better at consistency and realism, since it could be fine tuned and hand crafted to best represent the desired world. It could be used to make interactive experiences, games, perhaps even feature films!
Alas.
Honestly MiniMax H3 looks better than this at most of the things I've been able to generate locally. These look super AAA game like - movement is uncanny and feels like it was trained on GTA3 or something like that - is this just me?
Wow, wow! What a treat! Such a fascinating time to be alive. Talk about fast feedback loops! This is pretty quick loops for trying out ideas!
I'm very impressed!
The third video is proof that v4 can, ekhm, walk slower?
WERE NOT IN A SIMULATION! YAY!
This should not exist.
Why do we need this? No thank you. We don't need flying cars and more vendor lock in. We need to serve the current google within constraints. That should be driving direction. Being a better ecosystem member. Another fascist product from google.
Good moving toward open world:)
I am again so disappointed with the discourse about AI in these threads. It rarely if ever focuses on anything remotely technical. People just trot out their agenda and their opinions (much like what I am doing now). We are building civilization level tools and the most technical analysis I see is that the blog post announcing it was written by an LLM (it wasn't) or that an ogre riding a unicorn into the clouds of Venus was missing a claw (wth?).
Yah, meta is discouraged here but why are we as a community putting up with this level of meta disruption? I guess I'm supposed to adapt to the new community but there's nothing there there.
I will stick with the more technical posts here and avoid anything to do with a new AI model and the complete and shocking thoughtlessness of the commenters (most of them).
[edit: grammar and spelling]
This is part of analyzing model output against others and known capabilities. The Orge riding a bike is just a QA test. Maybe you can work on civilization level testing tools.
I didn't see any comments related to that - now I understand. Your reply does not seem to be in good faith, then you pick on the poster not the message.
Who wants this
Is anyone actually making any games with these tools? Like, can I play one on Steam or something?
Why are they plowing money into this garbage? I don't understand it. I guess it is because Google is lagging behind in the AI race but the roi of this bs must be close to zero. Clowns.
Looks cool....
So if those genie 4 characters have all individual chain of thought reasoning and persistent retrospection / recollection, how do these human characters know they are not living in a simulation made by biological humans? Not sure you understand what i'm getting at. Some ethical questions do arise.
Failure modes?
Show them or I'll assume this is pure marketing.
You know you can already use previous versions of Genie and they look pretty damn close to this right?
The implications for what we now call video games are crazy.
It’s going to be bizarre when we’re all playing slightly different versions of the same game together.
It could be even weirder. We might be playing video games that are essentially built from the ground up by and for the individual gamer, such that the concept of a "game", let alone the same game, doesn't even make sense. Same for what we have called movies and TV for the last 100 years.
You can already go and generate a book to read or a song to listen to, no one does this, I don't know why you expect it to be different with video games. People want to consume art, they don't want to make it.
You can't just remove the societal/cultural aspects of art without neutering its emotional appeal and impact. Ripping all humanity of out movies/video games is not going to result in higher quality art.
You're getting awfully close to telling people what the should and shouldn't enjoy. Ultimately, this will be decided by the market.
im realy excited for this
If it’s not open weights, no one should care.
It doesn’t matter how impressive your demo is if it takes a literal data center to run.
Actual innovation can’t be seen at the let’s just throw money at it level. But, it can be seen at the that’s cool. I can run it on my laptop level.
Hi, I always loved watching Pixar movies, partly because I knew that what took them days to render (in their literal data center) would eventually be done in real-time on my desktop.
In the same way, seeing that something is possible gives hope to the people who are making those open models that you want.
What the fuck are the little wasd/arrow keys supposed to mean? They have no relationship to the movement of the character or the camera. It's just flashing buttons.
Why can't Google just announce something and release at the same day, not always this stupid wait list stuff.
wait list, then it releases, and you get 10 api requests *per day*.
Until you've spend $100-150 on their platform. Then you get 250 per day until you spend $2000 and it becomes properly unlimited.
It is really annoying. I'm literally trying to pay you for a service and you won't let me.
Prompt: make a press release that isn't a slideshow on this company's own browser and hardware platform.
Who'd be down to write a science fiction novel where creating fake images is a crime and not a product?
I guess we'll find out when they publish it.
Hmm, wonder if Databricks trademarked their Genie AI product.
So does this export 3d assets or an Unreal/godot/unity project so it can be turne dinto a game or is it limited to their runtime?
"Building world models responsibly"
... I'm suddenly reminded of Iain M Banks' "Surface Detail".
Impressive slop. But slop nonetheless.
supper cool
But to what end? How does this help humanity? How does this help us encounter truth and beauty? Whether you look at this from a utilitarian angle or from a transcendental one, it is a collosal waste of time, energy, and money.
“audio-like music”
What is this for? I cannot understand these models. They are very much polluting the internet with nonsense and use capacity that people and businesses need for actual work.
If these are the cause of why I have to pay 2-4x I used to for a computer, then Google should go fuck themselves.
i am starting to get model fatigue..
this is the gta6 I am was hoping for
Not really impressive at all at this point. That dragon clip was pretty awful. If you told me this page was from 2024 I’d believe you.
pretty cool !
why does this need to exist?
From the auto-generated captions on the YouTube video on this page.
> Genie form [sic] can simulate scenarios
It's kind of amazing: the juxtaposition of such advanced generative AI and such terrible speech-to-text coming from the same parent company.
The volume of content on YouTube is enormous, I’m kind of surprised they offer auto captions at all given that they have little to no competition in the space.
Is it surprising? Indexing for search is the backbone of Google's business model.
It's why if you search a practical task you will often see results link to specific parts of a video.
Ok but like, surely the Deepmind team could sneeze and get flawless captions for this video. Why rely on the default YouTube captions instead of just attaching correct ones? It seems silly.
Yeah if they cared they could create their own captions for sure. They probably didn’t even think about it. Maybe some intern will do it after reading this thread
They're not proof-reading the blog posts either. The second sentence:
> Genie 4 is natively multimodal, enabling audio-like music, ambient noises, and interactive dialogue to be simultaneously generated with the world.
(Emphasis mine). Like, yeah, I'd sure hope the music is similar to audio.
I wonder if this was a de-LLMese pass gone wrong, and the original was "enabling audio -- like music, ambient noises, and interactive dialogue -- to be simultaneously generated with the world.
When you're high on vibes, the line between music and audio-like music tends to blur.
These days, unproofread text adds authenticity.
Only when it's a mistake that a human being would plausibly make.
Typos mean the text came from a human.
Looks like we are inching closer to on the fly open world game creation soon. While the updates excite me as a senior data scientist but as a human they all just look scary. Just me?
"Inching closer", yes. "Soon" seems presumptive. FSD everywhere seemed "soon" 20 years ago.
As I said before, World models will become simulation engines of AAA games where there is no need to release the actual game artifact and even if the game exists as a binary artifact, it will never be released and will be available on their hosted cloud back-end, then simulated and streamed back to you.
So, Stadia + World Models = New generation of DRM for games.
Why? Why would a non-deterministic engine be better for a game when it can be instead structured and made deterministic and then more effectively distributed to consumer hardware?
It wouldn't be better for players. It would be better for companies because the user no longer owns the computer that runs the game.
You can still stream a deterministic world to the player.
And a month or so of development time. Pure IP re-selling.
October? Yeah, just release the "Spiderman saves Halloween" prompt.
This is a wet dream for big studios.
it's not, it just sounds cooler to say prophetic things like this these days
Game industry is still struggling to find a way to incorporate regular LLMs without suffering catastrophic edge cases. A long way to go before these can do more than fancy tech demo, I think.
Because it wasn’t bad enough to need a $5000 5090 to play a game, now you’re going to need a full rack of them!
They can try, my household won't be purchasing those.
We'll be riding the huge wave of indie titles that are incoming.
> We'll be riding the huge wave of indie titles that are incoming.
The problem will be separating the wheat (people with good stories and concepts enabled by Ai) from the chaff (people with no taste slopping out trash with Ai)
Yes, but for me the discovery of treasures is part of the fun.
I also love thrifting and artifact hunting.
Isn't that what streamers are for?
I mean - reviewers!
streamers and best-in-genre videos have become my primary filter tbh
been checking out the archives lately, nothing recent excites me
None of this makes sense. Think of the economics involved here: is it cheaper to run tens/hundreds of thousands of GPU clusters to simulate a unique game world for millions of players, or would it be cheaper to just write the god damn code once, and then host it for download, or to allow them to be streamed?
Because stadia was a real banger right?
Do you have even the slightest idea of the level of expectation that even the most casual gamer has?
This isn’t achieving much beyond prototyping. They’re burning so much money for no reason.
And? Just spin up some open world model and pay for the compute and play your own games?
There are so many options for games now that just sticking with the biggest AAA companies seems like a waste.
Stadia is dead, the reason is playing games over the internet sucks, input lag and serious graphics degradation - this is why I cancelled
Ai generated games suck even more, no skill; no taste, there is no TLC put into them
Then there is the economics of it, cost prohibitive to (re)generate a scene and ship that over internet pipes
Stadia is dead because if you have a data-center of GPUs you are using them for AI. It has nothing to do with input lag and graphics degradation.
Oh no my guy, you are misremembering, stadia died a quiet death long before AI.
The latency was no good even on fast internet for even slow games. It is and was pointless.
Stadia was pretty good, be it OnLive was similar quality in like 2010. It even had the best version of Cyberpunk 2077 when that launched. But the model was free version of the service for a limited resolution AND you have to buy games there OR a subscription based option and you still have to buy games there.
That made no sense and wasn't a viable business model.
stadia died way beforehand, history is useful
https://www.denofgeek.com/games/why-google-stadia-failed-shu...
I am sure the are other typical Google fumbles that would have meant they were not the victor; but I am sure an equivalent service would be mainstream today if AI had not dramatically changed the economics. [also Google bought DeepMind in 2014; 'way beforehand' is and odd way to look at it]
I disagree, dropping below 60 FPS will be noticed and turn players away, you are not getting that with good input handling even today, it has nothing to do with GPUs and everything to do with networking
> beforehand...
is about the craze, the ChatGPT moment and beyond, when GPUs demand went exponential, not when various types of Ai algorithms were available
It is about when Google realized GPUs were about to go exponential; they may have missed some opportunities but they were well ahead of the curve.
sure... despite all the other people who used it and are saying otherwise, and the fact that google totally fumbled in the Ai race as well as built custom silicon for Ai...
... history is helpful ... google didn't - and still doesn't - understand gamers, same for all the other Ai hucksters, gamers are among the most anti-Ai
https://www.youtube.com/watch?v=c1Fv1uKTd-w
I used stadia and it was a great experience, latency wasn't an issue at all.
Geforce Now is still a thing
And nobody will play that trash, and the world won't change.
Also, lmao, they're already trying to nickel and dime you and sell you microtransactions that took 5 minutes to make for 10 bucks. If you think they're going to pay the costs for running a world model for every single person, lol.
Y tho?
Every increment of advance in this field is an increment of evidence that we are simulations.
No it doesn't, not even close. You are looking at a video represented as pixels changing on a flat screen. None of it actually exists. You exist, the computer or phone in front of you is composed of mass and energy, and absolutely none of it is algorithmic. World models, as complex as they are, are still just pushing numbers around on paper.
Even if we can show that it is technically possible to model a universe like ours to the degree that our current perception of reality suggests exists, there are three other big factors that would need to be weighed:
Is it feasible? What Dyson sphere level of power would be needed to maintain such a simulation? Is that level of energy available to whomever is running the simulation?
Is there a positive ROI? What does this advanced civilization get out of running such an expensive simulation?
Is it moral? The simulation involves sentient creatures who are capable of deeply contemplating their own existence, capable of feeling rapturous pleasure and indescribable pain and loss, and ostensibly capable of making moral choices that range from heroic to depraved. Doesn't there come a point where the beings in a simulation are so close to the beings in "base reality" that the same moral principles apply? If you press a button that kicks off a simulation where humans can torture and murder each other, behaving essentially identically to humans in "base reality," aren't you in some sense facilitating those deaths?
> What Dyson sphere level of power would be needed to maintain such a simulation
Why would you assume energy is relevant to whoever turned on our simulation? If I created a simulation where the only source of energy was cotton candy, my creations might ask "but where would our creator even get enough cotton candy to run a simulation like this", not realize that it's absurd to think that the creator of a simulation has any of the same constraints that exist inside the simulation.
You basically described the plot of World on a Wire.
You run it to see if your military tactics will defeat the enemy and then you keep running it until you find a strategy that guarantees victory. Who cares if some sim lives are churned through the simulation, our lives are at stake!
Reductionist but I feel like that’s how it would go.
How does that argument work? I've seen more than a few advances, yet I don't see how any of it provides evidence that we are simulations. This might be one of those things that comes down to the definition of "simulation" I guess.
Not sure if you're joking, but the only such evidence I could think of would be something like finding some way to influence the hardware of our simulation and seeing a glitch in our world
Who is asking for this? Do advertisers really want this? I feel dread about many of the AI products being sold. They're mostly tools to help funnel money from the users to the bosses.