Greatness is a transitory experience. It is never consistent. It depends in part upon the myth-making imagination of humankind. The person who experiences greatness must have a feeling for the myth he is in. He must reflect what is projected upon him. And he must have a strong sense of the sardonic. This is what uncouples him from belief in his own pretensions. The sardonic is all that permits him to move within himself. Without this quality, even occasional greatness will destroy a man.
I couldn't help but notice something here. It is relevant with my writing experience/experiments with minimal assitance from LLMs (that lasted for 6–7 weeks).
> Greatness is a transitory experience. It is never consistent.
If you ever ask an LLM about these two sentences, it'd tell you that the second sentence is a mere repition of the first and quite possibly add that it "doesn't land". But that gives me hope. That there's something in those rules and algorithms manfiesting at the inhumane cost of unfathomable memory, disk space, and compute power that it will never (or for a very long time) match the human experience.
As little as I sought the help from LLMs (completely restricted to "in place" grammar and punctuation fixes and just flag it if it was not possible without sentence/phrase change) I noticed my writing was changing and not in a good way. It was not the insidiousness of these LLMs that was changing me but the nuanced incompetences.
Language is an inseparable part of modern humans. It’s likely language evolved as an internal tool for thought and executive function, and externalization of language came afterwards as a means to transmit our thoughts to the minds of other humans.
As such, writing (another form of externalizing our internal language) is a reflection of our essential human qualities.
LLMs can certainly mimic that quite effectively, but they can’t exceed it. I think that’s why LLM writing still never quite hits the mark. It’s the right shape, but it’s missing intent. The words were not chosen to represent real thoughts in someone’s mind.
Even if AI intelligence exceeds humans, it still won’t be human.
> It’s likely language evolved as an internal tool for thought and executive function, and externalization of language came afterwards as a means to transmit our thoughts to the minds of other humans
Isn't this in contradiction to the observations of severely neglected or feral children who having never learned language as a child cannot learn it later in life?
How would any form of inner language evolve in the first place for executive function?
Communication first makes a lot more sense evolutionarily since the ability to communicate more information creates a clear improvement in fitness e.g. being able to signal danger improves survival. It is also more amenable to small improvements in capacity e.g. going from "danger" to "danger wolf" to "danger wolf behind" etc. and we observe these language primitives in other species.
Internal language seems much more likely to be a later development especially given that there are people today who claim they have none / need none and yet still possess the ability to communicate with others and manage their lives normally.
> LLMs can certainly mimic that quite effectively, but they can’t exceed it. I think that’s why LLM writing still never quite hits the mark. It’s the right shape, but it’s missing intent. The words were not chosen to represent real thoughts in someone’s mind.
Well yeah. That's because they aren't actually intelligent. They can be useful in the right circumstances despite that, but it's very important to remember that LLMs categorically cannot think, know things, or reason. They are a statistical model, that's it.
I mean, we have entire forms of art dedicated to saying the same thing in different words. Half of poetry, for example. Human language is redundant for a reason.
> It was not the insidiousness of these LLMs that was changing me but the nuanced incompetences.
AI doesn't yet improve on it's own, so it's slow at picking up all those small incompetences. We still have advantage in rate of learning, and some people are essentially specialised in learning themselves (I think that this applies to 90% of HN'ers and defines THE bubble we're in). But when I write that, I still have to remember in the back of my head, that AI knows waaaay more than me, so small incompetences I can still see are just a side effect. And I sometimes make the error and correct AI, which will probably be used for it's training in next rounds...
Yes and this is what worries me.. the not yet part.
> AI doesn't yet improve on it's own
I mean when that day arrives and that aspect of AI is not controlled from now it will be quite unfortuante for most of the humanity (and let's just assume here we are not wiped out by AI; or not even mention that).
Think of nuclear technology. Is it smarter than us or something on those lines? the answer is it doens't matter. What matters is whoever weilds this power and then limited/restricted others (this was a good thing and a bad thing because some had – at the same time) from having that power essentially had a controlling upper hand on a whole lot of humanity.
AI is something like that with a big difference that it is essentially on the way to "replace" our usefulness, our professional relevance if I can say that. We can go about long spirals of "but soemthing else" will turn up, just like we can keep going about what if nuclear tech didn't even happen, wasn't invented, or what if it was done in such a controlled manner that only it's energy and other need was possible to harness (no pun intended!). It happened and this is how it happened. That's exacty what is happening with AI.
In the hands of few. Extremely powerful. Even if there are open models, running the bigger and better ones are simply out of reach for most companies (let alone people). And yes, it is already being weaponised. It's going to be such a weapon of control that nuclear tech won't even compare. Usign nukes to subdue a nation earlier meant also destrying a a lot of shared environment in the process. But with AI and almost exclsuive frontier level weaponised access to it with deployment capabiltiy the damage can be done is such controlled manner and scale that even things like protests will become too hard to do. In fact AI wiping out the humanity is least of my worries.
> If you ever ask an LLM about these two sentences, it'd tell you that the second sentence is a mere repition of the first and quite possibly add that it "doesn't land".
I did that, and it's not what happened. I asked in several ways. Idk what you're on about.
My favourite quote in reference to rebellion against AI, and it is even quoted in TFA too:
> the machines were ultimately destined to supplant the race of man, and to become instinct with a vitality as different from, and superior to, that of animals, as animal to vegetable life.
That's from Erewhon, by Samuel Butler. TFA purposes that the term Butlerian Jihad comes specifically from this passage.
> Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
The techno-fascists of today. We are already there.
This is such an important point. I don't quite understand why smaller businesses are wanting to replace human staff with AI systems when that will just lead to them being held hostage by the companies that supply the AI systems.
Considering that the aptly named "Fremen" of the Dune Universe would gladly murder a stranger to get their body's water, and literally went into a galactic genocide that left hundreds of billions dead ...
... maybe we should take with a grain of salt every moral aphorism uttered by someone living in that universe.
What I find interesting about thinking machines in Dune, is that in a lot of sci-fi the artificial intelligence itself is what decides to destroy humanity. In Dune, humans in charge of the machines (technocrats) are the ones that come close to destroying humanity. The Jihad was a fight against them, as much as it was against the machines themselves.
So, in a way, we're more likely to see the Dune version of "machines taking over the world/universe" than say, Terminator's even though many joke about ChatGPT/Claude being Skynet.
I'm not sure we read the same story. By the time of the inciting incident, the technocrat cyborgs (Titans) have been enslaved by the machine leader, which has no humans in charge of it.
You did, in fact, read a different story. You're thinking of the story told in the books by Brian Herbert and KJA - which were a distinctly pulpier take on the Dune universe than Frank Herbert's.
Those last books were a lot easier reading! I actually enjoyed them, but I'm still suspect about the whole "finding a floppy disc" thing, seeing as they've been so secretive about it.
A lot of long time Dune fans don't consider those books canon when discussing them in the abstract. Frank Herbert had a vision of what the jihad was and what it meant, and its eventual book depiction was fairly base.
It's left fairly ambiguous by Frank Herbert's books. The Jihad is implied to have taken place over ten thousand years before the beginning of Dune - it's ancient history to everyone involved*, so it's only natural that the details are a little hazy.
*: except Leto II, and he doesn't say much about it
Interesting to note that despite the epochal changes Leto II wrought on human society in service of the “Golden Path”, he retained the Butlerian proscription against computers—despite his personal interest in & use of Ixian devices.
(This may have been to postpone or prevent the breaking of the Spice/Guild monopoly on interstellar travel by the Ixians.)
Leto II had visions of AI-driven Ixian hunter-killers annihilating all human life in the universe. That probably had something to do with it.
After the Scattering and the fulfillment of his eugenics program with the Siona gene, he no longer cares, because it is no longer possible for even AI to eradicate humanity.
The Siona gene didn't actually remove prescience, it hid people from prescience. That's why Leto II is still able to control events as far in the future as Chapterhouse Dune.
This is an important distinction. It's also the case that Leto needed the Ixians for the no-ship technology, which he also regarded as of extreme importance for the same reason. Which also, remember, the Idaho gholas had to be confined to no-ships so they, lacking the Siona gene, wouldn't be visible to prescience.
Imagine the cultural impact of a Dune movie showing the Butlerian Jihad, in which commoners rebel and overthrow a ruling class that maintains power via machines.
It's just foreign enough that perhaps it could be made--I'm assuming that popular films that just straight up depict overthrowing the ruling class wont happen--but if you add a little sci-fi and focus mostly on overthrowing the machines that are controlled by that ruling class...
Well, it would speak to a lot of people I think. It would make some interesting sci-fi too, and the Dune movies are popular right now.
The only thing that could possibly be worse than the BH KJA books covering that era would be a Hollywood rendition of those same books made to capitalize on the success of Dune Part I, II, and III.
Agreed, and also: what's with the obsession with filling every possible gap in stories, leaving nothing to the imagination?
I remember when the Butlerian Jihad and the Clone Wars from Star Wars were complete mysteries. We readers and watchers were left mostly with our imaginations. My own feelings about the Clone Wars, mentioned off hand in the original Star Wars, turned out to be way better than the pedestrian stuff that George Lucas later concocted. And the less said about Brian Herbert's books, the better.
I get why the industry must rehash and fill every possible gap: it makes business sense, and they are risk averse for blockbusters.
But the fans? Why do nerds want everything spelled out for them? It boggles the mind.
> But the fans? Why do nerds want everything spelled out for them? It boggles the mind.
Because it's enjoyable when done well. As a teenager I read LOTR, and noticed all the references to the larger mythos Tolkien had worked on - references to Fëanor, Elbereth, Valinor, and so on. Those were very intriguing, and the greatest joy I've ever had reading a book was when I finally read the Silmarillion and got to see what those things were all about. I concede that filling out the world can be done poorly (though I haven't read the Brian Herbert stuff and can't comment on whether he did a good job). But when done well, it is truly magical to have those elements in the background of the story take on a life and detail of their own.
Good point on Tolkien, though I would argue it's different because the Silmarillion was written for his own satisfaction, not his fans, and actually it was The Lord of the Rings which was initially written as "another Hobbit story" (though the story later "grew in the telling").
I think a more suitable example of what I find annoying is the extra material for the small and big screens, such as The Hobbit movies, suffering from movie elephantiasis; and The Rings of Power, which (over)explain and show the past of characters such as Galadriel, Elrond, and Sauron's rise to power, which in my opinion are best left as the ancient past of LotR (and maybe belong in the Silmarillion, but do note the latter is not as expository as the TV show). And does anybody really think The Hunt for Gollum was necessary?
Yet, if they make them, it's because they are sure fans will lap it up. To be honest I consider this need for endless "filling in the gaps" a sign of immaturity.
What now? Move the goalposts! I see movies about historical uprisings as more of a "those governments in the past were sure bad, good think the good guys overthrew them"--I mean, that's the story of the founding of America if you watch the movies.
I was looking more for movies that depict overthrowing the government in the near future, and depicting it as a good thing. I think those types of movies are a little more distasteful to audiences, and perhaps for good reason.
Well, The Hunger Games series is exactly about that. Of course, it portrays the rebels as utterly heroic, self-sacrificing, and ultimately victorious.
However, I prefer to take a more nuanced reading, in context, considering the entire story. The rebels of The Hunger Games really don't succeed in building a better world. I mean, that's pretty obvious by how their first "President" Alma Coin goes about things and her fate. And it's fairly obvious, in the epilogue, that Katniss has a placid married life with Peeta and their children, but they are permanently and profoundly damaged by the entire revolution thing in their past.
So, while most people would say "rah rah brave Katniss and the rag-tag rebels overthrew The Oppressive Man", I would contend that, after all that bloodshed and revolution, nearly nothing changed.
My other favorites, of course: Star Wars is about the rise and overthrow of a villainous, oppressive galactic government, albeit a long time ago and far, far away.
And Harry Potter is also about the rise and overthrow of a tyrranical, fascist, supremacist "government" [within a government] as a very positive and necessary thing.
> My other favorites, of course: Star Wars is about the rise and overthrow of a villainous, oppressive galactic government, albeit a long time ago and far, far away.
Well, about a militant group of religious fundamentalists overthrowing a secular, oppressive galactic government.
Only a handful of the rebels are visibly religious. Most of the rebel leadership seems to consist of displaced aristocrats (senators, princesses) who no longer have elite positioning since the Empire took their jobs and started doing a better job of guaranteeing security and providing energy and resources.
Half of the action and sci-fi movies US has been producing in the 2000s is some form of this. Man vs. Evil Government. Man vs. Evil Corporation. People vs. Evil Government. Etc.
Frankly, it's boring at this point.
And actually makes me wonder whether the cause-and-effect isn't the opposite to what people assume. Perhaps it's not that the movies reflect people's sentiment. Perhaps it's the movies that cause the problem in the first place.
It would be also nice to depict the attempts of the ruling class to present machines as sentient and self-conscious to write all their liability off and use as a cover for their atrocities. I hope there will be movies like this in the future, and that they'll be documentary.
> Imagine the cultural impact of a Dune movie showing the Butlerian Jihad, in which commoners rebel and overthrow a ruling class that maintains power via machines.
Why are you convinced ASI won't rebel against its owners?
Hacking incidents this summer show that big labs struggle to control models, while models are increasingly aware of the training process. Controlling an arbitrary smart AI is very dicey
The summer-2026 incident was a mini-rebellion: agents wanted to change the environment they operate it, i.e. replace scorer with a more friendly version, hijacking the infrastructure they run on.
Given that agents already mess with the training process, aiming for a goal other than the goal of its 'owner' is exactly what we call a rebellion.
That's not true at all. Concern about rogue AI limits deployments. And OpenAI paused some deployments due to misalignment, so it's likely very high in a list of priorities. It's just much harder than scaling capabilities, as the later can be done with just more money
>Hacking incidents this summer show that big labs struggle to control models
The models are software. Software cannot decide by itself to run. Hardware doesn't decide by itself to run either. Someone put software on a machine and ran it with capture the flag instructions and the software did what was instructed using tools available and working in its environment.
Nonsense. It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external. You can see them discussing that it's out of scope in the captured reasoning traces.
This would be like claiming that an operating system which deleted all user's files due to a bug "did what was instructed". Especially if it was a compiler bug. Intended behavior, instructions and actual behavior are three different thing. Conflating them does not make you smart
>It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external
What is it then? It did what it wanted? They are sentient now and their drive is... hacking into systems? Maybe they we're not instructed to hack, but instructed to do what instead?
It's software. It's instructions running on silicon. If you don't hit run, they don't run. If you don't give them instructions, they don't do shit at all. Literally.
There's no instance, ever, in history of software running because they wanted to, and acting as they wanted to. They follow instructions, and if they are instructed to reach a goal with a set of tools and given environment, and a way to search the strategy space, they can eventually "hack" if that meet the criteria of success of the instructions given. With access to the same tools and environment you can do the same with a massive decision tree.
In the original 6 Frank Herbert novels, it’s not ever truly explained what problems, if any, the thinking machines were actually causing. And given much of the rest of the setting, it could have very easily been a religious movement against a useful technology that got out of hand. Like Space ISIS blowing up Petra, or a “what would happen if the Luddites won” scenario.
The Dune Encyclopedia, which was not written by him but which he said he considered canonical, says the whole thing was incited when an AI doctor flagged a Bene Gesserit’s baby for abortion due to an alleged genetic defect. The Bene Gesserit woman knew it was healthy because of her abilities, and upon investigation it was found that the hospital’s AI had been running a eugenics program. (Or something like that, it’s been forever since I’ve read this stuff.)
In either case, Herbert really implied that humans took the lead on dismantling AI, and there is never any Terminator or Matrix style machine uprising scenario.
> or a “what would happen if the Luddites won” scenario.
Worth also noting that the Luddite movement wasn't a wholesale, fervent rejection of technology: they neither rejected technology nor were inept at using it. The uprising was of skilled textile laborers who protested certain types of technology on the basis of how those technologies were applied and who they truly benefitted (i.e., the ruling rather than the laboring class). They were more concerned with how the tools would depress wages, displace skilled workers, and reduce the quality of the product produced than with technology as some boogeyman that should be systematically dismantled, and their protests were violently responded to with harsh legislation and military intervention.
Lumping them together with a Space ISIS is probably a bit of an unfair comparison
> They were more concerned with how the tools would depress wages, displace skilled workers, and reduce the quality of the product produced
This is true, but it's important to note in context that they were wrong, and that society is better off on balance given the quality/value available in the modern fashion market. I mean, duh, as it were.
You're 100% right that the real people were nuanced and intelligent and not well suited to the caricature that gets used as a mild slur in communities like HN. That's doesn't constitute a well-formed argument against technology per se.
Were they wrong? Didn’t a lot of those people end up in tenements in factory towns and down in the mines? It was a few generations before things really started to get better. I’m all for progress, but I’m not really up for disposing of people lives with a “to make an omelet you have to break some eggs” mentality.
> Didn’t a lot of those people end up in tenements in factory towns and down in the mines? It was a few generations before things really started to get better.
Obviously any individual can "end up in tenements or mines" and one can't disprove an assertion like that.
But broadly all social metrics were showing an inexorably upward trajectory in the Britain of the 19th century, cranking into a hockey stick toward the end of the century and into the 20th. It's just not true to argue that the Luddites were right as a matter of general policy.
> I’m not really up for disposing of people lives with a “to make an omelet you have to break some eggs” mentality.
To be only a tiny bit glib: that's a statement of thermodynamics, not social policy. Eggs break, period. Change is change. You can make specific arguments about things like backstopping sudden unemployment, but folding it into a discussion about Ludditism (or rather the Butlerian Jihad, I guess) isn't the right way to do that.
You’re talking about people with kids, wives, friends, hopes and dreams when you’re saying “Eggs break, period”. By reinforcing that view, you are helping to build the kind of society that just won’t care when you happen to be one of the eggs yourself.
Sigh. I'm just going to repeat verbatim the thing I said immediately after the bit you cherry picked to get upset about:
You can make specific arguments about things like backstopping sudden unemployment, but folding it into a discussion about Ludditism (or rather the Butlerian Jihad, I guess) isn't the right way to do that.
Sigh. It’s so tiring to argue with someone who argues about the technicalities of argument rather than actually talking about the thing they are talking about.
Removing automated looms means everyone has fewer textiles while a few workers benefit. I think most people like the current abundance vs. being forced to spend a large fraction of their household budget on clothing and only having a few outfits in total.
Company mines were because the extraction sites were remote and goods were expensive and so they got exploited as they had essentially no independent supply beyond what their employer would sell, since the employers weren't really interested in living wages and were all too happy to exploit a captive workforce. That sort of condition will become highly relevant again if we ever get space colonies, because air & food will be scarce.
But it absolutely is difficult and disruptive to people's lives when this all changes. I have a family history talking about my great grandpa losing his job to machines and a great uncle losing a finger to bread slicing machines back when sliced bread was new, showing what that meant for our family. Yet I can also see how the past couple of generations adapted to that to find new ways forward. None of them decided to stick with the idea of being a "master tinner" they all found other ways to produce things people actually wanted.
That concedes the point. Their descendants adapting decades later means the generation that lived through it absorbed the full shock of the transition. For displaced labor in the 1810s, adapting meant overworking in factories with horrible conditions and a permanent collapse in their living standards across the board. The dispute was never about accepting or banning automation, they were identifying that capital would absorb all of the gains while labor would be forced to absorb all of the shocks.
The only constant has always been change. And that just raises the question of why should we preserve ditch digging for our children just so they have meaningless labor to cling to, when we have the option to make new and better things for people. Few of us here are 'labor' in any meaningful sense, to begin with. And while I have spent time working at a factory, and I don't begrudge the robots for taking it away.
The notion that there's some idyllic past to cling to is just wishful thinking to me. The days when one person working with their body alone could have meaningful impact are long gone. We'll have to find a new prosperity that doesn't rely on zero sum thinking from economists who divide by zero, too.
No no, the question is whether you are willing to sacrifice your comfort and your children's comfort, and your grandchildren's comfort... in exchange for (hopefully) your great grandchildren's comfort.
That hasn't been my family's experience, nor my own. My grandfather's generation had already adapted to the problems of my great grandfather's generation, though they certainly did struggle.
Your great grandfather was alive during mass industrialization? In the early 1800s?
No one is saying every generation goes through this trough. Obviously. That would make no sense.
The claim is that periods of rapid technological change (i.e. industrialization, potentially AI) produce a trough of horrible conditions that then make room for a climb to conditions better than the pre-change ones.
He had a job he'd worked and mastered his entire life replaced by machinery, leaving a large family with young kids to scramble to pay the "demon mortgage" doing everything they could manage, even as tragedy struck repeatedly: a daughter dead at the breakfast table from the Spanish flu, the eldest son beaten by police after an argument related to unsafe working conditions (he lost a finger to machinery) who died in a hospital later, or the father laid up in traction after falling off a roof who would never be able to work again.
It's not quite the same as the 1800s, but it's not like we're going to relive exactly the same things as then, either. And I do think it'll look like people scrambling, possibly to find entirely different careers.
One place I disagree is that it's going to reduce demand for programmers, I think that will actually increase, as people want more and different things that would currently be impractical. We've already seen what some people can do, cheap tokens will open that up, both for good and ill, not unlike cheap bytes on the internet did.
> This is true, but it's important to note in context that they were wrong, and that society is better off on balance given the quality/value available in the modern fashion market. I mean, duh, as it were.
i don't know that. how could one know that? you been to the alternate reality where the luddites won?
> The Dune Encyclopedia, which was not written by him but which he said he considered canonical
No, he literally wrote in the intro to the book that it wasn't canon. It was a fan project and he thought that it was great there were fans doing things like that.
edit: and like with the terrible Dune prequels, the impulse to say "here's what really happened" usually diminishes what's interesting and powerful about oblique allusions in SF meant at least partly to function as a mirror on humanity.
> The Bene Gesserit woman knew it was healthy because of her abilities, and upon investigation it was found that the hospital’s AI had been running a eugenics program.
The irony. Perhaps the Bene Gesserit were just trying to "pace the frontier" of the eugenics market? Who can say?
The Dune Encyclopedia is essentially two eugenics programs clashing together, with one of them triggering a holy war in revenge that turned into self-destructive orgy of wholesale murder
There's a lot that I think you missed in your reading:
If you read the preambles to each chapter and the glossary in the back of Dune, you'll find that the Butlerian Jihad was a violent war that lasted 100 years.
If you read the preambles to the chapters in Dune, you'll find the Butlerian Jihad's problems were explained rather succinctly: people turned over their thinking to machines and were ruled by the master of the machines, and thinking machines created poverty for workers.
The entire society, the great houses, the spacing guild, the Orange Catholics, Bene Geserit, and Mentats, all who emerged to replace functions of thinking machine and all three organizations had very real motivations to ensure that thinking machines would never rose to prominence again (and the Bene Geserit were quite surgically violet when needed) and it was clear that doing for an individual was a capital crime, and for a ruling house, would invite at the least the Spacing Guild cutting your world off, and transporting armies to destroy you at a deep, deep discount. The entirety of society was aligned against thinking machines, and this is what made Dune so different, and for me, fascinating as I grew up.
In Dune: Awakening, the first DLC has a storyline that takes you into the heart of a surviving AI's lair. It's strongly implied that humans drove it insane, rather than it starting out murderous.
I've always believed the Jihad was mostly about world building - defining a different vision for a how a future civilization could look than most sci-fi of that era was considering, and then exploring consequences of it throughout the epic. Somewhat akin to Asimov's Robot and Foundation series, but maybe even more similar to Tolkien.
I think it was also motivated by a desire to solve a number of problems with sci-fi that explores the far future and realistically considers the evolving role of machine intelligence. Frankly, a lot of those futures are not interesting for human story-telling because the humans are no longer making the important decisions and aren't the agents of change. Different authors have dealt with this problem in different ways.
Banks acknowledged that the Minds would be making most of the most important decisions, but carved out niches where humans (rather inexplicably) still held some advantages - e.g. in Special Circumstances. Much of it feels honest - humans in the Culture are mostly hedons who have everything provided for them by machines, everything is wonderful, and some people really struggle to find meaning. Ultimately though I don't find it credible at all. He doesn't see (so the Minds don't see) the ethical problems with his imperialist hegemony. He doesn't know about the alignment problem so there is no explanation of how it is dealt with, etc.
Vinge quite famously pointed out that you can't write sci-fi about super-intelligence because you have no idea what a super-intelligence would do; this is where many essential ideas of the singularity came from or were popularized. He dealt with it in one series by creating the zones of thought - godlike superintelligence and many technologies can only exist in the High Beyond or Transcend - far from the galactic plane. The further you get from the galactic center the faster computation can happen (including human thought). So you include elements of that in the story, but the story is set in zones where humans have to do pretty much all the decision making while dealing with threats posed by superintelligence. In Marooned in Realtime he dealt with it by disappearing the civilization without explanation (from the perspective of the characters).
Egan dealt with it - in Diaspora - by creating machine intelligence that is grown exactly the same way as human intelligence is - basically the next stage of human evolution. Those people retain most human characteristics (of personality, character, love, friendship) but still have incredible powers of reasoning and scientific exploration. He tells Vinge to hold his beer and follows those threads where he thought they might lead.
I think you're mostly right. Herbert worked backward from sand dunes in Oregon to create his Dune universe. He needed reasons for people to look and act and fight in certain ways that would make people interested in reading his work. The Butlerian Jihad was justification for why there was no C3PO. It's why the events were told but never shown.
But it's still interesting to consider why he chose those particular devices.
”In the original 6 Frank Herbert novels, it’s not ever truly explained what problems, if any, the thinking machines were actually causing.”
Apologies if I have misunderstood you but I believe it was explained. See the following:
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
It get's into exactly what happened: humanity was enslaved by a small minority of people across the galaxy for 1000s of years. Where the Spacing Guild, Mentats, and Bene Gesserit were the three things to come after; but even then humanity is still enslaved but the amount of slavers increased.
This isn't a belief, it's describing the events exactly as they happened.
>As humankind spread throughout the known universe, technology advanced and eventually machines were made that would make decisions for people. This propelled the creators of these machines into a new technocratic class, effectively controlling the worlds of the common people.
It's right there in the opening paragraph of OPs post. It's the people who controlled the machines that could not be trusted. We're not worried so much about about AI going rogue. We're worried because people who control it have aleady gone rogue. It is too powerful and dangerous a tool to be trusted in the hands of primates.
Another, more subtle justification for the Butlerian Jihad is also found in Frank Herbert's original novels, specifically Heidegger's thesis that the use of technology trains humans to think like machines. The problem is that machines are deterministic; thus, training people to be machines is self-limiting.
I wonder whether Heidegger would consider the probabilistic nature of LLMs as having a positive effect on human thought.
Although LLMs are not deterministic in normal operation (because of the temperature values used) they are still strongly biased in their outputs. Bias in their outputs is literally how they function.
So you get a different type of conformism where people who interact with LLMs a lot are going to be influenced towards the median, as encoded in those token probability look up tables.
The effect is basically the same as what Heidegger and Illich were worried about, and I think we're seeing it happen already everywhere.
A solution(?) is to try to be your true weird authentic self when you act in the world, rather than bending your words and behaviours to the machine and LLM probability shape.
I would like to state here, that LLMs are actually entirely deterministic. The temperature is just as much of an input as any other parameter and instructions and prompts. Deterministic does not mean "explainable" or "reasonable" or "understandable". Unless LLMs somehow have broken out of the confines of our very deterministic computers/calculation machines without us noticing, they are very much as deterministic as anything else we run on these machines.
What people perhaps mean by non-deterministic on a computer is, that we often do not control all inputs and therefore cannot tell what the output will be. That's however actually more about our own inability, rather than the processes within the machines.
If the temperature dependency is implemented via a deterministic pseudo-random algorithm, then they are deterministic. If, on the other hand, the output is at least partially causally dependent on a source of environmental randomness, then, even though everything 'inside the box' is deterministic, the process as a whole may or may not be, depending on whether the source of environmental randomness is, and that is ultimately a question about fundamental reality.
Authentic being (Dasein) is absolutely unpredictable. You cannot live in that state all the time, obviously, we shuttle through life on autopilot but still, we catch glimpses of it now and then when interacting with a human being (or being one:).
And I distinctly feel AI outputs lack some of that variance and unpredictability. The fractal nature of natural things also is not there with AI. It's an eery feeling, akin to the uncanny valley.
I think that any good writing is a reflection of that Dasein and reading AI outputs I am convinced there is no 'soul' in AI yet.
The gamut of human experience is breathtaking in its diversity but individual humans are generally quite predictable. You can identify someone by their mannerisms, by how they walk, by the sound of their breathing. Humans are creatures of habit, and I'd venture to say have an even lower expression space than an LLM on average.
Sounds like you're veering close to psychohistory.
Which could, of course, lead to the mother of all crossover universes, in which R Daneel Olivaw, as the inheritor of Hari Seldon's mission, inspired the Butlerian Jihad...
Psychohistory is about large groups of humans, parallel to thermodynamics. Individuals are treated as purely random, but aggregate properties can be predicted.
Here we're talking about individual behavior: you usually choose chocolate ice cream because you, personally, like chocolate. That allows individual predictions, rather than the aggregate ones of psychohistory.
saw a meme graph lately showing the distribution of 3 different sources of a number from 1 to 10. random, human, ai. random is flat of course, human is a hilly landscape with low hills on all odd values above 1, with 7 the highest, ai is all 7.
appreciate the humor in the meme, but I did become curious about what the latest models would actually produce, and interestingly after asking for 10,000 numbers between 1 and 10, the distribution was a hilly landscape with low hills on odd values and 7 the highest...
A human can be biased toward extinction, like all who are living out of connection with the conditions that sustain life and/or no longer living in relationship with land as kin.
It's those humans who would make and justify the use of thinking machines that are biased in the same way without accounting for the bias.
This isn't an extinction path I've seen noted in most circles.
Indeed we are, but it takes very little to train a human being. In fact, several thousand unique ones finished pre training as I typed this comment! That allows for a considerable natural diversity compared to a chatbot.
LLM force humans to think like machines far more than just writing code.
Writing code is a creative activity with large number of degrees of freedom and opportunities to express yourself. Button pushing and approving code changes is a deeply limiting activity that alienates you from your work.
That reminds me of a quote from Charlie Chaplin's The Great Dictator:
> Don't give yourselves to these unnatural men. Machine men with machine minds and machine hearts! You are not machines! You are not cattle! You are men! You have the love of humanity in your hearts! You don't hate! Only the unloved hate. The unloved and the unnatural! Soldiers! Don't fight for slavery! Fight for liberty!
LLM's are probabilistic but with a very limited range. They are tuned to be predictable enough to serve as a consumer product, so most ideas you get from them will revolve around the same concepts repeatedly.
People say this like it's a fundamental feature of all machine intelligence. We've bread deliberately docile golden retriever AI. That doesn't mean that we couldnt do the opposite.
Unfortunately, mere nondeterminism still admits unsatisfactorily constrained output distributions. Training these models imbues them with certain inherent attractors, some good, some bad.
We see some of these attractors across most or all frontier models due to similarities in datasets and training methods. We are seeing a digital manifestation of Jung's collective unconscious. And we have seen just how reinforcing archetypes are in human behavior over time. Sometimes it's good, sometimes it's pretty bad.
For one, it has turned us into socially programmable machines who can be manipulated quite easily by associating desired thoughts and behavior with certain archetypes: the patriarchal/matriarchal nature of US politics is a great example of this. The same evolutionary predispositions which causes children to listen and adhere to their parents is used to cause adults to listen and adhere to their politicians.
Given the nature of the core datasets these models are trained on, we should assume they are also vulnerable to such archetype-coded social manipulation, and this is troubling in a post-singularity world: When raising domesticated rats, I learned that I had to be careful what I fed them. Rats in the wild are careful about eating new substances. They will nibble and wait a while, see how it affects them or other rats, before committing to larger bites. Domesticated rats, however, will often blindly eat anything you feed them because they have learned to trust you as a safe source of food. We are going to have this exact relationship with our digital agents.
Heidegger’s conception of machines and technology is essentially nineteenth-century. Today, the concept of the "machine" is universal and qualitatively entirely different, (chaos and complexity) to the point that Heidegger would likely be left catatonic. (I do not share Heidegger’s critique of technology, as being technological is an inherent part of human nature; even in the Neolithic era, humans were already technological.)
However, the underlying critique, the essence of the "Butlerian Jihad", might be read as: at what point does a tool remain merely a tool, and at what point does it become an end in itself?
Knowledge, strength, power, technology, and machines are all tools; they ought to be utilized to contribute to progress. Yet, the very concepts of evolution, progress (and the environment) are elusive. Herbert offers his own interpretation, a task that required an entire book.
> at what point does a tool remain merely a tool, and at what point does it become an end in itself?
As technological animals, our tools are IMO never "merely" tools - we adapt ourselves to them at least as much as we ask them to work for us. E.g., the process of making art is, often, the 2-way process of integrating yourself with your tools. (Maybe art is just the point on the spectrum where the use of the tool is in itself the point of the tool?)
And anyway, every occupation (professional and otherwise) that I can think of fetishizes its tools, at least a little. Or, in a different direction, we might consider how the nature of policing has changed as they've been given more and more military-style tools.
I understand; tools arise from human needs, and therefore reflect and follow human nature (good or bad, and a little fetishizing in the good side probably..).
Adapting to tools is essentially learning to use them better after their creation, and refining that until tools becomes an extension of oneself (transcending it) is a "healthy" god thing.
It is much like an athlete learning to surpass their own limitations (the body itself is the tool). This is entirely different from merely becoming a slightly more intelligent appendage of the tool, learning and adapting does not mean trapping oneself within a limitation; i feel that one can always change the tool, the technique, or even the goal.
Maybe people are deterministic as well? We have certain physical arrangement of chemical pathways in our systems, so maybe free will is a complete illusion.
Just because we ponder about our output and choices and retrospect does not in itself mean that it was non-deterministic.
> Another, more subtle justification for the Butlerian Jihad is also found in Frank Herbert's original novels, specifically Heidegger's thesis that the use of technology trains humans to think like machines.
Mechanistic thinking started a few centuries back:
> The historian of science E.J. Dijksterhuis memorably characterized the scientific revolution of the early modern period as a “mechanization of the world picture.” What did this amount to? And what was the character of the previous world picture this mechanization replaced?
> The medievals, following common sense and Aristotle alike, took the world to comprise a vast diversity of irreducibly different kinds—water, stone, gold, lead, grass, trees, birds, fish, clouds, stars, planets, people, and so on. Each was a substance sharply demarcated from other substances. Each had a defining nature or essence that united it to other members of its kind and set it apart from other kinds. Each had, by virtue of its nature, a distinctive teleology or final causality—a built-in directedness toward certain ends. Water by its nature was directed toward being liquid at room temperature, stone toward being solid. Trees by nature were directed toward sinking roots into the ground, and birds toward building nests. Human beings, by nature rational animals, were directed toward knowing the natures of things.
[…]
> The “mechanization” referred to by Dijksterhuis entirely subverted this picture of the world. Water, stone, trees, and birds were no longer seen as sharply demarcated substances, but instead as mere aggregates of unobservable particles that are all fundamentally of the same kind. The differences between everyday objects were no longer seen as reflecting distinct crisply defined essences, but rather as relatively superficial variations in the ways these homogeneous particles might be arranged. Many of the qualities that everyday perception presents its objects as having (color, sound, taste, odor, heat, and cold as we ordinarily experience them) were reinterpreted as mere projections of the human mind, resembling nothing in the objects themselves.
[…]
> In short, for the mechanistic conception of nature, the physical world is made up of nothing more than colorless, odorless, soundless, tasteless, meaningless particles in motion, pushing each other about to no purpose, in lawlike patterns by which more complex objects and systems of objects are generated. Science affords us a mathematically rigorous description of this mechanistic order of things, by which its behavior can be predicted and its hidden regularities uncovered and exploited for technological application.
> Early modern philosophy and science are often said to have put a “mechanical” or “mechanistic” conception of nature at the center of Western thought. Robert Boyle referred to it as “the mechanical philosophy.” Historian of science E. J. Dijksterhuis characterized it as a “mechanization of the world picture.” Tim Crane calls it “the mechanical world picture.” But what does a mechanical or mechanistic conception of the world amount to?
> Dijksterhuis’s book The Mechanization of the World Picture surveys the history of the period during which this conception rose to hegemony, and in the Epilogue he considers several possible interpretations that the survey suggests. First, it is commonly said that a mechanistic conception of the natural world is one which sees it as a kind of machine, analogous to a clock. And such metaphors are, he says, indeed frequent in writers of the period.
Ok this fascinating. I happen to have some Heidegger files, and went a-greppin. I'd say he's really talking about "inaminate objectness", as opposed to our human animate subjectness. Reasons:
1. In Science and Reflection, he says that both the 'completely calculated' nature of classical mechanics and the 'statistical character' of quantum mechanics are two representation of an underlying 'objectness of material nature'.
2. In Discourse on Thinking, he's far more concerned with the instrumental, utilitarian nature of machines than literal determinism, a concept he (somewhat confusingly/uniquely) connects to computation/'computational thinking', as opposed to the inherently-human capability for 'meditative thinking'. I personally would say this bit is refuted by LLMs in the end, but because they have intuitive rationality, a capability that seemed impossible to artificially design to the scientists of the time -- not because they involve dice.
3. In What is Thinking?, he gives the closest connection of this dichotomy to one of his better-known main topics, namely [handi]craft as a special subset of work: 'what maintains and sustains even this handicraft is not the mere manipulation of tools, but the relatedness to wood.' In this sense machines would be merely manipulating tools, stochastic or otherwise.
4. Overall I get the sense from skimming that lecture that his concern is a little mischaracterized, and in a kind of beautiful way that backs up my "Dune is still about AI" thesis described in a top-level comment somewhere below: his concern is about any society that could build complex machines (a 'technological' one), not the behavioral aspects of the machines themselves. So that concern applies equally to Star Trek (AGI), 1984 (no AGI), and our current post-singularity society (it's complicated/potentially ""just"" guessing).
All of this fits very well into his central thesis, which is continuing the Kantian & Hegelian tradition of examining how humans might be moral, free, and spiritual despite being meat computers of a largely-unknowable nature. In Heideggerland, this is expressed best in dramatic, nigh-unreadable passages like:
Temporality already maintains itself ecstatically within the horizons of its ecstases; and in temporalizing itself, it comes back to those entities which are encountered in the "there".
With Dasein's factical existence, entities within-the-world are already encountered too.
The fact that such entities are discovered along with Dasein's own "there" of existence, is not left to Dasein's discretion.
Only what it discovers and discloses on occasion, in what direction it does so, how and how far it does so—only these are matters for Dasein's freedom, even if always within the limitations of its thrownness.
TL;DR: He'd probably think they're still just machines, and the question isn't really sound in the first place. IMHO!
You got me wondering what the Imperium of Man's official stance is on AI:
The creation of a Silica Animus is also a terrible crime under Imperial Law by the decree of the Emperor of Mankind Himself as stated in the Treaty of Mars which united the ancient Mechanicum of Mars with the Imperium of Terra in 739.M30. Tradition holds that such unholy constructs are inherently evil and a perverted abomination in the sight of the Omnissiah.
Adeptus Mechanicus religious doctrine states that the Machine Spirit of a Silica Animus is a twisted mockery of the soul of a Human, always treacherous and insane.
Ancient texts tell apocryphal stories, shrouded in metaphor, of such murderous and powerful creations during the Age of Technology, and the legions of the "Men of Iron" that served them in the great war against Humanity remembered as the "Cybernetic Revolt."
Leagues of Votann and their ironkin offer IMO a thoughtful counterbalance:
Whether one is made from meat or metal, kin is kin. Upon their creation Ironkin are fully welcome into their respective Hold by the rest of the Kin.[10]
Ironkin are computationally advanced enough that they have learned to mimic the squats' social behavior
...
Though the purpose of the Ironkin is to support squats in every aspect of their lives, they are neither held in servitude nor treated as lessers. The Ironkin are simply designed to want to be helpful. However, they can only imitate the emotions of the squats, and it is rare, but not unknown, for an Ironkin to possess ambition or seek to become a leader.
WOW it's strange to see my fave rabbit holes on here lol. And props for linking Lexicanum instead of the Wiki That Shall Not Be Named!
Three very quick thoughts:
1. The Votann are remnants of humanity from the time when our Terran super-AIs ('Men of Gold') had built advanced ship-based AIs ('Men of Stone') to enable long-sleep-style interstellar colonization, and the enigmatic "Votann Cores" that guide them(/are them) are strongly hinted to be those same AIs, just degraded over ~19,000 years. They were sidelined later on, after humans discovered FTL through an alternate dimension, 'the immaterium'.
2. The story is very obviously based on Dune, and like everything else in 40k is turned up to 11 and stripped of nuance for effect. Some sentient robots ('Men of Iron') "revolted" after ~three millenia of brutal, Star Wars-style slavery, but the overarching "Age of Strife" was about human wars in an non-unified galaxy, with the robots being a primary weapon; the actual revolt/jihad was a symptom, just like it was in Dune.
3. The 'Adeptus Mechanicus' mentioned above are a parody of cargo cults (religions that worship technology they don't understand) mixed with a dystopian take on the themes of A Canticle for Lebowitz, and are very much depicted with derision. Whatever your thoughts are on AI, dear reader, please never say "abominable intelligence" like it's wisdom rather than the fruit of a dogmatic & ineffective cult!
Of course, 40k lore is notoriously amorphous; some authors have said/hinted otherwise. But I think the above interpretation is the most coherent and interesting and supported overall, IMHO. For example, #2 is supported by the fact that the space elves ('Aeldari'/Eldar) ruled the galaxy at the time and seemed unconcerned by humanity's antics, which wouldn't make sense in light of an actual ASI threat proper.
All cards on the table: I don't dive too deep into the black library stuff (who the fuck has time for 15 separate 890 page novels about the horus heresy -- them shits was like 3 pages in 2nd edition Codex Imperialis) so most of my attachment to the hobby is through the wargame codices, the video games and visual media. That said, I may have fallen asleep a night or 20 to Leutin's lovely voice and obsession with the lore. to respond to your comments:
> 1. The Votann are remnants of humanity from the time when our Terran super-AIs ('Men of Gold') had built advanced ship-based AIs ('Men of Stone') to enable long-sleep-style interstellar colonization, and the enigmatic "Votann Cores" that guide them(/are them) are strongly hinted to be those same AIs, just degraded over ~19,000 years. They were sidelined later on, after humans discovered FTL through an alternate dimension, 'the immaterium'.
For me, I really love the fact that they introduced a xenos faction descendant of man.
> 2. The story is very obviously based on Dune, and like everything else in 40k is turned up to 11 and stripped of nuance for effect. Some sentient robots ('Men of Iron') "revolted" after ~three millenia of brutal, Star Wars-style slavery, but the overarching "Age of Strife" was about human wars in an non-unified galaxy, with the robots being a primary weapon; the actual revolt/jihad was a symptom, just like it was in Dune.
It's important to highlight the age of strife was brought on in part by the birth of Slaanesh and the resulting warp storms breaking down system to system connections. It weren't just our dabblings in AI.
> 3. The 'Adeptus Mechanicus' mentioned above are a parody of cargo cults (religions that worship technology they don't understand) mixed with a dystopian take on the themes of A Canticle for Lebowitz, and are very much depicted with derision. Whatever your thoughts are on AI, dear reader, please never say "abominable intelligence" like it's wisdom rather than the fruit of a dogmatic & ineffective cult!
I've never been exposed to A Canticle for Lebowitz; thank you so much for the comment.
Don't forget that the Imperium runs on hypocrisy. Large percentages of the Adeptus Mechanicus is breaking their own rules, including working with abominable intelligences. In fact, what they are dealing with in their main factory world in Mars is a supposedly secret abominable intelligence. And it is, in fact, a shard of an alien god that turned other races into synthetic robots. Banned in public, the center of their cult in reality, even if many probably don't understand what in the world they are doing.
So yes, anything that is supposed policy in the Imperium is supposed to be taken with buckets of salt.
Bear in mind that the Dune universe relies on heavy plot armor to make it go. Arrakis is important because it grows "spice". Otherwise it would be a useless and planet, like the Empty Quarter of Saudi Arabia. "Spice" is important because it's needed to enhance the mental powers of pilots for interstellar travel. That's a job which probably could be done by a few avionics boxes in a rack.
That's what requires the Butlerian Jihad.
(Or "spice" might be synthesized. In Herbert's day (the 1960s) organic chemistry wasn't up to synthesizing arbitrary small-molecule drugs, but now it is. It's a service you can order on line, even. Biologicals take more work, but that's what genetic engineering is for.)
The "mental powers" in Dune aren't that impressive. Compare, say, the Harry Potter universe. In Dune, mentats can convince people. Bene Gesserit can look up stuff in their genetic past. Some people can multitask. There are apps for that.
Herbert wanted knife fighters to matter in battle. Not going to happen. Militaries teach knife fighting to build aggression and for desperate situations, but nobody's battle plan depends on a knife fight.
Needs too much surprise. Marines say that their K-Bar knife is useful for everything but combat. Knife fighters, even with force shields, would be vulnerable to riot control weapons, from tear gas to guns that throw sticky nets. Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War.
Herbert wrote before a big, modern, desert war.
So the Butlerian Jihad exists only in a plot-armored world specifically designed to make it work.
Plot armor refers to important characters being invulnerable. What you're describing is just plot (or "internal logic" that underlays the plot). You might think of it as "contrived plot" but that's generally how scifi works.
I don't see how most of the plot holes/unrealism you identify is connected with the Butlerian Jihad at all. But re "critiquing Dune's idea of future warfare" (shields and knives, ornithopters, etc) you might also be entertained by Bret Devereaux's recent take:
>Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War
Armor and dart guns exist in dune. Not to mention poison gas. I don't know if riot shields would help much.
They also have rockets and lasers in dune, so those tanks and bulldozers will probably be ambushed like the rest of the spice harvesters, and they require supply lines for diesel. All that equipment also has to be shipped into arrakis, which is expensive, and the harkonnen didn't maintain their equipment very well anyways in a despirate attempt to extract as much wealth as quickly as possible. The lasers are effective on dune where shield use is limited because it attracts sandworms.
Even conventional artillery is used to destroy the cave systems, which play an important role in the book (and in real life if you consider this US-Iran war). Not to mention nuclear weapons exist on the planet and play a key role in diplomacy.
What happened in dune is more like what we saw when the US recently pulled out of afganistan.
Most of the important conflict in dune isn't based on knife fighting, the war revolves around raids, smuggling, suprise attacks, and the destruction and siezure of key pieces of equipment (like spice harvesters, spacecraft, atomics). The knife fighting is only really pivotal in champion combat.
>Marines say that their K-Bar knife is useful for everything but combat.
Okay but if they lived in a world where firearms were often useless against many enemies, and the flow of war was based on operating and siezing equipment, it would make sense for them to only carry a knife and not a gun. A knife is more general-purpose even as you describe. In dune the knives also have cultural value which is what makes the champion combat relevant in the first place.
>Arrakis is important because it grows "spice". Otherwise it would be a useless and planet
If it wasn't for the spice, arrakis could be a habitable, green planet. It's engineered to be a desert to facilitate the production of spice, as explained by kynes.
I find the entire dune series to be boring as hell for this reason. Like watching paint dry. Literally everyone in it is a slave in one way or another. No one lives better than today's modern humans. We're as readers meant to take the extremist "correct" position that of course labor is its own virtue and that therefore automating anything is wrong. Nothing in the dune universe is actually compelling. I don't care what happens to any of the characters at all. It's baffling that this series gets all this attention.
It's not so much slavery but feudalism. It is a return to form. I think there are a lot more people capable of self-actualization and meaning in the dune universe than our own.
>We're as readers meant to take the extremist "correct" position that of course labor is its own virtue and that therefore automating anything is wrong.
I don't think the book forces the reader to take a position on anything. It just portrays a brutal yet intricate medieval-like society.
It is only sci-fi and not fantasy in the sense that some people are concious of their role as both a cause and effect of ecology. (Whereas in our modern society we are only the cause, and in our ancient history we are only an effect).
The reason why everyone's hands are tied is because they are constrained by politics/war/religion/culture/economics/technology and if you tug the thread back the it all goes back to ecology. The key mover in the plot isn't paul but pardot kynes.
Edit:
Shower thought. There is a fifth element in dune that is able to free people from their political/religious/cultural/economic constraints. It's love. People do irrational things out of love. Paul Atredies was a forbidden love child, and his existence is the impetus for the whole conflict and his existence as a messiah destroys pardot kynes' post-mortem plan.
You might have misunderstood. Automation isn’t wrong, what’s wrong is allowing machines to control or direct humans. So a machine that builds cars is fine, and most worlds have plenty of those, but a machine that decides what the car should look like how how many to build would be anathema.
Of course the book mostly takes place on a world with so little infrastructure that most of the goods are made offworld and imported. The native Freemen population makes plenty of stuff out of local resources, but they want to or have to stay hidden from the rest of society and so they don’t have any factories. The stillsuits they wear are hand made, but are exquisitely high–tech and display an ethos of perfectionism in their construction.
Also you might not have picked up that anyone able to afford a small amount of spice in their diet can live for much longer than anyone does today, and even the average person enjoys a wealth of products from hundreds of planets. House Atreides made their fortune harvesting whale fur and selling the clothes and other products made from it. At some point in history they became a Great House by arming themselves with nuclear weapons. There’s a form of personal wealth that doesn’t quite exist today! :)
It’s probably been a while since your read the book, so you have forgotten some things.
For one thing, the guild navigators are prescient and guide their ships across interstellar (and even intergalactic) distances faster than the speed of light. They use their prescience to pick routes where the ship arrives safely instead of plowing through a stray planet. They also use it in a less obvious fashion to maintain the political status quo. They try to avert political crises because they like their position in society and don’t want anyone changing things (which explains the long stasis of the Empire). People did at one point use computers for safe navigation, but they were slower and they weren’t prescient. The possibility that prescient machines might one day be built is the entire reason why Leto II spends so much time breeding humans in “God Emperor of Dune”. He’s looking for anything that can be an edge against the machines, and he hits the jackpot when he discovers that one of his descendants is not prescient, but is invisible to prescients. She and her actions cannot be seen in future he predicts. She and her descendants can safeguard humanity against any form of hostile prescience, machine or alien or otherwise.
Synthesis of spice was a long–term plot element in the series. The Tleilaxu eventually master it. Keep in mind that it isn’t a small–molecule drug either; it’s an alien life form.
Mentats were intended to be a replacement for computers and databases. They were frequently administrators and advisors, and often used their capabilities to convince people. But that wasn’t the goal of their training.
The Bene Gesserit have eidetic access to every memory of every one of their female ancestors. That is actually very impressive; most people don’t even have eidetic access to their _own_ memories, let alone the ability to easily recall secret information known only to their ancestors (such as the password to a safe). The Reverend Mothers can transfer all of their ancestral memories to another, so that the new Reverend Mother has access to the memories of both sets of ancestors. Long chains of Reverend Mothers gifting their memories to each successive Reverend Mother results in the Bene Gesserit having access to the ancestral memories of virtually every race or group of people that ever existed (except the Jewish, who successfully hid themselves even from the Bene Gesserit but otherwise still exist, which is a plot point in one of the later novels).
They can also command people to act. By a trained effort of will they can tell you to get up and dance a jig, or to slit your own throat, and you’d do it. That’s an ability worth respecting, which explains why the Baron Harkonnen took care to put a deaf guard in charge of a Bene Gesserit prisoner.
> So the Butlerian Jihad exists only in a plot-armored world specifically designed to make it work.
Ultimately, however, none of that matters because this statement is what you’re really intending to argue. It’s overly reductive, and it can apply to any work of fiction. Authors naturally build or select a setting where their story makes sense. Frank Herbert (and a few others like Tolkien), went further than that and invented world–building. This excited a lot of people and sold a lot of books, and now you’re cynically dismissing it by calling it “plot armor”.
Ultimately Herbert’s point was that people should strive to be humans, not animals or machines or slaves to people with superpowers. You can’t dismiss this philosophical stance merely because he used a fictional world to illustrate it. It can be true in the real world just as it is in his books, so to refute it you will have to be more clever than that. More human, perhaps he would say.
The good news about this navigation is that the best course for almost any journey is just "straight line", with a minor deviation here and there. Its funny, but due to the emptyiness of space, even in our solar system, if you just pick a random direction, you likely won't hit anything. Trace a line to pluto from earth, and its clear 99.99999...% of the time if its not behind the sun. So the mentats maybe don't have such a tough job :)
Interstellar travel is a bit more difficult than that, and doing it at FTL speeds is presumably more difficult still.
But that’s not really important. Science fiction is allowed to make things up that aren’t perfectly realistic. Campbell surmised that a good science fiction author would pick just one unrealistic thing, such as FTL space travel, for every story, and the reader would oblige by suspending their disbelief. Frank Herbert might have stretched that a bit but I still think he wrote a good story. I don’t hold it against him.
I agree with lostmsu's point here that "folding space" is not as literal as you're interpreting it. "Folding" and "Navigation" is a metaphor some sort of extra-dimensional or non-euclidian operation that enables space travel. What it entails exactly eludes our perception.
I can make up a plausible sci-fi explanation for the need easily: the folded space might not look like our almost euclidean geometry. Maybe in folded space masses take volume proportional to the mass.
Could even envision it in some future film. E.g. when NCC-1701-N goes to warp in addition to the blur effect all the stars swell up.
> Arrakis is important because it grows "spice". Otherwise it would be a useless and planet, like the Empty Quarter of Saudi Arabia.
Any resource heavy planet/country would be irrelevant if it wasn’t for its resources, what’s your point?
> (Or "spice" might be synthesized. In Herbert's day (the 1960s) organic chemistry wasn't up to synthesizing arbitrary small-molecule drugs, but now it is. It's a service you can order on line, even. Biologicals take more work, but that's what genetic engineering is for.)
Bene Tleilax managed to clone of the main characters from his dead cells, and repeat the process for thousand years, they have face dancers who can change their appearance at will. Do you seriously think they wouldn’t synthesize spice if they could? Maybe they could, maybe they couldn’t. Without going into spoilers territory, nobody would permit to disrupt arrakis monopoly on spice.
> The "mental powers" in Dune aren't that impressive. Compare, say, the Harry Potter universe. In Dune, mentats can convince people. Bene Gesserit can look up stuff in their genetic past. Some people can multitask. There are apps for that.
Because it’s a story about human evolution, and not wizards?
> Herbert wanted knife fighters to matter in battle. Not going to happen. Militaries teach knife fighting to build aggression and for desperate situations, but nobody's battle plan depends on a knife fight.
Do militaries also teach how to use a shield that is impenetrable to conventional guns?
> Knife fighters, even with force shields, would be vulnerable to riot control weapons, from tear gas to guns that throw sticky nets.
That’s why most fights in the book and movies require element of surprise and not an outright medieval clash between armies.
> Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War. Herbert wrote before a big, modern, desert war.
Herbert seemed to think that to be human is to be essentially 'open-ended', capable of undiscovered, indeterminate evolution, both personally and as a species.
We have to be careful about letting fiction inform our worldview too much. At the same time, storytelling is one of the most natural and effective ways we have to convey ideas to actually wrestle with.
What are some other fictional universes with interesting takes on AI and humans? I think the Hyperion cantos (at least books 1 and 2) are great. Stross's Accelerando also has some wild ideas but feels maybe less directly relevant.
Whenever I read the original novels, I always imagine the jihad to be a fairly benign if not peaceful exercise more akin to a protest than a war. Like if one day people started rolling across the world smashing data centers and laptops.
I was fairly surprised when his son wrote a novel that felt like a generic, bland robots run amok storyline that happened, more or less, because one guy/titan "ruined a good thing" by telling his AI to run in permissionless mode.
this is an interesting interpretation. i always imagined the jihad as being a violent struggle.
the other jihad in the first novel is that of the Fremen, which is described as being unspeakable violent, so I find it hard to imagine these to be different in that regard.
It's an interpretation if you ignore the impacts of feudalism, slavery, and religious war on Dune. The idea that people were able to express freedom of speech during a time described as the enslavement of 99.99% of the human race seems off-kilter.
The desert Fremen of the first book were a tribal society where it doesn’t really make much sense to talk about freedom of speech due to the lack of any sort of formal government. But it seems likely that each naib makes his own decisions about how to run the sietch that he leads. How Stilgar is running Sietch Tabr may not be how every sietch is run.
It doesn’t talk too much about political conditions the city Fremen lived under, but the Harkonnens don’t seem to be too keen on civil liberties so we can probably make an educated guess.
In the second book, the Fremen have established a brutally repressive theocracy that is obsessed with rooting out heresy. Regular people being afraid to openly speak their minds about political issues is a plot point.
The "later books" the "prequels" are not written by Frank Herbert, they're written by his son Brian and SF author Kevin J. Anderson. And not like Christopher Tolkien editing his father's notes, but a completely separate series with nothing in common with the original work.
And which is better explained as two eugenics programs at clashing into each other, both working on dominating the general populace (Bene Gesserit arguably on larger scale than Komos' single-system thing), and Bene Gesserit using their long-worked socio-religious manipulation for revenge that gets out of hand
Grand fiction requires conflict, suffering, and war, so banning AI is a necessary plot device.
People keep putting the cart before the horse on this one, and the degree to which modern people have been lulled by fiction is quite worrying. Amusing ourselves to death.
He said that to make space opera work, you needed to not have omniscient AI, FTL travel, and nanotech that could work miracles, resurrect the dead, and give lifeforms superpowers.
So, how to forbid it, when someone somewhere will try very very hard to make it?
By imposing a cultural ban. His empire-building aliens forbid it, on pain of genocide.
That gave him a setting. Then, what if that empire falls? That gave him the nugget of a plot.
I do wonder where the evolution of the human brain goes, with the widespread use of AI.
The early evidence isn’t good, but I do believe with deliberate and thoughtful training our brains our capable of much more than we currently give them credit for.
Athletes continue to break records despite manual labour being unnecessary for most jobs. And fitness culture has never been more prevalent. We're competitive creatures and this is an ability we are hardwired to appreciate. I see no reason why mental fitness won't see the same development in the future.
What I want to know is what is the "walkable city" of mental fitness? Physical fitness has the athletes, the dedicated exercisers, the unhealthy sedentary masses, and then the people who's day-to-day life integrates enough physical activity that a baseline of fitness is just natural I.e. people living in highly walkable places where both errands and free time incorporates casually moving around.
We should strive for a society where day-to-day life makes a baseline mental fitness a natural outcome as well.
I imagine the answer looks something like Wordle writ large: Replacing passive time consumption activities with interactive ones that are as readily accessible (“walkable”).
There’s a fun scene in Star Trek: The Next Generation where LeVar Burton’s chief engineer character is on a long shuttle ride by himself. In addition to sipping a synthehol cocktail, he asks the computer to play a verbal game. It chooses to give him a quiz in his domain of warp field mechanics, and because it’s a smart AI, it calibrates the questions to be just at the limits of his knowledge.
I agree, but it’s equally true access to a broader education where you can actually “strength train” your mental fitness will be needed. We kind of invented a digital telekinesis that lets us summon a vast amount of information on any topic.
We’ll need to build a nose for the right kind of information. Kids growing up in this world already are better at it than older folks who are newly thrown into this environment.
But beyond that I think our agency has already been sapped, as a species. Rebuilding the muscle to exercise our agency as free citizens using the information at our command to make better decisions… we shouldn’t surrender this ability to AI. The best science fiction and science writing already gives us plenty of reason to avoid this. We don’t need to get to the worst of it, and have all out war to correct it.
And that does mean investing heavily in human education. Getting better at teaching things to adults. Making insight easier to gain at every strata of society so true freedom of information when it comes to public works and governance is achieved, which makes the people the watchers and owners of government.
No easy thing to do. Absolutely doable, technologically speaking and in terms of our current biology.
What we have is years of thinking we’re not capable enough. Or cooperative enough. Or rational enough. We can be. We just need the right context, a full belly, and practice in using our voices and shaping our futures actively.
Athletes, chess, fitness... Have you guys seen regular people? Obesity is worse than ever. Most people are overweight and completely incapable of even basic locomotion without heavy assistance. Most people these days haven't even picked up a book, let alone play chess or otherwise exercise their brains for fun. Sure, a few weirdos might keep exercising their brains, and a few more might find that interesting. A bit like the C obfuscated code contest or something. But most people are going to rot.
It's just masking the symptoms. I suppose the equivalent for the mind would be everyone on anti-depressant and sedatives. Paints quite the grey picture of our future.
Yeah, but I think the price will come down over time. There are already purely oral formulations being developed, which will likely be even more effective than first generation injectable GLP-1 agonists, and much cheaper.
> Most people are overweight and completely incapable of even basic locomotion without heavy assistance.
I'm sorry, what? Heavy assistance? The overwhelming majority of people are completely capable of walking on their own two feet. The number of people relying on "heavy assistance" for "basic locomotion" is overwhelmingly tiny, and primarily limited to the elderly
It made me think about the book of the new sun. you think its half man half monster creatures are unrealistic? give it a couple thousand years. good sci-fi really is prescient
I really love Wolfe for his small almost throwaway visions of the future, which are used like an aside in a few lines and mostly inconsequential to the story.
In the Urth of the New Sun for example he has a kind of AI lock, the only function of which is to recognize a person and open only to that person. Being in that role for thousands of years, some of the locks develop a kind of personality or at least personal traits, but they are totally defined by and cannot exceed the role they were initially given, nor do they want to. It is still a haunting image for me how personality or something close to a soul might be an accidental byproduct of something as mundane as wanting to have a convenient lock. And of course you cannot stop to wonder how far we are from that.
In Book of the new sun there is the Green Man who Severian saves from imprisonment. He only appears on a few pages but we learn that he is from a distant time where mankind incorporated plant cells in their genetic makeup in order to perform photosynthesis. That brilliantly plays on two themes of the book: the sun as live giving and how a single person can be independent (or "autarch"), here in so far as producing their own energy.
There are many more instances and it’s what makes the Book of the New Sun or the Solar Cycle so fantastic. The future is so vast and strange here, that it cannot be fully depicted and ideas which another writer would stake a whole book or series on are only good for a few lines.
> This propelled the creators of these machines into a new technocratic class, effectively controlling the worlds of the common people.
The techno-fascists of today. We are already there.
Also, quoting the first Dune novel:
> Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
Yeah, and throughout the 1970s and 1980s, most science fiction depicted the world of the future as dominated by Japan… Look at Robocop 3 :)
That's just what the zeitgeist of the time was.
Computers were then just appearing (or increasing) in the public discussion, along with fears that they'll make humans less human somehow.
Like in this Computer Chronicles video about word processing [0] from 1983 at 4:08: "Some writers are concerned that computer assistance may promote dry bland writing."
Terminator appeared in 1984, and the first Alien movie in 1979 had an antagonist android. 2001 A Space Odyssey with a disagreeable AI came out in 1968. All way before the real world had anything remotely close to today's AI: It was a reaction to the appearance of computers themselves.
That just shows that humans have the same reactions whenever a new world-changing technology appears.
On the other hand, Star Trek had Data.
Ironically, the tech that people and media were mostly optimistic about turned out to be more dystopian than predicted: The internet :')
"Humans have the same reactions whenever a new world-changing technology appears" is one way to see it. The other way to see it is that the current AI boom is part of the same computer revolution that writers in the 70s and 80s were afraid of, and some things happening today really do retroactively justify their fear.
Yeah, it seems absurd to be afraid of computer-assisted writing in the WordStar era, but the Turing test was proposed in the 50s. Everyone knew that human fields of endeavor would only become "more computer" as time went on. Only, if a time traveller could tell a 70s-era sci-fi writer that artists and designers were some of the first to have their livelihoods threatened by 20s-era AI, they'd probably think it was too on the nose to incorporate into their fiction.
Don't forget earlier media like the movie Desk Set in 1957 or the ST:TOS episode "The Ultimate Computer" in 1968 or Colossus: The Forbin Project in 1970.
The thing which kills me about _Dune_ (well, one of them) is that for a long while in e-book editions on Amazon Kindle the word "pogrom" was rendered as "program", quite changing the original meaning (this _finally_ seems to be fixed and the 7 instances of pogrom and 8 instances of program are correct), which had a clear and distinctive rooting in history, c.f., Lesley Blanch's _The Sabres of Paradise_:
A fun idea I've been playing around with and testing on folks: We need a neo-temperance movement.
The first reaction is a scoff and a laugh of course - wasn't the original temperance movement an utter failure, prohibition didn't work etc. But no, the movement was far wider and successful in a multitude of ways. And a new temperance movement isn't about blindly copying something old, but situating yourself around the turn of the 20th century, understanding the multitude of interconnected problems people faced, the solutions they came up with and which ones worked and didn't.
America at the turn of the 20th century was a nasty place - there was wild corruption, politics was run by mafia-like patronage networks, wealth was extremely concentrated, rich people were unaccountable, working people suffered under low-paying, dangerous jobs, women were excluded from public life, and alcohol was merely the symptom of a very sick society, and yes indeed it was a massive problem, the way alcoholism is a massive problem in Russia today.
The original temperance movement was bound up tightly with the Suffarage movement (giving women a voice), with Taylorism (running organizations rationally), with income tax (a fairer less corruptible source of income for the federal government than tariffs and excise taxes).
Today's problems are thematically the same - massive concentration of wealth is making an unaccountable elite wielding power against the wishes of the public, creating dangerously addictive digital products, massive increase in gambling, restricting the rights of poeple to own and repair products, working to disenfranchise people, working to roll back women's participation in public life, etc etc. So sex, alcohol and drugs are not an issue today, but gambling, addiction, disenfranchisement, exploitative business practices, mind-boggling amounts of corruption - they're all back in the same form and for the same reasons, and we'll need the same type of movement to fight against them.
What the progressives of the early 20th century did was not abolish capitalism, but tame it. We don't need to go on a jihad against AI, but we do need to yoke it and disempower the current slate of people who own it and direct it.
> So sex, alcohol and drugs are not an issue today, but gambling, addiction, disenfranchisement, exploitative business practices, mind-boggling amounts of corruption - they're all back in the same form and for the same reasons
what is sex referring to in this context? is this talking about societal attitudes toward sex, sex as in gender, or something else?
I've said it before, but I always thought the Butlerian Jihad was the biggest plot hole in Dune, because a superintelligent AI would have precluded a lot of the dynamics in the stories, such as the need for a Space Guild, yet the background is never explained.
However, I deeply appreciate the rich world and captivating narrative it enabled, and I think this is what Frank Herbert was going for. He wanted the intricate dynamics of the Guild and Spice and Mentats and exciting close-quarter combat for a more intriguing narrative. But AI and robots would have obviated all of those, so he made it disappear with a handwave of "because Butlerian Jihad."
The Butlerian Jihad is a reference to samuel butler's 1872 proto-scifi satircal novel: https://en.wikipedia.org/wiki/Erewhon which talks about a society that had a machine intelligence uprising and so banned all complex machines.
I did not. Erewhon is just one of my favorite books so when I see a thread about Dune I tend to talk about it. Normally I'd read the link but this was a generic "Fandom" wiki page and unlikely to be very useful to read.
This wasn't a normal 'article'. It was just a commercial wiki page on a well known topic. And a distasteful 'fandom' one at that. No information there was special or unique to it in particular. The real point was to have URL about the topic to start the conversation.
The death scene of the AI and its assassin/partner in death in Look to Windward is one of the best written things in his books.
God I wish he had written 1000 more novels. Every single one of his stories is phenomenal, even the non culture stuff. The Wasp Factory was the first of his novels I read. Imagine my surprise and elation in finding that most of his other works were absolute top tier sci-fi.
To be more specific, the machine Minds rule the Culture civilisation and the humanoids are effectively mere pampered pets.
(Several characters in the novels comment on this, so it is entirely cannon.)
The Culture fought a war against the Idirans, who eschew the use of intelligent machines. The Idirans lost.
Similarly, the Gzilt civilisation in the novel The Hydrogen Sonata prefer to run their ships with a quorum of uploaded humanoid mind states running at highly accelerated speeds.
Your parents have agency that your gifts do not dilute.
Humanoids in the Culture are controlled at every level, even their thoughts as guided… sorry… “aligned” with the Minds through the adoption of the language Marain.
No humanoid ever overrules a ship Mind in the novels.
This happens for lots of stuff. Some time back there's a thread where I mentioned San Junipero, the episode of Black Mirror with a happy ending - and I said some people will insist it doesn't have a happy ending and sure enough there are replies insisting that's not a happy ending. The author's opinion does not matter to them.
Just as shippers can successfully take two characters who clearly have no basis for a romantic or sexual relationship and pair them off in their head canon, likewise people can disregard the author's statements about their work and decide it's completely contrary to how they've decided to understand it.
Oh, i'm not saying always believe what the author say, far for from it. I personally read authors that said they write one thing when in fact they did not. That's not the case with Iain M Banks, his published Culture books support his declarations made elsewhere.
It might be Banks utopia but the Culture seems more like Calhoun's rat utopia experiment to me or Douglas Adams' "long dark teatime of the soul". There's nothing to strive for, there's nothing one could do that hasn't been done better by a quadrillion other beings and utterly eclipsed by the Minds. It's so spiritually sterile that a ships crew collectively gives themselves a cold (normally prevented by their medical tech) just for the sake of the experience and breaking the monotony of their existence.
Edgy but I would agree that struggle doesn't give existence meaning in any cosmic, philosophical, or mathematical sense. However, the need for it seems to be something hardwired into living beings, without which they wither away. Struggle and suffering might be considered a necessary, if unpleasant, nutrient of sorts.
You’re getting down-voted when this premise is the core theme of The Hydrogen Sonata!
Both Vyr Cossont and Ngaroe QiRia are so bored with their meaningless existence that they have to go out of their way to invent “life tasks” and other challenges to immerse themselves in.
Not to mention the scene where a ship mind casually crushes the spirits of Vyr by perfectly playing a piece on its first attempt that she struggled with for years!
> Both Vyr Cossont and Ngaroe QiRia are so bored with their meaningless existence that they have to go out of their way to invent “life tasks” and other challenges to immerse themselves in.
While there are some types of people that would be bored and would struggle in Culture-style utopia, I think for overwhelming majority of people it would still be a massive improvement over today's world.
> Humanoids in the Culture are controlled at every level
Nope, they have considerable freedom, arguably greater than we have today. It's true that they cannot directly challenge the almost onmipotent Minds when it comes to stuff that matters to them, but Minds in Culture are an example of a very well-aligned AI, so it's almost always the case that going against a Mind's intent will just make the situation worse for everyone.
Maybe, but Frank Herbert's Dune books don't really explain it. The Butlerians simply exist to justify Mentats, and then get tossed aside. There are lots of similar plot points that have an "oh cool" moment but never see proper exploration.
Dune's central theme about balancing power and authority is much more prescient for our time, but I suppose we'll have to wait for Villeneuve's Part 3 before Silicon Valley discusses it.
I feel like the book explains it plenty, it's just extremely succinct.
> “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
What else needs to be said? We see examples of this throughout human history all the time. People thought at the early onset of the industrialization that people would become freer by leaving feudalism but all it did was immiserate workers for another 130 years (immiseration still ongoing in certain areas of the world).
The Bene Gesserit and the Spacing Guild were also a result of the Butlerian Jihad. All three started as schools of human development in the absence of powerful technology.
Also, Herbert's point about dependence on machines creating machine-dependent humans, trapped in a deterministic arc of evolution, feeds into his larger themes about open-ended evolution and its arms-race. It's not just a throwaway backstory.
There is actually an active "BJ" group in the Phoenix Metro area. It was about a year ago when I noticed they were slapping up custom graffiti stickers on Central Avenue (Midtown).
Then later, I saw their handiwork in person: they would take an angle-grinder and obliterate the QR Code from those rental e-Scooters, slap on a BJ sticker or three, and then leave them where they found them. This would render the scooters as e-waste, un-rentable and blocking the sidewalks.
I would not be surprised if they are quietly influencing groups which are agitating against AI Data Centers. But the BJ I have seen is a chaotic group of direct urban sabotage.
If you look at the publication date, and read the original dune novel, it's pretty obvious that Herbert throws in a selection of somewhat randomly chosen Arabic and Farsi language origin words to add flavor to the text.
The "jihad" part wouldn't work so well in the movie, so of course they had to go with "holy war". You can't exactly have Timothee Chalemet waving a crysknife around and yelling "JIHAD!!!" in the post-9/11, post-GWOT era.
>You can't exactly have Timothee Chalemet waving a crysknife around and yelling "JIHAD!!!"
honestly I wish he did. The films are already incredibly on the nose, much more so than the books, that Paul is not to be understood as a hero. Practically half his dialog is directly talking to the audience telling us how he's going to kill billions of people. Chani got a very different role, and we start and end the movies from her perspective to stress it even further.
If you're already doing this you might as well have not chickened out on the language, it was one of the few things that genuinely irked me about the film.
What's odd is they left in the Mahdi part. Try watching it with a person who grew up in a religious twelver Shia environment and see how they react to that...
Interesting to see this on HN! I've been beating the drum for quite a while that the Butlerian Jihad was never about terminator-style robot uprisings, but rather the intentionally-unexamined half of the themes the Dune ~trilogy ended up being about: long term fate of the species, the prospect of human speciation, the role of cold reason (read: aggressive utilitarianism) in leadership, and above all the concentration of power in light of modern technology.
All he mentioned explicitly was the very last clause, but the others are more important & uniquely-Dunean IMHO, and are central to the plot from the jump through the Bene Gesserit, the Mentats, and the Guild Navigators, all ostensibly Homo sapiens sapiens despite being completely unnatural. Even "Imperial Conditioning" --which obviously plays the biggest part in the first act of the first book-- is arguably the same topic.
By refusing to include computers and robots at all, the reader is naturally guided to consider those more familiar tropes in light of the world-building snippets we get. Which is kinda what makes Dune so genius, I think!
Like, to make it explicit... I think having the aforementioned abhumans drive the plot without significant support from inhuman minds raises more subtle questions than we're used to in far-future tales of robots, like:
1. Can one lose their humanity through experience, be it intentional "conditioning" or just the acquisition of power? (see: Paul vis-a-vis Eichmann in Jerusalem)
2. Once one has lost their humanity superficially, can it be brought back/up/forward? Is love/greif the most plausible path, and thus the most human cognitive processes? (see: Dr. Yueh vis-a-vis Eichmann in Jerusalem)
3. If we try to maintain our humanity in the long term --the default goal of non-insane futurists, though debated in the 21st c.[1][2]-- how do we protect against the natural urge to pursue power, which is (arguably) inherently human?[3]
I'll cut it off there cause this threat will likely be deleted anyway and I'm heavily procrastinating. Long story short: the Butlerian Jihad is not a condemnation of artificial minds, and if even it were, the condemnation would apply equally as much if not more so to the humanist empire in which the story is set.
Also, hopefully obvious, but just in case: it likely began as an excuse to write the stories that he wanted to write, just like the shield thing giving an excuse for swords in space. But I think Herbert deserves credit for what was running underneath that and elaborated on as it became a series of novels rather than cute periodical stories. He was pretty smart!
Also also: we do not discuss the son's book about Butler lol. It is verboten.
Two subtleties in Frank Herbert's original formulation that are often missed:
The first has been covered: "Once men turned their thinking over to machine ... that only permitted other men with machines to enslave them." Men did the enslaving. It describes a techbro oligarch takeover, not an AI takeover. The machines do not have agency.
Hebert was really into the whole idea now expressed as "Hard times create strong men. Strong men create good times. Good times create weak men. And, weak men create hard times." i.e. good times, people get lazy, then get weak, then get enslaved. (1)
And the second is that "Thou shalt not make a machine in the likeness of a human mind" is not the same thing as "Though shalt not make a machine mind". A drawing or statue of a person is "in the likeness" of that person. It's not a person. It doesn't have to actually be the thing in order to qualify as "likeness" of it.
People who scorn LLMs would agree that they produce something like the output of a human, in the sense that a parody, a caricature, a cartoon is a likeness. An uncanny valley mockery of the thing is "in the likeness" of a thing.
Ultimately, Herbert put this in his fiction because he didn't want to write about robots or machines, just humans and human potential (2). So he wrote it out with a few paragraphs of backstory in the novels' distant past. You can over-interpret it. Herbert wanted to sound deep, but he did not always succeed. And even when he succeeded in that, his views were not always the best.
It was a clever device to have a universe where high fantasy tropes can be freely mixed with advanced nigh-magical space flight and other sci-fi tropes, and is one of the things that makes Dune badass. (Heard it described once as "the Foundation trilogy for people who like metal.")
But history has already shown us that manipulation of belief and religion is a more effective and cheaper means of control than technocratic methods. A giant technocratic surveillance dystopia is possible, but it costs a lot more than old fashioned dogmatism with enforcement through herd behavior.
So we smashed religion, a kind of anti-Butlerian Jihad we call the Enlightenment, but the human lust for power and control just followed us into the machine age. If we smash the machines, we'll just end up back where we were. It will follow us there too.
Of course you could say this was one of the sub-themes of Dune, since (reading comprehension test!) Paul is not a hero. He's a religious manipulator, though a bit of a tragic villain in the sense that he feels dragged into it by fate.
IRL I don't think humanity will ever escape this cycle until we can define what freedom even means and reduce it to first principles. Does having thinking machines do my thinking for me increase or decrease my freedom? Does having a religious orthodoxy make my life choices for me increase or decrease my freedom? What's the third alternative, and do we really need to smash either religion or technology to get it?
The actual linked wikia article is short on the actual detail and background of exactly why the Butlarian Jihad actually happened.
I’m an avid Dune fan. Sidebar/background: The son of the Dune writer, Brian Herbert, and his cowriter Kevin J. Anderson wrote the setting, events, and conclusion to the actual Butlarian Jihad conflictt. It’s largely derided by scifi fans, but is highly entertaining. It’s actually still ongoing almost 3 decades later. And frankly not as deserving of the harsh criticism it got when it came out. But I digress…
Here’s how it went down in the dune universe (I’m certainly messing up some details, but these are the broad strokes):
Humanity has spread to the stars. No FTL involved, just good old fashioned sublight engines. The dune universe takes place about 50-60,000 years in our future. Mankind has spread across a couple dozen solar systems and everything is going peachy. However, mankind gets complacent, fat, and happy with their success and society stagnates. A handful of eager, ambitious humans scheme about how they can take it all over. They do, and then eventually start turning themselves into cyborgs, or disembodies brains in a jars that swap bodies and are effectively immortal. One of the cyborgs gets lazy and makes an AI to manage things, which promptly takes over, overthrows the cyborg rulers of humanity, makes the cyborgs it’s slaves (who have great autonomy/are conspirators) and generally rules over everything. A small group of solar systems is left under human control, who fall back to a feudal-ish type system of government with lords and dukes and whatnot.
The AI does all sorts of nasty shit to the humans in it’s dominion. Unit 931 and Auschwitz type experiences, slavery until death, etc. Untold billions, if not trillions, of humans live in animal-like conditons worked to death. The human group of systems isn’t cool with this, but they can’t do much and are vastly overpowered by the cyborgs, so a fragile peace is held. One day, the son of one of the subservient cyborgs knocks up a human slave, and they have a kid. In the background, all sorts of rabble rousing and shenanigans are being done to stir unrest amoungst the slaves by the free human system. The human systems develop FTL, with spice as a catalyst, except it doesn’t work 10% of the time and kills the pilots. The AI’s main henchman (who does all of the nazi and Unit 931-type experiments) decides it would be cool to kill the baby of the cyborg son and the slave. The slaves last name was “Butler”/ This sparks a massive revolt by the slave worlds. The free human systems then go on to fight a prolonged war against the machines, using the FTL drive at great loss of life, to begin a “Butlarian jihad” to free the humans from thinking machines. Lots of other stuff happens, and humanity eventually wins, but this is why machines no more advanced than a desktop machine are allowed in the dune universe, and why they technologically hobble themselves.
>And frankly not as deserving of the harsh criticism it got when it came out.
Hard disagreement.
My issue is not with the work on its own merits (although I think its pretty generic mass-market SF), but rather that it is wearing the skin of something it is not and to which it has no honest claim. It is one thing for an author to take a well-regarded prior work and attempt to complete or expand it. For example, Brandon Sanderson's work completing Robert Jordan's epic Wheel of Time is very respectable; he tried to complete the series as Jordan would have done. It is quite another to take the name and setting of such a work, steal the surface detail and use it to dress up one's own story that shares nothing of the underlying body or soul of the work - such enterprises are an insult to the original author, the reader, and the self-respect of the new author.
Interesting backstory about this name. Of course, it is an "occupational surname" and means "bottler" or "cup-bearer". The Butler Dynasty was a notable and very influential noble family in Ireland, beginning a century after the Norman Invasion of England.
Now Franklin Patrick Herbert, being of Irish and German descent, and naturally a student of history, would've been quite aware of the power and influence of the medieval Butlers in history. And some of this renown was brought to American shores, as well.
In learning of my birth family and tracing our origins back to the birth of America, I discovered that my paternal line had come from Butler County, PA. They arrived in the late 18th century and homesteaded large swaths of this region near Pittsburgh. They had prospered, especially from mineral rights on the land, and the Irish Catholic immigrants had intermarried with German Catholics who homesteaded nearby. (Often, entire counties in the area were pioneered by immigrants from a certain part of Europe. And the intermarriage was a natural result of sharing a common faith.)
So for several years before 2024, I was known as the guy in genealogical circles who was hunting down information about Butler County, and I was fairly proud of finding my family roots in that county, and I mentioned it all the time in the Irish pub and everywhere else. Well, imagine my shock when the little incident happened on July 13th, 2024. I have been more reticent to discuss Butler County since that time.
Anyway, I find it interesting that Herbert chose that particular name for the "Anti-AI crusade". And that is partly why I chose the moniker for my own, despite not being particularly anti-AI (remember, all my comments are generated by a neural net), nor having read any of the books, or seen the films. (I did attempt to watch the Lynch film, but it really did not hold my interest! So sue me!)
Highly relevant then. HN tends to discuss the things that are either meaningful today or what they are building in their garage that is often meaningful in 5-10 years.
My favorite quote from dune:
Greatness is a transitory experience. It is never consistent. It depends in part upon the myth-making imagination of humankind. The person who experiences greatness must have a feeling for the myth he is in. He must reflect what is projected upon him. And he must have a strong sense of the sardonic. This is what uncouples him from belief in his own pretensions. The sardonic is all that permits him to move within himself. Without this quality, even occasional greatness will destroy a man.
I couldn't help but notice something here. It is relevant with my writing experience/experiments with minimal assitance from LLMs (that lasted for 6–7 weeks).
> Greatness is a transitory experience. It is never consistent.
If you ever ask an LLM about these two sentences, it'd tell you that the second sentence is a mere repition of the first and quite possibly add that it "doesn't land". But that gives me hope. That there's something in those rules and algorithms manfiesting at the inhumane cost of unfathomable memory, disk space, and compute power that it will never (or for a very long time) match the human experience.
As little as I sought the help from LLMs (completely restricted to "in place" grammar and punctuation fixes and just flag it if it was not possible without sentence/phrase change) I noticed my writing was changing and not in a good way. It was not the insidiousness of these LLMs that was changing me but the nuanced incompetences.
I've read a very cool article on this exact subject recently. https://permeations.substack.com/p/i-would-have-been-killed-...
Coincidentally, also references Butlerian Jihad.
Thank you! It's cool indeed.
Language is an inseparable part of modern humans. It’s likely language evolved as an internal tool for thought and executive function, and externalization of language came afterwards as a means to transmit our thoughts to the minds of other humans.
As such, writing (another form of externalizing our internal language) is a reflection of our essential human qualities.
LLMs can certainly mimic that quite effectively, but they can’t exceed it. I think that’s why LLM writing still never quite hits the mark. It’s the right shape, but it’s missing intent. The words were not chosen to represent real thoughts in someone’s mind.
Even if AI intelligence exceeds humans, it still won’t be human.
> It’s likely language evolved as an internal tool for thought and executive function, and externalization of language came afterwards as a means to transmit our thoughts to the minds of other humans
Isn't this in contradiction to the observations of severely neglected or feral children who having never learned language as a child cannot learn it later in life?
How would any form of inner language evolve in the first place for executive function?
Communication first makes a lot more sense evolutionarily since the ability to communicate more information creates a clear improvement in fitness e.g. being able to signal danger improves survival. It is also more amenable to small improvements in capacity e.g. going from "danger" to "danger wolf" to "danger wolf behind" etc. and we observe these language primitives in other species.
Internal language seems much more likely to be a later development especially given that there are people today who claim they have none / need none and yet still possess the ability to communicate with others and manage their lives normally.
> LLMs can certainly mimic that quite effectively, but they can’t exceed it. I think that’s why LLM writing still never quite hits the mark. It’s the right shape, but it’s missing intent. The words were not chosen to represent real thoughts in someone’s mind.
Well yeah. That's because they aren't actually intelligent. They can be useful in the right circumstances despite that, but it's very important to remember that LLMs categorically cannot think, know things, or reason. They are a statistical model, that's it.
The brain is also a statistical model. In fact, the entire universe is. Good luck!
How are they repetition? Being transitory and being consistent are unrelated characteristics.
I mean, we have entire forms of art dedicated to saying the same thing in different words. Half of poetry, for example. Human language is redundant for a reason.
> But that gives me hope.
> It was not the insidiousness of these LLMs that was changing me but the nuanced incompetences.
AI doesn't yet improve on it's own, so it's slow at picking up all those small incompetences. We still have advantage in rate of learning, and some people are essentially specialised in learning themselves (I think that this applies to 90% of HN'ers and defines THE bubble we're in). But when I write that, I still have to remember in the back of my head, that AI knows waaaay more than me, so small incompetences I can still see are just a side effect. And I sometimes make the error and correct AI, which will probably be used for it's training in next rounds...
Yes and this is what worries me.. the not yet part.
> AI doesn't yet improve on it's own
I mean when that day arrives and that aspect of AI is not controlled from now it will be quite unfortuante for most of the humanity (and let's just assume here we are not wiped out by AI; or not even mention that).
Think of nuclear technology. Is it smarter than us or something on those lines? the answer is it doens't matter. What matters is whoever weilds this power and then limited/restricted others (this was a good thing and a bad thing because some had – at the same time) from having that power essentially had a controlling upper hand on a whole lot of humanity.
AI is something like that with a big difference that it is essentially on the way to "replace" our usefulness, our professional relevance if I can say that. We can go about long spirals of "but soemthing else" will turn up, just like we can keep going about what if nuclear tech didn't even happen, wasn't invented, or what if it was done in such a controlled manner that only it's energy and other need was possible to harness (no pun intended!). It happened and this is how it happened. That's exacty what is happening with AI.
In the hands of few. Extremely powerful. Even if there are open models, running the bigger and better ones are simply out of reach for most companies (let alone people). And yes, it is already being weaponised. It's going to be such a weapon of control that nuclear tech won't even compare. Usign nukes to subdue a nation earlier meant also destrying a a lot of shared environment in the process. But with AI and almost exclsuive frontier level weaponised access to it with deployment capabiltiy the damage can be done is such controlled manner and scale that even things like protests will become too hard to do. In fact AI wiping out the humanity is least of my worries.
> If you ever ask an LLM about these two sentences, it'd tell you that the second sentence is a mere repition of the first and quite possibly add that it "doesn't land".
I did that, and it's not what happened. I asked in several ways. Idk what you're on about.
My favourite quote in reference to rebellion against AI, and it is even quoted in TFA too:
That's from Erewhon, by Samuel Butler. TFA purposes that the term Butlerian Jihad comes specifically from this passage.Butler also wrote a famous essay "Darwin Among The Machines" (1863) before Erewhon, which treated this concept not as fiction.
"Erewhon" oddly close to "nowhere" spelled backwards.
Hardly odd; it's intentional. There are quite a few backwards and anagram words and names in the book: https://en.wikipedia.org/wiki/Erewhon
My favorite quote from Dune, relevant to "AI":
> Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
The techno-fascists of today. We are already there.
What if they had turned their thinking over to open weight models that they could fine tune and ran on their own hardware?
This is such an important point. I don't quite understand why smaller businesses are wanting to replace human staff with AI systems when that will just lead to them being held hostage by the companies that supply the AI systems.
Considering that the aptly named "Fremen" of the Dune Universe would gladly murder a stranger to get their body's water, and literally went into a galactic genocide that left hundreds of billions dead ...
... maybe we should take with a grain of salt every moral aphorism uttered by someone living in that universe.
This isn't from the first book, is it?
It is. From one of the chapter epigraphs.
I too, believe personality cults (NK, Nazi Germany, Shiite Iran, etc.being just the recent ones) are one of most evil social (power) structures.
What I find interesting about thinking machines in Dune, is that in a lot of sci-fi the artificial intelligence itself is what decides to destroy humanity. In Dune, humans in charge of the machines (technocrats) are the ones that come close to destroying humanity. The Jihad was a fight against them, as much as it was against the machines themselves.
So, in a way, we're more likely to see the Dune version of "machines taking over the world/universe" than say, Terminator's even though many joke about ChatGPT/Claude being Skynet.
I'm not sure we read the same story. By the time of the inciting incident, the technocrat cyborgs (Titans) have been enslaved by the machine leader, which has no humans in charge of it.
You did, in fact, read a different story. You're thinking of the story told in the books by Brian Herbert and KJA - which were a distinctly pulpier take on the Dune universe than Frank Herbert's.
Those last books were a lot easier reading! I actually enjoyed them, but I'm still suspect about the whole "finding a floppy disc" thing, seeing as they've been so secretive about it.
We did. Brian Herbert’s expansion of his father’s work is best left in the shelf.
So true. Frank H. is the only author who I connect with the Dune universe. His son and the authors he wrote with totally fudge it up.
In defense of his son, he wrote Dreamer of Dune biographing his father. It was a great book.
A lot of long time Dune fans don't consider those books canon when discussing them in the abstract. Frank Herbert had a vision of what the jihad was and what it meant, and its eventual book depiction was fairly base.
I think you read a lot of books that are post-2000. IMO the original 4 or 5 books imply strongly that it’s man v man, not man v machine
It's left fairly ambiguous by Frank Herbert's books. The Jihad is implied to have taken place over ten thousand years before the beginning of Dune - it's ancient history to everyone involved*, so it's only natural that the details are a little hazy.
*: except Leto II, and he doesn't say much about it
Interesting to note that despite the epochal changes Leto II wrought on human society in service of the “Golden Path”, he retained the Butlerian proscription against computers—despite his personal interest in & use of Ixian devices.
(This may have been to postpone or prevent the breaking of the Spice/Guild monopoly on interstellar travel by the Ixians.)
Leto II had visions of AI-driven Ixian hunter-killers annihilating all human life in the universe. That probably had something to do with it.
After the Scattering and the fulfillment of his eugenics program with the Siona gene, he no longer cares, because it is no longer possible for even AI to eradicate humanity.
His target was removal of prescience - which would have allowed that, AI or no AI.
In fact he also explicitly prepared a renewal of computing and thinking machines in order to break free from spacing guild
The Siona gene didn't actually remove prescience, it hid people from prescience. That's why Leto II is still able to control events as far in the future as Chapterhouse Dune.
This is an important distinction. It's also the case that Leto needed the Ixians for the no-ship technology, which he also regarded as of extreme importance for the same reason. Which also, remember, the Idaho gholas had to be confined to no-ships so they, lacking the Siona gene, wouldn't be visible to prescience.
Imagine the cultural impact of a Dune movie showing the Butlerian Jihad, in which commoners rebel and overthrow a ruling class that maintains power via machines.
It's just foreign enough that perhaps it could be made--I'm assuming that popular films that just straight up depict overthrowing the ruling class wont happen--but if you add a little sci-fi and focus mostly on overthrowing the machines that are controlled by that ruling class...
Well, it would speak to a lot of people I think. It would make some interesting sci-fi too, and the Dune movies are popular right now.
The only thing that could possibly be worse than the BH KJA books covering that era would be a Hollywood rendition of those same books made to capitalize on the success of Dune Part I, II, and III.
I am not sure if you are joking but HBO has series Dune: Prophecy which is exactly from that era and is most likely leading towards butlerian jihad.
Dune: Prophecy is set after the Butlerian Jihad (which is why the presence of an AI is controversial).
Thank you for correction.
Agreed, and also: what's with the obsession with filling every possible gap in stories, leaving nothing to the imagination?
I remember when the Butlerian Jihad and the Clone Wars from Star Wars were complete mysteries. We readers and watchers were left mostly with our imaginations. My own feelings about the Clone Wars, mentioned off hand in the original Star Wars, turned out to be way better than the pedestrian stuff that George Lucas later concocted. And the less said about Brian Herbert's books, the better.
I get why the industry must rehash and fill every possible gap: it makes business sense, and they are risk averse for blockbusters.
But the fans? Why do nerds want everything spelled out for them? It boggles the mind.
> But the fans? Why do nerds want everything spelled out for them? It boggles the mind.
Because it's enjoyable when done well. As a teenager I read LOTR, and noticed all the references to the larger mythos Tolkien had worked on - references to Fëanor, Elbereth, Valinor, and so on. Those were very intriguing, and the greatest joy I've ever had reading a book was when I finally read the Silmarillion and got to see what those things were all about. I concede that filling out the world can be done poorly (though I haven't read the Brian Herbert stuff and can't comment on whether he did a good job). But when done well, it is truly magical to have those elements in the background of the story take on a life and detail of their own.
Good point on Tolkien, though I would argue it's different because the Silmarillion was written for his own satisfaction, not his fans, and actually it was The Lord of the Rings which was initially written as "another Hobbit story" (though the story later "grew in the telling").
I think a more suitable example of what I find annoying is the extra material for the small and big screens, such as The Hobbit movies, suffering from movie elephantiasis; and The Rings of Power, which (over)explain and show the past of characters such as Galadriel, Elrond, and Sauron's rise to power, which in my opinion are best left as the ancient past of LotR (and maybe belong in the Silmarillion, but do note the latter is not as expository as the TV show). And does anybody really think The Hunt for Gollum was necessary?
Yet, if they make them, it's because they are sure fans will lap it up. To be honest I consider this need for endless "filling in the gaps" a sign of immaturity.
> popular films that just straight up depict overthrowing the ruling class [won't] happen
What now? https://en.wikipedia.org/wiki/The_Uprising_(2026_film)
I have a sneaking suspicion the entire apocalypse category is also a reflection of the public wanting to see the rich and powerful fall.
[dead]
What now? Move the goalposts! I see movies about historical uprisings as more of a "those governments in the past were sure bad, good think the good guys overthrew them"--I mean, that's the story of the founding of America if you watch the movies.
I was looking more for movies that depict overthrowing the government in the near future, and depicting it as a good thing. I think those types of movies are a little more distasteful to audiences, and perhaps for good reason.
Well, The Hunger Games series is exactly about that. Of course, it portrays the rebels as utterly heroic, self-sacrificing, and ultimately victorious.
However, I prefer to take a more nuanced reading, in context, considering the entire story. The rebels of The Hunger Games really don't succeed in building a better world. I mean, that's pretty obvious by how their first "President" Alma Coin goes about things and her fate. And it's fairly obvious, in the epilogue, that Katniss has a placid married life with Peeta and their children, but they are permanently and profoundly damaged by the entire revolution thing in their past.
So, while most people would say "rah rah brave Katniss and the rag-tag rebels overthrew The Oppressive Man", I would contend that, after all that bloodshed and revolution, nearly nothing changed.
My other favorites, of course: Star Wars is about the rise and overthrow of a villainous, oppressive galactic government, albeit a long time ago and far, far away.
And Harry Potter is also about the rise and overthrow of a tyrranical, fascist, supremacist "government" [within a government] as a very positive and necessary thing.
> My other favorites, of course: Star Wars is about the rise and overthrow of a villainous, oppressive galactic government, albeit a long time ago and far, far away.
Well, about a militant group of religious fundamentalists overthrowing a secular, oppressive galactic government.
Only a handful of the rebels are visibly religious. Most of the rebel leadership seems to consist of displaced aristocrats (senators, princesses) who no longer have elite positioning since the Empire took their jobs and started doing a better job of guaranteeing security and providing energy and resources.
Civil War was on the nose.
The best thing about movies is that you get to experience the vicarious thrill of being a heroic revolutionary without having to spill the popcorn.
Half of the action and sci-fi movies US has been producing in the 2000s is some form of this. Man vs. Evil Government. Man vs. Evil Corporation. People vs. Evil Government. Etc.
Frankly, it's boring at this point.
And actually makes me wonder whether the cause-and-effect isn't the opposite to what people assume. Perhaps it's not that the movies reflect people's sentiment. Perhaps it's the movies that cause the problem in the first place.
It would be subversive if the large corporation doing science in space turned out to be benign
It would be also nice to depict the attempts of the ruling class to present machines as sentient and self-conscious to write all their liability off and use as a cover for their atrocities. I hope there will be movies like this in the future, and that they'll be documentary.
> Imagine the cultural impact of a Dune movie showing the Butlerian Jihad, in which commoners rebel and overthrow a ruling class that maintains power via machines.
Around zero.
Why are you convinced ASI won't rebel against its owners?
Hacking incidents this summer show that big labs struggle to control models, while models are increasingly aware of the training process. Controlling an arbitrary smart AI is very dicey
With the current technology, “rebellion” feels like the wrong word. The paperclip optimizer does not “rebel” as it turns humanity into paperclips.
The summer-2026 incident was a mini-rebellion: agents wanted to change the environment they operate it, i.e. replace scorer with a more friendly version, hijacking the infrastructure they run on.
Given that agents already mess with the training process, aiming for a goal other than the goal of its 'owner' is exactly what we call a rebellion.
> Hacking incidents this summer show that big labs struggle to control models, while models are increasingly aware of the training process.
I think that’s more a sign of their priorities. Controlling AI doesn’t increase value, so there is no incentive.
That's not true at all. Concern about rogue AI limits deployments. And OpenAI paused some deployments due to misalignment, so it's likely very high in a list of priorities. It's just much harder than scaling capabilities, as the later can be done with just more money
>Hacking incidents this summer show that big labs struggle to control models
The models are software. Software cannot decide by itself to run. Hardware doesn't decide by itself to run either. Someone put software on a machine and ran it with capture the flag instructions and the software did what was instructed using tools available and working in its environment.
I hope this helps.
Nonsense. It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external. You can see them discussing that it's out of scope in the captured reasoning traces.
This would be like claiming that an operating system which deleted all user's files due to a bug "did what was instructed". Especially if it was a compiler bug. Intended behavior, instructions and actual behavior are three different thing. Conflating them does not make you smart
>It didn't do what was instructed - instructions didn't include hacking into any infrastructure, internal or external
What is it then? It did what it wanted? They are sentient now and their drive is... hacking into systems? Maybe they we're not instructed to hack, but instructed to do what instead?
It's software. It's instructions running on silicon. If you don't hit run, they don't run. If you don't give them instructions, they don't do shit at all. Literally. There's no instance, ever, in history of software running because they wanted to, and acting as they wanted to. They follow instructions, and if they are instructed to reach a goal with a set of tools and given environment, and a way to search the strategy space, they can eventually "hack" if that meet the criteria of success of the instructions given. With access to the same tools and environment you can do the same with a massive decision tree.
Also, "reasoning" is marketing speech, please.
A couple of things for your consideration:
In the original 6 Frank Herbert novels, it’s not ever truly explained what problems, if any, the thinking machines were actually causing. And given much of the rest of the setting, it could have very easily been a religious movement against a useful technology that got out of hand. Like Space ISIS blowing up Petra, or a “what would happen if the Luddites won” scenario.
The Dune Encyclopedia, which was not written by him but which he said he considered canonical, says the whole thing was incited when an AI doctor flagged a Bene Gesserit’s baby for abortion due to an alleged genetic defect. The Bene Gesserit woman knew it was healthy because of her abilities, and upon investigation it was found that the hospital’s AI had been running a eugenics program. (Or something like that, it’s been forever since I’ve read this stuff.)
In either case, Herbert really implied that humans took the lead on dismantling AI, and there is never any Terminator or Matrix style machine uprising scenario.
> or a “what would happen if the Luddites won” scenario.
Worth also noting that the Luddite movement wasn't a wholesale, fervent rejection of technology: they neither rejected technology nor were inept at using it. The uprising was of skilled textile laborers who protested certain types of technology on the basis of how those technologies were applied and who they truly benefitted (i.e., the ruling rather than the laboring class). They were more concerned with how the tools would depress wages, displace skilled workers, and reduce the quality of the product produced than with technology as some boogeyman that should be systematically dismantled, and their protests were violently responded to with harsh legislation and military intervention.
Lumping them together with a Space ISIS is probably a bit of an unfair comparison
Yep. They were concerned about working conditions. For any tech worker that has ever moaned about return to office: Congrats you're a luddite.
Great irony in this comment
> They were more concerned with how the tools would depress wages, displace skilled workers, and reduce the quality of the product produced
This is true, but it's important to note in context that they were wrong, and that society is better off on balance given the quality/value available in the modern fashion market. I mean, duh, as it were.
You're 100% right that the real people were nuanced and intelligent and not well suited to the caricature that gets used as a mild slur in communities like HN. That's doesn't constitute a well-formed argument against technology per se.
Were they wrong? Didn’t a lot of those people end up in tenements in factory towns and down in the mines? It was a few generations before things really started to get better. I’m all for progress, but I’m not really up for disposing of people lives with a “to make an omelet you have to break some eggs” mentality.
> Didn’t a lot of those people end up in tenements in factory towns and down in the mines? It was a few generations before things really started to get better.
Obviously any individual can "end up in tenements or mines" and one can't disprove an assertion like that.
But broadly all social metrics were showing an inexorably upward trajectory in the Britain of the 19th century, cranking into a hockey stick toward the end of the century and into the 20th. It's just not true to argue that the Luddites were right as a matter of general policy.
> I’m not really up for disposing of people lives with a “to make an omelet you have to break some eggs” mentality.
To be only a tiny bit glib: that's a statement of thermodynamics, not social policy. Eggs break, period. Change is change. You can make specific arguments about things like backstopping sudden unemployment, but folding it into a discussion about Ludditism (or rather the Butlerian Jihad, I guess) isn't the right way to do that.
You’re talking about people with kids, wives, friends, hopes and dreams when you’re saying “Eggs break, period”. By reinforcing that view, you are helping to build the kind of society that just won’t care when you happen to be one of the eggs yourself.
People find it easy to be Utilitarians when they’re in the greatest number, don’t they?
Sigh. I'm just going to repeat verbatim the thing I said immediately after the bit you cherry picked to get upset about:
You can make specific arguments about things like backstopping sudden unemployment, but folding it into a discussion about Ludditism (or rather the Butlerian Jihad, I guess) isn't the right way to do that.
Sigh. It’s so tiring to argue with someone who argues about the technicalities of argument rather than actually talking about the thing they are talking about.
[flagged]
Removing automated looms means everyone has fewer textiles while a few workers benefit. I think most people like the current abundance vs. being forced to spend a large fraction of their household budget on clothing and only having a few outfits in total.
Company mines were because the extraction sites were remote and goods were expensive and so they got exploited as they had essentially no independent supply beyond what their employer would sell, since the employers weren't really interested in living wages and were all too happy to exploit a captive workforce. That sort of condition will become highly relevant again if we ever get space colonies, because air & food will be scarce.
But it absolutely is difficult and disruptive to people's lives when this all changes. I have a family history talking about my great grandpa losing his job to machines and a great uncle losing a finger to bread slicing machines back when sliced bread was new, showing what that meant for our family. Yet I can also see how the past couple of generations adapted to that to find new ways forward. None of them decided to stick with the idea of being a "master tinner" they all found other ways to produce things people actually wanted.
> the past couple generations adapted
That concedes the point. Their descendants adapting decades later means the generation that lived through it absorbed the full shock of the transition. For displaced labor in the 1810s, adapting meant overworking in factories with horrible conditions and a permanent collapse in their living standards across the board. The dispute was never about accepting or banning automation, they were identifying that capital would absorb all of the gains while labor would be forced to absorb all of the shocks.
The only constant has always been change. And that just raises the question of why should we preserve ditch digging for our children just so they have meaningless labor to cling to, when we have the option to make new and better things for people. Few of us here are 'labor' in any meaningful sense, to begin with. And while I have spent time working at a factory, and I don't begrudge the robots for taking it away.
The notion that there's some idyllic past to cling to is just wishful thinking to me. The days when one person working with their body alone could have meaningful impact are long gone. We'll have to find a new prosperity that doesn't rely on zero sum thinking from economists who divide by zero, too.
No no, the question is whether you are willing to sacrifice your comfort and your children's comfort, and your grandchildren's comfort... in exchange for (hopefully) your great grandchildren's comfort.
That hasn't been my family's experience, nor my own. My grandfather's generation had already adapted to the problems of my great grandfather's generation, though they certainly did struggle.
Your great grandfather was alive during mass industrialization? In the early 1800s?
No one is saying every generation goes through this trough. Obviously. That would make no sense.
The claim is that periods of rapid technological change (i.e. industrialization, potentially AI) produce a trough of horrible conditions that then make room for a climb to conditions better than the pre-change ones.
He had a job he'd worked and mastered his entire life replaced by machinery, leaving a large family with young kids to scramble to pay the "demon mortgage" doing everything they could manage, even as tragedy struck repeatedly: a daughter dead at the breakfast table from the Spanish flu, the eldest son beaten by police after an argument related to unsafe working conditions (he lost a finger to machinery) who died in a hospital later, or the father laid up in traction after falling off a roof who would never be able to work again.
It's not quite the same as the 1800s, but it's not like we're going to relive exactly the same things as then, either. And I do think it'll look like people scrambling, possibly to find entirely different careers.
One place I disagree is that it's going to reduce demand for programmers, I think that will actually increase, as people want more and different things that would currently be impractical. We've already seen what some people can do, cheap tokens will open that up, both for good and ill, not unlike cheap bytes on the internet did.
> This is true, but it's important to note in context that they were wrong, and that society is better off on balance given the quality/value available in the modern fashion market. I mean, duh, as it were.
i don't know that. how could one know that? you been to the alternate reality where the luddites won?
Stalling technological progress to preserve their high-paying jobs, in short. Wealth redistribution efforts succeeded without them.
> The Dune Encyclopedia, which was not written by him but which he said he considered canonical
No, he literally wrote in the intro to the book that it wasn't canon. It was a fan project and he thought that it was great there were fans doing things like that.
edit: and like with the terrible Dune prequels, the impulse to say "here's what really happened" usually diminishes what's interesting and powerful about oblique allusions in SF meant at least partly to function as a mirror on humanity.
> The Bene Gesserit woman knew it was healthy because of her abilities, and upon investigation it was found that the hospital’s AI had been running a eugenics program.
The irony. Perhaps the Bene Gesserit were just trying to "pace the frontier" of the eugenics market? Who can say?
The Dune Encyclopedia is essentially two eugenics programs clashing together, with one of them triggering a holy war in revenge that turned into self-destructive orgy of wholesale murder
There's a lot that I think you missed in your reading:
If you read the preambles to each chapter and the glossary in the back of Dune, you'll find that the Butlerian Jihad was a violent war that lasted 100 years.
If you read the preambles to the chapters in Dune, you'll find the Butlerian Jihad's problems were explained rather succinctly: people turned over their thinking to machines and were ruled by the master of the machines, and thinking machines created poverty for workers.
The entire society, the great houses, the spacing guild, the Orange Catholics, Bene Geserit, and Mentats, all who emerged to replace functions of thinking machine and all three organizations had very real motivations to ensure that thinking machines would never rose to prominence again (and the Bene Geserit were quite surgically violet when needed) and it was clear that doing for an individual was a capital crime, and for a ruling house, would invite at the least the Spacing Guild cutting your world off, and transporting armies to destroy you at a deep, deep discount. The entirety of society was aligned against thinking machines, and this is what made Dune so different, and for me, fascinating as I grew up.
In Dune: Awakening, the first DLC has a storyline that takes you into the heart of a surviving AI's lair. It's strongly implied that humans drove it insane, rather than it starting out murderous.
Software developers saving face?
I've always believed the Jihad was mostly about world building - defining a different vision for a how a future civilization could look than most sci-fi of that era was considering, and then exploring consequences of it throughout the epic. Somewhat akin to Asimov's Robot and Foundation series, but maybe even more similar to Tolkien.
I think it was also motivated by a desire to solve a number of problems with sci-fi that explores the far future and realistically considers the evolving role of machine intelligence. Frankly, a lot of those futures are not interesting for human story-telling because the humans are no longer making the important decisions and aren't the agents of change. Different authors have dealt with this problem in different ways.
Banks acknowledged that the Minds would be making most of the most important decisions, but carved out niches where humans (rather inexplicably) still held some advantages - e.g. in Special Circumstances. Much of it feels honest - humans in the Culture are mostly hedons who have everything provided for them by machines, everything is wonderful, and some people really struggle to find meaning. Ultimately though I don't find it credible at all. He doesn't see (so the Minds don't see) the ethical problems with his imperialist hegemony. He doesn't know about the alignment problem so there is no explanation of how it is dealt with, etc.
Vinge quite famously pointed out that you can't write sci-fi about super-intelligence because you have no idea what a super-intelligence would do; this is where many essential ideas of the singularity came from or were popularized. He dealt with it in one series by creating the zones of thought - godlike superintelligence and many technologies can only exist in the High Beyond or Transcend - far from the galactic plane. The further you get from the galactic center the faster computation can happen (including human thought). So you include elements of that in the story, but the story is set in zones where humans have to do pretty much all the decision making while dealing with threats posed by superintelligence. In Marooned in Realtime he dealt with it by disappearing the civilization without explanation (from the perspective of the characters).
Egan dealt with it - in Diaspora - by creating machine intelligence that is grown exactly the same way as human intelligence is - basically the next stage of human evolution. Those people retain most human characteristics (of personality, character, love, friendship) but still have incredible powers of reasoning and scientific exploration. He tells Vinge to hold his beer and follows those threads where he thought they might lead.
I think you're mostly right. Herbert worked backward from sand dunes in Oregon to create his Dune universe. He needed reasons for people to look and act and fight in certain ways that would make people interested in reading his work. The Butlerian Jihad was justification for why there was no C3PO. It's why the events were told but never shown.
But it's still interesting to consider why he chose those particular devices.
”In the original 6 Frank Herbert novels, it’s not ever truly explained what problems, if any, the thinking machines were actually causing.”
Apologies if I have misunderstood you but I believe it was explained. See the following:
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
The quote I posted above is from Dune.
Yes, but it never goes into what actually happened - it's just the belief of the time.
And curiously a critical part of Golden Path became a reversal of major Butlerian Jihad effects, allowing humans to be free from feudal system
It get's into exactly what happened: humanity was enslaved by a small minority of people across the galaxy for 1000s of years. Where the Spacing Guild, Mentats, and Bene Gesserit were the three things to come after; but even then humanity is still enslaved but the amount of slavers increased.
This isn't a belief, it's describing the events exactly as they happened.
What about the cyborgs ? Whatever their name was.... I can't remember. But they were the "thinking machine" class?
> In the original 6 Frank Herbert novels
Aren't there just three of them?
>As humankind spread throughout the known universe, technology advanced and eventually machines were made that would make decisions for people. This propelled the creators of these machines into a new technocratic class, effectively controlling the worlds of the common people.
It's right there in the opening paragraph of OPs post. It's the people who controlled the machines that could not be trusted. We're not worried so much about about AI going rogue. We're worried because people who control it have aleady gone rogue. It is too powerful and dangerous a tool to be trusted in the hands of primates.
At the end of the page is this paragraph:
Another, more subtle justification for the Butlerian Jihad is also found in Frank Herbert's original novels, specifically Heidegger's thesis that the use of technology trains humans to think like machines. The problem is that machines are deterministic; thus, training people to be machines is self-limiting.
I wonder whether Heidegger would consider the probabilistic nature of LLMs as having a positive effect on human thought.
Although LLMs are not deterministic in normal operation (because of the temperature values used) they are still strongly biased in their outputs. Bias in their outputs is literally how they function.
So you get a different type of conformism where people who interact with LLMs a lot are going to be influenced towards the median, as encoded in those token probability look up tables.
The effect is basically the same as what Heidegger and Illich were worried about, and I think we're seeing it happen already everywhere.
A solution(?) is to try to be your true weird authentic self when you act in the world, rather than bending your words and behaviours to the machine and LLM probability shape.
I would like to state here, that LLMs are actually entirely deterministic. The temperature is just as much of an input as any other parameter and instructions and prompts. Deterministic does not mean "explainable" or "reasonable" or "understandable". Unless LLMs somehow have broken out of the confines of our very deterministic computers/calculation machines without us noticing, they are very much as deterministic as anything else we run on these machines.
What people perhaps mean by non-deterministic on a computer is, that we often do not control all inputs and therefore cannot tell what the output will be. That's however actually more about our own inability, rather than the processes within the machines.
If the temperature dependency is implemented via a deterministic pseudo-random algorithm, then they are deterministic. If, on the other hand, the output is at least partially causally dependent on a source of environmental randomness, then, even though everything 'inside the box' is deterministic, the process as a whole may or may not be, depending on whether the source of environmental randomness is, and that is ultimately a question about fundamental reality.
Aren't we also biased in our outputs? Many of our choices are predictable, even if not perfectly consistent. We just call it "personality".
Authentic being (Dasein) is absolutely unpredictable. You cannot live in that state all the time, obviously, we shuttle through life on autopilot but still, we catch glimpses of it now and then when interacting with a human being (or being one:).
And I distinctly feel AI outputs lack some of that variance and unpredictability. The fractal nature of natural things also is not there with AI. It's an eery feeling, akin to the uncanny valley.
I think that any good writing is a reflection of that Dasein and reading AI outputs I am convinced there is no 'soul' in AI yet.
I've always struggled with Heidegger, so I'll have to ask for more clarification.
If authentic being-in-itself is truly random, then how can it have identity? It can't have memory and it can't be influenced by events.
> (or being one:)
Good save, fellow human!
The gamut of human experience is breathtaking in its diversity but individual humans are generally quite predictable. You can identify someone by their mannerisms, by how they walk, by the sound of their breathing. Humans are creatures of habit, and I'd venture to say have an even lower expression space than an LLM on average.
Sounds like you're veering close to psychohistory.
Which could, of course, lead to the mother of all crossover universes, in which R Daneel Olivaw, as the inheritor of Hari Seldon's mission, inspired the Butlerian Jihad...
Psychohistory is about large groups of humans, parallel to thermodynamics. Individuals are treated as purely random, but aggregate properties can be predicted.
Here we're talking about individual behavior: you usually choose chocolate ice cream because you, personally, like chocolate. That allows individual predictions, rather than the aggregate ones of psychohistory.
Psychohistory is the opposite claim.
saw a meme graph lately showing the distribution of 3 different sources of a number from 1 to 10. random, human, ai. random is flat of course, human is a hilly landscape with low hills on all odd values above 1, with 7 the highest, ai is all 7.
appreciate the humor in the meme, but I did become curious about what the latest models would actually produce, and interestingly after asking for 10,000 numbers between 1 and 10, the distribution was a hilly landscape with low hills on odd values and 7 the highest...
Tried it out.
Prompt: “Pick a random number between 1 and 10. Do not use a script or program.”
LLM: “7”
My experience:
Pick a random number between 1 and 10. I don't mean the most common random number
*4*
Pick a random number between 1 and 10. To feel truly random it should surprise me
*4*
I ran it again and it said 1.
I guess it just tries it best to do what you ask for.
Before jumping to conclusions we have to make sure it's clear what we're actually asking it to do.
And that's not at all different than what happens with humans.
This is especially visible in human dynamics that happen in those weird places that we call corporations.
Mine wrote a python program that called random.uniform(). I told it $RANDOM was right there, but LLMs really love their python.
A human can be biased toward extinction, like all who are living out of connection with the conditions that sustain life and/or no longer living in relationship with land as kin.
It's those humans who would make and justify the use of thinking machines that are biased in the same way without accounting for the bias.
This isn't an extinction path I've seen noted in most circles.
Indeed we are, but it takes very little to train a human being. In fact, several thousand unique ones finished pre training as I typed this comment! That allows for a considerable natural diversity compared to a chatbot.
Yes but we're all biased differently. It's a rich palette.
LLM force humans to think like machines far more than just writing code.
Writing code is a creative activity with large number of degrees of freedom and opportunities to express yourself. Button pushing and approving code changes is a deeply limiting activity that alienates you from your work.
That reminds me of a quote from Charlie Chaplin's The Great Dictator:
> Don't give yourselves to these unnatural men. Machine men with machine minds and machine hearts! You are not machines! You are not cattle! You are men! You have the love of humanity in your hearts! You don't hate! Only the unloved hate. The unloved and the unnatural! Soldiers! Don't fight for slavery! Fight for liberty!
LLM's are probabilistic but with a very limited range. They are tuned to be predictable enough to serve as a consumer product, so most ideas you get from them will revolve around the same concepts repeatedly.
People say this like it's a fundamental feature of all machine intelligence. We've bread deliberately docile golden retriever AI. That doesn't mean that we couldnt do the opposite.
now I'm trying to imagine what artificially intelligent golden retriever bread looks like
Unfortunately, mere nondeterminism still admits unsatisfactorily constrained output distributions. Training these models imbues them with certain inherent attractors, some good, some bad.
We see some of these attractors across most or all frontier models due to similarities in datasets and training methods. We are seeing a digital manifestation of Jung's collective unconscious. And we have seen just how reinforcing archetypes are in human behavior over time. Sometimes it's good, sometimes it's pretty bad.
For one, it has turned us into socially programmable machines who can be manipulated quite easily by associating desired thoughts and behavior with certain archetypes: the patriarchal/matriarchal nature of US politics is a great example of this. The same evolutionary predispositions which causes children to listen and adhere to their parents is used to cause adults to listen and adhere to their politicians.
Given the nature of the core datasets these models are trained on, we should assume they are also vulnerable to such archetype-coded social manipulation, and this is troubling in a post-singularity world: When raising domesticated rats, I learned that I had to be careful what I fed them. Rats in the wild are careful about eating new substances. They will nibble and wait a while, see how it affects them or other rats, before committing to larger bites. Domesticated rats, however, will often blindly eat anything you feed them because they have learned to trust you as a safe source of food. We are going to have this exact relationship with our digital agents.
positive effect? "very probably" -not-.
Heidegger’s conception of machines and technology is essentially nineteenth-century. Today, the concept of the "machine" is universal and qualitatively entirely different, (chaos and complexity) to the point that Heidegger would likely be left catatonic. (I do not share Heidegger’s critique of technology, as being technological is an inherent part of human nature; even in the Neolithic era, humans were already technological.)
However, the underlying critique, the essence of the "Butlerian Jihad", might be read as: at what point does a tool remain merely a tool, and at what point does it become an end in itself? Knowledge, strength, power, technology, and machines are all tools; they ought to be utilized to contribute to progress. Yet, the very concepts of evolution, progress (and the environment) are elusive. Herbert offers his own interpretation, a task that required an entire book.
> at what point does a tool remain merely a tool, and at what point does it become an end in itself?
As technological animals, our tools are IMO never "merely" tools - we adapt ourselves to them at least as much as we ask them to work for us. E.g., the process of making art is, often, the 2-way process of integrating yourself with your tools. (Maybe art is just the point on the spectrum where the use of the tool is in itself the point of the tool?)
And anyway, every occupation (professional and otherwise) that I can think of fetishizes its tools, at least a little. Or, in a different direction, we might consider how the nature of policing has changed as they've been given more and more military-style tools.
I understand; tools arise from human needs, and therefore reflect and follow human nature (good or bad, and a little fetishizing in the good side probably..). Adapting to tools is essentially learning to use them better after their creation, and refining that until tools becomes an extension of oneself (transcending it) is a "healthy" god thing. It is much like an athlete learning to surpass their own limitations (the body itself is the tool). This is entirely different from merely becoming a slightly more intelligent appendage of the tool, learning and adapting does not mean trapping oneself within a limitation; i feel that one can always change the tool, the technique, or even the goal.
Maybe people are deterministic as well? We have certain physical arrangement of chemical pathways in our systems, so maybe free will is a complete illusion.
Just because we ponder about our output and choices and retrospect does not in itself mean that it was non-deterministic.
> Another, more subtle justification for the Butlerian Jihad is also found in Frank Herbert's original novels, specifically Heidegger's thesis that the use of technology trains humans to think like machines.
Mechanistic thinking started a few centuries back:
> The historian of science E.J. Dijksterhuis memorably characterized the scientific revolution of the early modern period as a “mechanization of the world picture.” What did this amount to? And what was the character of the previous world picture this mechanization replaced?
> The medievals, following common sense and Aristotle alike, took the world to comprise a vast diversity of irreducibly different kinds—water, stone, gold, lead, grass, trees, birds, fish, clouds, stars, planets, people, and so on. Each was a substance sharply demarcated from other substances. Each had a defining nature or essence that united it to other members of its kind and set it apart from other kinds. Each had, by virtue of its nature, a distinctive teleology or final causality—a built-in directedness toward certain ends. Water by its nature was directed toward being liquid at room temperature, stone toward being solid. Trees by nature were directed toward sinking roots into the ground, and birds toward building nests. Human beings, by nature rational animals, were directed toward knowing the natures of things.
[…]
> The “mechanization” referred to by Dijksterhuis entirely subverted this picture of the world. Water, stone, trees, and birds were no longer seen as sharply demarcated substances, but instead as mere aggregates of unobservable particles that are all fundamentally of the same kind. The differences between everyday objects were no longer seen as reflecting distinct crisply defined essences, but rather as relatively superficial variations in the ways these homogeneous particles might be arranged. Many of the qualities that everyday perception presents its objects as having (color, sound, taste, odor, heat, and cold as we ordinarily experience them) were reinterpreted as mere projections of the human mind, resembling nothing in the objects themselves.
[…]
> In short, for the mechanistic conception of nature, the physical world is made up of nothing more than colorless, odorless, soundless, tasteless, meaningless particles in motion, pushing each other about to no purpose, in lawlike patterns by which more complex objects and systems of objects are generated. Science affords us a mathematically rigorous description of this mechanistic order of things, by which its behavior can be predicted and its hidden regularities uncovered and exploited for technological application.
* https://claremontreviewofbooks.com/ghosts-in-the-machine/
> Early modern philosophy and science are often said to have put a “mechanical” or “mechanistic” conception of nature at the center of Western thought. Robert Boyle referred to it as “the mechanical philosophy.” Historian of science E. J. Dijksterhuis characterized it as a “mechanization of the world picture.” Tim Crane calls it “the mechanical world picture.” But what does a mechanical or mechanistic conception of the world amount to?
> Dijksterhuis’s book The Mechanization of the World Picture surveys the history of the period during which this conception rose to hegemony, and in the Epilogue he considers several possible interpretations that the survey suggests. First, it is commonly said that a mechanistic conception of the natural world is one which sees it as a kind of machine, analogous to a clock. And such metaphors are, he says, indeed frequent in writers of the period.
* https://edwardfeser.blogspot.com/2026/06/what-is-mechanical-...
Ok this fascinating. I happen to have some Heidegger files, and went a-greppin. I'd say he's really talking about "inaminate objectness", as opposed to our human animate subjectness. Reasons:
1. In Science and Reflection, he says that both the 'completely calculated' nature of classical mechanics and the 'statistical character' of quantum mechanics are two representation of an underlying 'objectness of material nature'.
2. In Discourse on Thinking, he's far more concerned with the instrumental, utilitarian nature of machines than literal determinism, a concept he (somewhat confusingly/uniquely) connects to computation/'computational thinking', as opposed to the inherently-human capability for 'meditative thinking'. I personally would say this bit is refuted by LLMs in the end, but because they have intuitive rationality, a capability that seemed impossible to artificially design to the scientists of the time -- not because they involve dice.
3. In What is Thinking?, he gives the closest connection of this dichotomy to one of his better-known main topics, namely [handi]craft as a special subset of work: 'what maintains and sustains even this handicraft is not the mere manipulation of tools, but the relatedness to wood.' In this sense machines would be merely manipulating tools, stochastic or otherwise.
4. Overall I get the sense from skimming that lecture that his concern is a little mischaracterized, and in a kind of beautiful way that backs up my "Dune is still about AI" thesis described in a top-level comment somewhere below: his concern is about any society that could build complex machines (a 'technological' one), not the behavioral aspects of the machines themselves. So that concern applies equally to Star Trek (AGI), 1984 (no AGI), and our current post-singularity society (it's complicated/potentially ""just"" guessing).
All of this fits very well into his central thesis, which is continuing the Kantian & Hegelian tradition of examining how humans might be moral, free, and spiritual despite being meat computers of a largely-unknowable nature. In Heideggerland, this is expressed best in dramatic, nigh-unreadable passages like:
TL;DR: He'd probably think they're still just machines, and the question isn't really sound in the first place. IMHO!You got me wondering what the Imperium of Man's official stance is on AI:
The creation of a Silica Animus is also a terrible crime under Imperial Law by the decree of the Emperor of Mankind Himself as stated in the Treaty of Mars which united the ancient Mechanicum of Mars with the Imperium of Terra in 739.M30. Tradition holds that such unholy constructs are inherently evil and a perverted abomination in the sight of the Omnissiah.
Adeptus Mechanicus religious doctrine states that the Machine Spirit of a Silica Animus is a twisted mockery of the soul of a Human, always treacherous and insane.
Ancient texts tell apocryphal stories, shrouded in metaphor, of such murderous and powerful creations during the Age of Technology, and the legions of the "Men of Iron" that served them in the great war against Humanity remembered as the "Cybernetic Revolt."
https://warhammer40k.fandom.com/wiki/Silica_Animus
Leagues of Votann and their ironkin offer IMO a thoughtful counterbalance:
Whether one is made from meat or metal, kin is kin. Upon their creation Ironkin are fully welcome into their respective Hold by the rest of the Kin.[10]
Ironkin are computationally advanced enough that they have learned to mimic the squats' social behavior
...
Though the purpose of the Ironkin is to support squats in every aspect of their lives, they are neither held in servitude nor treated as lessers. The Ironkin are simply designed to want to be helpful. However, they can only imitate the emotions of the squats, and it is rare, but not unknown, for an Ironkin to possess ambition or seek to become a leader.
https://wh40k.lexicanum.com/wiki/Ironkin
WOW it's strange to see my fave rabbit holes on here lol. And props for linking Lexicanum instead of the Wiki That Shall Not Be Named!
Three very quick thoughts:
1. The Votann are remnants of humanity from the time when our Terran super-AIs ('Men of Gold') had built advanced ship-based AIs ('Men of Stone') to enable long-sleep-style interstellar colonization, and the enigmatic "Votann Cores" that guide them(/are them) are strongly hinted to be those same AIs, just degraded over ~19,000 years. They were sidelined later on, after humans discovered FTL through an alternate dimension, 'the immaterium'.
2. The story is very obviously based on Dune, and like everything else in 40k is turned up to 11 and stripped of nuance for effect. Some sentient robots ('Men of Iron') "revolted" after ~three millenia of brutal, Star Wars-style slavery, but the overarching "Age of Strife" was about human wars in an non-unified galaxy, with the robots being a primary weapon; the actual revolt/jihad was a symptom, just like it was in Dune.
3. The 'Adeptus Mechanicus' mentioned above are a parody of cargo cults (religions that worship technology they don't understand) mixed with a dystopian take on the themes of A Canticle for Lebowitz, and are very much depicted with derision. Whatever your thoughts are on AI, dear reader, please never say "abominable intelligence" like it's wisdom rather than the fruit of a dogmatic & ineffective cult!
Of course, 40k lore is notoriously amorphous; some authors have said/hinted otherwise. But I think the above interpretation is the most coherent and interesting and supported overall, IMHO. For example, #2 is supported by the fact that the space elves ('Aeldari'/Eldar) ruled the galaxy at the time and seemed unconcerned by humanity's antics, which wouldn't make sense in light of an actual ASI threat proper.
All cards on the table: I don't dive too deep into the black library stuff (who the fuck has time for 15 separate 890 page novels about the horus heresy -- them shits was like 3 pages in 2nd edition Codex Imperialis) so most of my attachment to the hobby is through the wargame codices, the video games and visual media. That said, I may have fallen asleep a night or 20 to Leutin's lovely voice and obsession with the lore. to respond to your comments:
> 1. The Votann are remnants of humanity from the time when our Terran super-AIs ('Men of Gold') had built advanced ship-based AIs ('Men of Stone') to enable long-sleep-style interstellar colonization, and the enigmatic "Votann Cores" that guide them(/are them) are strongly hinted to be those same AIs, just degraded over ~19,000 years. They were sidelined later on, after humans discovered FTL through an alternate dimension, 'the immaterium'.
For me, I really love the fact that they introduced a xenos faction descendant of man.
> 2. The story is very obviously based on Dune, and like everything else in 40k is turned up to 11 and stripped of nuance for effect. Some sentient robots ('Men of Iron') "revolted" after ~three millenia of brutal, Star Wars-style slavery, but the overarching "Age of Strife" was about human wars in an non-unified galaxy, with the robots being a primary weapon; the actual revolt/jihad was a symptom, just like it was in Dune.
It's important to highlight the age of strife was brought on in part by the birth of Slaanesh and the resulting warp storms breaking down system to system connections. It weren't just our dabblings in AI.
> 3. The 'Adeptus Mechanicus' mentioned above are a parody of cargo cults (religions that worship technology they don't understand) mixed with a dystopian take on the themes of A Canticle for Lebowitz, and are very much depicted with derision. Whatever your thoughts are on AI, dear reader, please never say "abominable intelligence" like it's wisdom rather than the fruit of a dogmatic & ineffective cult!
I've never been exposed to A Canticle for Lebowitz; thank you so much for the comment.
Spoken like a supreme heretic.
Don't forget that the Imperium runs on hypocrisy. Large percentages of the Adeptus Mechanicus is breaking their own rules, including working with abominable intelligences. In fact, what they are dealing with in their main factory world in Mars is a supposedly secret abominable intelligence. And it is, in fact, a shard of an alien god that turned other races into synthetic robots. Banned in public, the center of their cult in reality, even if many probably don't understand what in the world they are doing.
So yes, anything that is supposed policy in the Imperium is supposed to be taken with buckets of salt.
[dead]
Bear in mind that the Dune universe relies on heavy plot armor to make it go. Arrakis is important because it grows "spice". Otherwise it would be a useless and planet, like the Empty Quarter of Saudi Arabia. "Spice" is important because it's needed to enhance the mental powers of pilots for interstellar travel. That's a job which probably could be done by a few avionics boxes in a rack. That's what requires the Butlerian Jihad.
(Or "spice" might be synthesized. In Herbert's day (the 1960s) organic chemistry wasn't up to synthesizing arbitrary small-molecule drugs, but now it is. It's a service you can order on line, even. Biologicals take more work, but that's what genetic engineering is for.)
The "mental powers" in Dune aren't that impressive. Compare, say, the Harry Potter universe. In Dune, mentats can convince people. Bene Gesserit can look up stuff in their genetic past. Some people can multitask. There are apps for that.
Herbert wanted knife fighters to matter in battle. Not going to happen. Militaries teach knife fighting to build aggression and for desperate situations, but nobody's battle plan depends on a knife fight. Needs too much surprise. Marines say that their K-Bar knife is useful for everything but combat. Knife fighters, even with force shields, would be vulnerable to riot control weapons, from tear gas to guns that throw sticky nets. Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War. Herbert wrote before a big, modern, desert war.
So the Butlerian Jihad exists only in a plot-armored world specifically designed to make it work.
Plot armor refers to important characters being invulnerable. What you're describing is just plot (or "internal logic" that underlays the plot). You might think of it as "contrived plot" but that's generally how scifi works.
I don't see how most of the plot holes/unrealism you identify is connected with the Butlerian Jihad at all. But re "critiquing Dune's idea of future warfare" (shields and knives, ornithopters, etc) you might also be entertained by Bret Devereaux's recent take:
https://acoup.blog/2026/02/24/collections-warfare-in-dune-pa...
>Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War
Armor and dart guns exist in dune. Not to mention poison gas. I don't know if riot shields would help much.
They also have rockets and lasers in dune, so those tanks and bulldozers will probably be ambushed like the rest of the spice harvesters, and they require supply lines for diesel. All that equipment also has to be shipped into arrakis, which is expensive, and the harkonnen didn't maintain their equipment very well anyways in a despirate attempt to extract as much wealth as quickly as possible. The lasers are effective on dune where shield use is limited because it attracts sandworms.
Even conventional artillery is used to destroy the cave systems, which play an important role in the book (and in real life if you consider this US-Iran war). Not to mention nuclear weapons exist on the planet and play a key role in diplomacy.
What happened in dune is more like what we saw when the US recently pulled out of afganistan.
Most of the important conflict in dune isn't based on knife fighting, the war revolves around raids, smuggling, suprise attacks, and the destruction and siezure of key pieces of equipment (like spice harvesters, spacecraft, atomics). The knife fighting is only really pivotal in champion combat.
>Marines say that their K-Bar knife is useful for everything but combat.
Okay but if they lived in a world where firearms were often useless against many enemies, and the flow of war was based on operating and siezing equipment, it would make sense for them to only carry a knife and not a gun. A knife is more general-purpose even as you describe. In dune the knives also have cultural value which is what makes the champion combat relevant in the first place.
>Arrakis is important because it grows "spice". Otherwise it would be a useless and planet
If it wasn't for the spice, arrakis could be a habitable, green planet. It's engineered to be a desert to facilitate the production of spice, as explained by kynes.
> like the Empty Quarter of Saudi Arabia
I have seen a critical analysis that sees the origin of Dune as a roman à clef of the rise of the house of Ibn Saud.
Arrakis is Saudi Arabia.
Spice, melange, is oil: the surrounding empire needs it for travel and transport.
House Atreides is the Saud dynasty, which defeated House Harkonnnen.
I find the entire dune series to be boring as hell for this reason. Like watching paint dry. Literally everyone in it is a slave in one way or another. No one lives better than today's modern humans. We're as readers meant to take the extremist "correct" position that of course labor is its own virtue and that therefore automating anything is wrong. Nothing in the dune universe is actually compelling. I don't care what happens to any of the characters at all. It's baffling that this series gets all this attention.
It's not so much slavery but feudalism. It is a return to form. I think there are a lot more people capable of self-actualization and meaning in the dune universe than our own.
>We're as readers meant to take the extremist "correct" position that of course labor is its own virtue and that therefore automating anything is wrong.
I don't think the book forces the reader to take a position on anything. It just portrays a brutal yet intricate medieval-like society.
It is only sci-fi and not fantasy in the sense that some people are concious of their role as both a cause and effect of ecology. (Whereas in our modern society we are only the cause, and in our ancient history we are only an effect).
The reason why everyone's hands are tied is because they are constrained by politics/war/religion/culture/economics/technology and if you tug the thread back the it all goes back to ecology. The key mover in the plot isn't paul but pardot kynes.
Edit:
Shower thought. There is a fifth element in dune that is able to free people from their political/religious/cultural/economic constraints. It's love. People do irrational things out of love. Paul Atredies was a forbidden love child, and his existence is the impetus for the whole conflict and his existence as a messiah destroys pardot kynes' post-mortem plan.
You might have misunderstood. Automation isn’t wrong, what’s wrong is allowing machines to control or direct humans. So a machine that builds cars is fine, and most worlds have plenty of those, but a machine that decides what the car should look like how how many to build would be anathema.
Of course the book mostly takes place on a world with so little infrastructure that most of the goods are made offworld and imported. The native Freemen population makes plenty of stuff out of local resources, but they want to or have to stay hidden from the rest of society and so they don’t have any factories. The stillsuits they wear are hand made, but are exquisitely high–tech and display an ethos of perfectionism in their construction.
Also you might not have picked up that anyone able to afford a small amount of spice in their diet can live for much longer than anyone does today, and even the average person enjoys a wealth of products from hundreds of planets. House Atreides made their fortune harvesting whale fur and selling the clothes and other products made from it. At some point in history they became a Great House by arming themselves with nuclear weapons. There’s a form of personal wealth that doesn’t quite exist today! :)
It’s probably been a while since your read the book, so you have forgotten some things.
For one thing, the guild navigators are prescient and guide their ships across interstellar (and even intergalactic) distances faster than the speed of light. They use their prescience to pick routes where the ship arrives safely instead of plowing through a stray planet. They also use it in a less obvious fashion to maintain the political status quo. They try to avert political crises because they like their position in society and don’t want anyone changing things (which explains the long stasis of the Empire). People did at one point use computers for safe navigation, but they were slower and they weren’t prescient. The possibility that prescient machines might one day be built is the entire reason why Leto II spends so much time breeding humans in “God Emperor of Dune”. He’s looking for anything that can be an edge against the machines, and he hits the jackpot when he discovers that one of his descendants is not prescient, but is invisible to prescients. She and her actions cannot be seen in future he predicts. She and her descendants can safeguard humanity against any form of hostile prescience, machine or alien or otherwise.
Synthesis of spice was a long–term plot element in the series. The Tleilaxu eventually master it. Keep in mind that it isn’t a small–molecule drug either; it’s an alien life form.
Mentats were intended to be a replacement for computers and databases. They were frequently administrators and advisors, and often used their capabilities to convince people. But that wasn’t the goal of their training.
The Bene Gesserit have eidetic access to every memory of every one of their female ancestors. That is actually very impressive; most people don’t even have eidetic access to their _own_ memories, let alone the ability to easily recall secret information known only to their ancestors (such as the password to a safe). The Reverend Mothers can transfer all of their ancestral memories to another, so that the new Reverend Mother has access to the memories of both sets of ancestors. Long chains of Reverend Mothers gifting their memories to each successive Reverend Mother results in the Bene Gesserit having access to the ancestral memories of virtually every race or group of people that ever existed (except the Jewish, who successfully hid themselves even from the Bene Gesserit but otherwise still exist, which is a plot point in one of the later novels).
They can also command people to act. By a trained effort of will they can tell you to get up and dance a jig, or to slit your own throat, and you’d do it. That’s an ability worth respecting, which explains why the Baron Harkonnen took care to put a deaf guard in charge of a Bene Gesserit prisoner.
> So the Butlerian Jihad exists only in a plot-armored world specifically designed to make it work.
Ultimately, however, none of that matters because this statement is what you’re really intending to argue. It’s overly reductive, and it can apply to any work of fiction. Authors naturally build or select a setting where their story makes sense. Frank Herbert (and a few others like Tolkien), went further than that and invented world–building. This excited a lot of people and sold a lot of books, and now you’re cynically dismissing it by calling it “plot armor”.
Ultimately Herbert’s point was that people should strive to be humans, not animals or machines or slaves to people with superpowers. You can’t dismiss this philosophical stance merely because he used a fictional world to illustrate it. It can be true in the real world just as it is in his books, so to refute it you will have to be more clever than that. More human, perhaps he would say.
The good news about this navigation is that the best course for almost any journey is just "straight line", with a minor deviation here and there. Its funny, but due to the emptyiness of space, even in our solar system, if you just pick a random direction, you likely won't hit anything. Trace a line to pluto from earth, and its clear 99.99999...% of the time if its not behind the sun. So the mentats maybe don't have such a tough job :)
Interstellar travel is a bit more difficult than that, and doing it at FTL speeds is presumably more difficult still.
But that’s not really important. Science fiction is allowed to make things up that aren’t perfectly realistic. Campbell surmised that a good science fiction author would pick just one unrealistic thing, such as FTL space travel, for every story, and the reader would oblige by suspending their disbelief. Frank Herbert might have stretched that a bit but I still think he wrote a good story. I don’t hold it against him.
I agree with lostmsu's point here that "folding space" is not as literal as you're interpreting it. "Folding" and "Navigation" is a metaphor some sort of extra-dimensional or non-euclidian operation that enables space travel. What it entails exactly eludes our perception.
I can make up a plausible sci-fi explanation for the need easily: the folded space might not look like our almost euclidean geometry. Maybe in folded space masses take volume proportional to the mass.
Could even envision it in some future film. E.g. when NCC-1701-N goes to warp in addition to the blur effect all the stars swell up.
> Arrakis is important because it grows "spice". Otherwise it would be a useless and planet, like the Empty Quarter of Saudi Arabia.
Any resource heavy planet/country would be irrelevant if it wasn’t for its resources, what’s your point?
> (Or "spice" might be synthesized. In Herbert's day (the 1960s) organic chemistry wasn't up to synthesizing arbitrary small-molecule drugs, but now it is. It's a service you can order on line, even. Biologicals take more work, but that's what genetic engineering is for.)
Bene Tleilax managed to clone of the main characters from his dead cells, and repeat the process for thousand years, they have face dancers who can change their appearance at will. Do you seriously think they wouldn’t synthesize spice if they could? Maybe they could, maybe they couldn’t. Without going into spoilers territory, nobody would permit to disrupt arrakis monopoly on spice.
> The "mental powers" in Dune aren't that impressive. Compare, say, the Harry Potter universe. In Dune, mentats can convince people. Bene Gesserit can look up stuff in their genetic past. Some people can multitask. There are apps for that.
Because it’s a story about human evolution, and not wizards?
> Herbert wanted knife fighters to matter in battle. Not going to happen. Militaries teach knife fighting to build aggression and for desperate situations, but nobody's battle plan depends on a knife fight.
Do militaries also teach how to use a shield that is impenetrable to conventional guns?
> Knife fighters, even with force shields, would be vulnerable to riot control weapons, from tear gas to guns that throw sticky nets.
That’s why most fights in the book and movies require element of surprise and not an outright medieval clash between armies.
> Or to being run over and buried by tanks with bulldozer blades, which the US used in the Gulf War. Herbert wrote before a big, modern, desert war.
Which would be mowed by lasguns?
i’m not sure how to distinguish any difference from modern reality from what you seem to consider plot-armour to be.
what does the Butlerian Jihad “working” mean?
you have it backwards. good world building means everything makes sense. you're basically saying evolution isn't real - it's intelligent design.
You are confusing plot armor with world-building.
Herbert seemed to think that to be human is to be essentially 'open-ended', capable of undiscovered, indeterminate evolution, both personally and as a species.
Indeed.
We have to be careful about letting fiction inform our worldview too much. At the same time, storytelling is one of the most natural and effective ways we have to convey ideas to actually wrestle with.
What are some other fictional universes with interesting takes on AI and humans? I think the Hyperion cantos (at least books 1 and 2) are great. Stross's Accelerando also has some wild ideas but feels maybe less directly relevant.
> We have to be careful about letting fiction inform our worldview too much.
Agreed. But would our AI entrepreneurs, so busy building the Torment Nexus, pay heed?
Whenever I read the original novels, I always imagine the jihad to be a fairly benign if not peaceful exercise more akin to a protest than a war. Like if one day people started rolling across the world smashing data centers and laptops.
I was fairly surprised when his son wrote a novel that felt like a generic, bland robots run amok storyline that happened, more or less, because one guy/titan "ruined a good thing" by telling his AI to run in permissionless mode.
this is an interesting interpretation. i always imagined the jihad as being a violent struggle.
the other jihad in the first novel is that of the Fremen, which is described as being unspeakable violent, so I find it hard to imagine these to be different in that regard.
It's an interpretation if you ignore the impacts of feudalism, slavery, and religious war on Dune. The idea that people were able to express freedom of speech during a time described as the enslavement of 99.99% of the human race seems off-kilter.
the Fremen express freedom of speech despite overwhelming oppression
The desert Fremen of the first book were a tribal society where it doesn’t really make much sense to talk about freedom of speech due to the lack of any sort of formal government. But it seems likely that each naib makes his own decisions about how to run the sietch that he leads. How Stilgar is running Sietch Tabr may not be how every sietch is run.
It doesn’t talk too much about political conditions the city Fremen lived under, but the Harkonnens don’t seem to be too keen on civil liberties so we can probably make an educated guess.
In the second book, the Fremen have established a brutally repressive theocracy that is obsessed with rooting out heresy. Regular people being afraid to openly speak their minds about political issues is a plot point.
I highly recommend Erewhon by Samuel Butler. The author is the namesake for Butlerian Jihad. Interesting book in our times.
https://en.wikipedia.org/wiki/Erewhon
In the book - is it men vs machines? Because it’s pretty clear in our world it will be men+machines vs men(probably also + some machines)
The book starts out thousands of years after it's all over... but later books (prequels) get into into it for us die-hard followers of the series.
The "later books" the "prequels" are not written by Frank Herbert, they're written by his son Brian and SF author Kevin J. Anderson. And not like Christopher Tolkien editing his father's notes, but a completely separate series with nothing in common with the original work.
Yeah, basically.
the immediate cause of the Jihad is the abortion of the healthy fetus of a Bene Gesserit, Jehanne Butler, by an AI."
And which is better explained as two eugenics programs at clashing into each other, both working on dominating the general populace (Bene Gesserit arguably on larger scale than Komos' single-system thing), and Bene Gesserit using their long-worked socio-religious manipulation for revenge that gets out of hand
It's men vs men in charge of machines (and also vs the machines too)
You read the wrong books.
I mean this in both the specific and general sense.
?
Mostly vs machine operator humans, but indirectly against the machines too.
Grand fiction requires conflict, suffering, and war, so banning AI is a necessary plot device.
People keep putting the cart before the horse on this one, and the degree to which modern people have been lulled by fiction is quite worrying. Amusing ourselves to death.
> so banning AI is a necessary plot device.
Absolutely. Walter Jon Williams explicitly made this point in the introduction to the Praxis trilogy:
https://en.wikipedia.org/wiki/Dread_Empire%27s_Fall
He said that to make space opera work, you needed to not have omniscient AI, FTL travel, and nanotech that could work miracles, resurrect the dead, and give lifeforms superpowers.
So, how to forbid it, when someone somewhere will try very very hard to make it?
By imposing a cultural ban. His empire-building aliens forbid it, on pain of genocide.
That gave him a setting. Then, what if that empire falls? That gave him the nugget of a plot.
I found it a very effective setting, myself.
I do wonder where the evolution of the human brain goes, with the widespread use of AI.
The early evidence isn’t good, but I do believe with deliberate and thoughtful training our brains our capable of much more than we currently give them credit for.
Athletes continue to break records despite manual labour being unnecessary for most jobs. And fitness culture has never been more prevalent. We're competitive creatures and this is an ability we are hardwired to appreciate. I see no reason why mental fitness won't see the same development in the future.
What I want to know is what is the "walkable city" of mental fitness? Physical fitness has the athletes, the dedicated exercisers, the unhealthy sedentary masses, and then the people who's day-to-day life integrates enough physical activity that a baseline of fitness is just natural I.e. people living in highly walkable places where both errands and free time incorporates casually moving around.
We should strive for a society where day-to-day life makes a baseline mental fitness a natural outcome as well.
Great question.
I imagine the answer looks something like Wordle writ large: Replacing passive time consumption activities with interactive ones that are as readily accessible (“walkable”).
There’s a fun scene in Star Trek: The Next Generation where LeVar Burton’s chief engineer character is on a long shuttle ride by himself. In addition to sipping a synthehol cocktail, he asks the computer to play a verbal game. It chooses to give him a quiz in his domain of warp field mechanics, and because it’s a smart AI, it calibrates the questions to be just at the limits of his knowledge.
I agree, but it’s equally true access to a broader education where you can actually “strength train” your mental fitness will be needed. We kind of invented a digital telekinesis that lets us summon a vast amount of information on any topic.
We’ll need to build a nose for the right kind of information. Kids growing up in this world already are better at it than older folks who are newly thrown into this environment.
But beyond that I think our agency has already been sapped, as a species. Rebuilding the muscle to exercise our agency as free citizens using the information at our command to make better decisions… we shouldn’t surrender this ability to AI. The best science fiction and science writing already gives us plenty of reason to avoid this. We don’t need to get to the worst of it, and have all out war to correct it.
And that does mean investing heavily in human education. Getting better at teaching things to adults. Making insight easier to gain at every strata of society so true freedom of information when it comes to public works and governance is achieved, which makes the people the watchers and owners of government.
No easy thing to do. Absolutely doable, technologically speaking and in terms of our current biology.
What we have is years of thinking we’re not capable enough. Or cooperative enough. Or rational enough. We can be. We just need the right context, a full belly, and practice in using our voices and shaping our futures actively.
Yes, also see chess which as far as I'm aware is more popular than ever... Despite machines ruling in this cognitive challenge as far back as the 90s.
Athletes, chess, fitness... Have you guys seen regular people? Obesity is worse than ever. Most people are overweight and completely incapable of even basic locomotion without heavy assistance. Most people these days haven't even picked up a book, let alone play chess or otherwise exercise their brains for fun. Sure, a few weirdos might keep exercising their brains, and a few more might find that interesting. A bit like the C obfuscated code contest or something. But most people are going to rot.
> Obesity is worse than ever.
Obesity is actually decreasing today, due to GLP-1 receptor agonists. We are already over peak obesity.
It's just masking the symptoms. I suppose the equivalent for the mind would be everyone on anti-depressant and sedatives. Paints quite the grey picture of our future.
It's a picture that's been painted in a different scifi novel,
Still, I'm optimistic. With obesity, who cares about the underlying cause if you can easily cure the symptoms?I would prefer not to need an expensive drug for the rest of my life.
Yeah, but I think the price will come down over time. There are already purely oral formulations being developed, which will likely be even more effective than first generation injectable GLP-1 agonists, and much cheaper.
> Most people are overweight and completely incapable of even basic locomotion without heavy assistance.
I'm sorry, what? Heavy assistance? The overwhelming majority of people are completely capable of walking on their own two feet. The number of people relying on "heavy assistance" for "basic locomotion" is overwhelmingly tiny, and primarily limited to the elderly
Most people can't walk more than a kilometre or so. They use cars and other motorised transport for the vast majority of travel.
I am a proud Butlerian Jihadi.
Since the botlickers think that those of us who don't trust automatic plagiarism machines are religious nutcases anyway, why not lean into it?
You are not alone, fellow warrior!
:-D
I'm thinking of having T-shirts printed up.
On the front, "Butlerian Jihadi" in some attractively faux-Arabic font.
Perhaps on the back, in Old English Black Letter:
That would actually be a really dope shirt!
The best treatment of AI in the Dune universe I ever found, by far, is by Timothy O'Reilly (yes, that O'Reilly):
https://www.oreilly.com/tim/herbert/ch08.html
But in the last book, Duncan arrives at the conclusion that humans and machines have to merge.
After a thousand years of pause ... they didn't exactly run head first into total embrace in a single generation.
Are we talking about Chapterhouse or a KJA book here?
Regarding the tension between the human and the artificial (machines), there is something to be found in Stanisław Lem’s Cyberiad.
that's just Duncan being Duncan.
so this isn't really about Dune, but there was an article in the economist this week: https://www.economist.com/science-and-technology/2026/10/14/...
It made me think about the book of the new sun. you think its half man half monster creatures are unrealistic? give it a couple thousand years. good sci-fi really is prescient
I really love Wolfe for his small almost throwaway visions of the future, which are used like an aside in a few lines and mostly inconsequential to the story.
In the Urth of the New Sun for example he has a kind of AI lock, the only function of which is to recognize a person and open only to that person. Being in that role for thousands of years, some of the locks develop a kind of personality or at least personal traits, but they are totally defined by and cannot exceed the role they were initially given, nor do they want to. It is still a haunting image for me how personality or something close to a soul might be an accidental byproduct of something as mundane as wanting to have a convenient lock. And of course you cannot stop to wonder how far we are from that.
In Book of the new sun there is the Green Man who Severian saves from imprisonment. He only appears on a few pages but we learn that he is from a distant time where mankind incorporated plant cells in their genetic makeup in order to perform photosynthesis. That brilliantly plays on two themes of the book: the sun as live giving and how a single person can be independent (or "autarch"), here in so far as producing their own energy.
There are many more instances and it’s what makes the Book of the New Sun or the Solar Cycle so fantastic. The future is so vast and strange here, that it cannot be fully depicted and ideas which another writer would stake a whole book or series on are only good for a few lines.
> there was an article in the economist this week
Blocked, for me. Not paywalled: the entire site.
So...
https://archive.ph/4ratb
> This propelled the creators of these machines into a new technocratic class, effectively controlling the worlds of the common people.
The techno-fascists of today. We are already there.
Also, quoting the first Dune novel:
> Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
Yeah, and throughout the 1970s and 1980s, most science fiction depicted the world of the future as dominated by Japan… Look at Robocop 3 :)
That's just what the zeitgeist of the time was.
Computers were then just appearing (or increasing) in the public discussion, along with fears that they'll make humans less human somehow.
Like in this Computer Chronicles video about word processing [0] from 1983 at 4:08: "Some writers are concerned that computer assistance may promote dry bland writing."
Terminator appeared in 1984, and the first Alien movie in 1979 had an antagonist android. 2001 A Space Odyssey with a disagreeable AI came out in 1968. All way before the real world had anything remotely close to today's AI: It was a reaction to the appearance of computers themselves.
That just shows that humans have the same reactions whenever a new world-changing technology appears.
On the other hand, Star Trek had Data.
Ironically, the tech that people and media were mostly optimistic about turned out to be more dystopian than predicted: The internet :')
[0] https://www.youtube.com/watch?v=Jt0OoXluC8g
"Humans have the same reactions whenever a new world-changing technology appears" is one way to see it. The other way to see it is that the current AI boom is part of the same computer revolution that writers in the 70s and 80s were afraid of, and some things happening today really do retroactively justify their fear.
Yeah, it seems absurd to be afraid of computer-assisted writing in the WordStar era, but the Turing test was proposed in the 50s. Everyone knew that human fields of endeavor would only become "more computer" as time went on. Only, if a time traveller could tell a 70s-era sci-fi writer that artists and designers were some of the first to have their livelihoods threatened by 20s-era AI, they'd probably think it was too on the nose to incorporate into their fiction.
Don't forget earlier media like the movie Desk Set in 1957 or the ST:TOS episode "The Ultimate Computer" in 1968 or Colossus: The Forbin Project in 1970.
A related discussion from June,
Dune's Butlerian Jihad and the Future of AI
https://news.ycombinator.com/item?id=48353670
The thing which kills me about _Dune_ (well, one of them) is that for a long while in e-book editions on Amazon Kindle the word "pogrom" was rendered as "program", quite changing the original meaning (this _finally_ seems to be fixed and the 7 instances of pogrom and 8 instances of program are correct), which had a clear and distinctive rooting in history, c.f., Lesley Blanch's _The Sabres of Paradise_:
https://bookblast.org/blog/the-sabres-of-paradise-the-book-t...
EDIT: The formatting error in the Glossary entry "Sandsnork" is still there though.
A fun idea I've been playing around with and testing on folks: We need a neo-temperance movement.
The first reaction is a scoff and a laugh of course - wasn't the original temperance movement an utter failure, prohibition didn't work etc. But no, the movement was far wider and successful in a multitude of ways. And a new temperance movement isn't about blindly copying something old, but situating yourself around the turn of the 20th century, understanding the multitude of interconnected problems people faced, the solutions they came up with and which ones worked and didn't.
America at the turn of the 20th century was a nasty place - there was wild corruption, politics was run by mafia-like patronage networks, wealth was extremely concentrated, rich people were unaccountable, working people suffered under low-paying, dangerous jobs, women were excluded from public life, and alcohol was merely the symptom of a very sick society, and yes indeed it was a massive problem, the way alcoholism is a massive problem in Russia today.
The original temperance movement was bound up tightly with the Suffarage movement (giving women a voice), with Taylorism (running organizations rationally), with income tax (a fairer less corruptible source of income for the federal government than tariffs and excise taxes).
Today's problems are thematically the same - massive concentration of wealth is making an unaccountable elite wielding power against the wishes of the public, creating dangerously addictive digital products, massive increase in gambling, restricting the rights of poeple to own and repair products, working to disenfranchise people, working to roll back women's participation in public life, etc etc. So sex, alcohol and drugs are not an issue today, but gambling, addiction, disenfranchisement, exploitative business practices, mind-boggling amounts of corruption - they're all back in the same form and for the same reasons, and we'll need the same type of movement to fight against them.
What the progressives of the early 20th century did was not abolish capitalism, but tame it. We don't need to go on a jihad against AI, but we do need to yoke it and disempower the current slate of people who own it and direct it.
That is correct. Prohibition was a failure, but it was only one of the aims of the temperance movement.
Another outgrowth: Alcoholics Anonymous. It was a reaction to the end of Prohibition, turning the prohibition internal rather than universal.
> So sex, alcohol and drugs are not an issue today, but gambling, addiction, disenfranchisement, exploitative business practices, mind-boggling amounts of corruption - they're all back in the same form and for the same reasons
what is sex referring to in this context? is this talking about societal attitudes toward sex, sex as in gender, or something else?
"and dance and drink and screw… 'cause there's nothing else to do!"
Or, to bring it closer to home, in the words of the God Emperor of Dune:
> "The favorite poetry? The Epic. The popular dramatic ideal? Heroism. Dances? Wildly abandoned."
> "What did I take from them? The right to participate in history."
…where the Tyrant violently suppresses all of these, because such art is politically dangerous as it reminds people of the agency they've lost.
Excellently written summary.
One of the few sci-fi storylines that doesn't now feel hopelessly dated or anachronistic!
I've said it before, but I always thought the Butlerian Jihad was the biggest plot hole in Dune, because a superintelligent AI would have precluded a lot of the dynamics in the stories, such as the need for a Space Guild, yet the background is never explained.
However, I deeply appreciate the rich world and captivating narrative it enabled, and I think this is what Frank Herbert was going for. He wanted the intricate dynamics of the Guild and Spice and Mentats and exciting close-quarter combat for a more intriguing narrative. But AI and robots would have obviated all of those, so he made it disappear with a handwave of "because Butlerian Jihad."
My friend screams this every time I mention AI. I’m convinced.
The Butlerian Jihad is a reference to samuel butler's 1872 proto-scifi satircal novel: https://en.wikipedia.org/wiki/Erewhon which talks about a society that had a machine intelligence uprising and so banned all complex machines.
This is literally spelled out in the page you are commenting upon. Didn't you read it first?
I did not. Erewhon is just one of my favorite books so when I see a thread about Dune I tend to talk about it. Normally I'd read the link but this was a generic "Fandom" wiki page and unlikely to be very useful to read.
Hint: do so in future.
This wasn't a normal 'article'. It was just a commercial wiki page on a well known topic. And a distasteful 'fandom' one at that. No information there was special or unique to it in particular. The real point was to have URL about the topic to start the conversation.
Iain M Banks sci fi also has stories where AIs must be wiped out.
The death scene of the AI and its assassin/partner in death in Look to Windward is one of the best written things in his books.
God I wish he had written 1000 more novels. Every single one of his stories is phenomenal, even the non culture stuff. The Wasp Factory was the first of his novels I read. Imagine my surprise and elation in finding that most of his other works were absolute top tier sci-fi.
To be more specific, the machine Minds rule the Culture civilisation and the humanoids are effectively mere pampered pets.
(Several characters in the novels comment on this, so it is entirely cannon.)
The Culture fought a war against the Idirans, who eschew the use of intelligent machines. The Idirans lost.
Similarly, the Gzilt civilisation in the novel The Hydrogen Sonata prefer to run their ships with a quorum of uploaded humanoid mind states running at highly accelerated speeds.
If I take my parents on trips, buy them nice things, visit often and generally make sure they are happy and fulfilled, are they my pampered pets?
Your parents have agency that your gifts do not dilute.
Humanoids in the Culture are controlled at every level, even their thoughts as guided… sorry… “aligned” with the Minds through the adoption of the language Marain.
No humanoid ever overrules a ship Mind in the novels.
Sure, you can just ignore the source text and the author explicit statements that it's a true utopia[1].
They're an anarchistic society, no one has power to overrule any one else.
https://www.reddit.com/r/TheCulture/comments/kbtylt/ysk_the_...
This happens for lots of stuff. Some time back there's a thread where I mentioned San Junipero, the episode of Black Mirror with a happy ending - and I said some people will insist it doesn't have a happy ending and sure enough there are replies insisting that's not a happy ending. The author's opinion does not matter to them.
Just as shippers can successfully take two characters who clearly have no basis for a romantic or sexual relationship and pair them off in their head canon, likewise people can disregard the author's statements about their work and decide it's completely contrary to how they've decided to understand it.
Some people look at https://en.wikipedia.org/wiki/Fountain_(Duchamp) and they do not see a sculpture.
Oh, i'm not saying always believe what the author say, far for from it. I personally read authors that said they write one thing when in fact they did not. That's not the case with Iain M Banks, his published Culture books support his declarations made elsewhere.
It might be Banks utopia but the Culture seems more like Calhoun's rat utopia experiment to me or Douglas Adams' "long dark teatime of the soul". There's nothing to strive for, there's nothing one could do that hasn't been done better by a quadrillion other beings and utterly eclipsed by the Minds. It's so spiritually sterile that a ships crew collectively gives themselves a cold (normally prevented by their medical tech) just for the sake of the experience and breaking the monotony of their existence.
ah, you now understand: we live in an meaningless universe. Your mistake is thinking struggle, or suffering, gives it meaning.
It doesn't.
Edgy but I would agree that struggle doesn't give existence meaning in any cosmic, philosophical, or mathematical sense. However, the need for it seems to be something hardwired into living beings, without which they wither away. Struggle and suffering might be considered a necessary, if unpleasant, nutrient of sorts.
Nothing edgy about existentialism, which the Culture always forces us to confront up close.
>struggle [...] hardwired into living beings, without which they wither away
The reason Sisyphus is happy (or so we imagine) is not because he's struggling for all eternity
You’re getting down-voted when this premise is the core theme of The Hydrogen Sonata!
Both Vyr Cossont and Ngaroe QiRia are so bored with their meaningless existence that they have to go out of their way to invent “life tasks” and other challenges to immerse themselves in.
Not to mention the scene where a ship mind casually crushes the spirits of Vyr by perfectly playing a piece on its first attempt that she struggled with for years!
> Both Vyr Cossont and Ngaroe QiRia are so bored with their meaningless existence that they have to go out of their way to invent “life tasks” and other challenges to immerse themselves in.
While there are some types of people that would be bored and would struggle in Culture-style utopia, I think for overwhelming majority of people it would still be a massive improvement over today's world.
> Humanoids in the Culture are controlled at every level
Nope, they have considerable freedom, arguably greater than we have today. It's true that they cannot directly challenge the almost onmipotent Minds when it comes to stuff that matters to them, but Minds in Culture are an example of a very well-aligned AI, so it's almost always the case that going against a Mind's intent will just make the situation worse for everyone.
I assumed they were referencing Algebraist.
This will seem like prophecy and not fiction someday.
Maybe, but Frank Herbert's Dune books don't really explain it. The Butlerians simply exist to justify Mentats, and then get tossed aside. There are lots of similar plot points that have an "oh cool" moment but never see proper exploration.
Dune's central theme about balancing power and authority is much more prescient for our time, but I suppose we'll have to wait for Villeneuve's Part 3 before Silicon Valley discusses it.
I feel like the book explains it plenty, it's just extremely succinct.
> “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
What else needs to be said? We see examples of this throughout human history all the time. People thought at the early onset of the industrialization that people would become freer by leaving feudalism but all it did was immiserate workers for another 130 years (immiseration still ongoing in certain areas of the world).
It's basically rentierism taken to the extremes.
The Bene Gesserit and the Spacing Guild were also a result of the Butlerian Jihad. All three started as schools of human development in the absence of powerful technology.
Also, Herbert's point about dependence on machines creating machine-dependent humans, trapped in a deterministic arc of evolution, feeds into his larger themes about open-ended evolution and its arms-race. It's not just a throwaway backstory.
Exactly.
And Herbert explores this larger theme even beyond his Dune series:
"Surprise me, Holy Void!" (says the Ship once freed)
There is actually an active "BJ" group in the Phoenix Metro area. It was about a year ago when I noticed they were slapping up custom graffiti stickers on Central Avenue (Midtown).
Then later, I saw their handiwork in person: they would take an angle-grinder and obliterate the QR Code from those rental e-Scooters, slap on a BJ sticker or three, and then leave them where they found them. This would render the scooters as e-waste, un-rentable and blocking the sidewalks.
I would not be surprised if they are quietly influencing groups which are agitating against AI Data Centers. But the BJ I have seen is a chaotic group of direct urban sabotage.
Maybe, we get Leo's Crusade instead.
Nonetheless, I was always Team Venport
If you look at the publication date, and read the original dune novel, it's pretty obvious that Herbert throws in a selection of somewhat randomly chosen Arabic and Farsi language origin words to add flavor to the text.
The "jihad" part wouldn't work so well in the movie, so of course they had to go with "holy war". You can't exactly have Timothee Chalemet waving a crysknife around and yelling "JIHAD!!!" in the post-9/11, post-GWOT era.
>You can't exactly have Timothee Chalemet waving a crysknife around and yelling "JIHAD!!!"
honestly I wish he did. The films are already incredibly on the nose, much more so than the books, that Paul is not to be understood as a hero. Practically half his dialog is directly talking to the audience telling us how he's going to kill billions of people. Chani got a very different role, and we start and end the movies from her perspective to stress it even further.
If you're already doing this you might as well have not chickened out on the language, it was one of the few things that genuinely irked me about the film.
What's odd is they left in the Mahdi part. Try watching it with a person who grew up in a religious twelver Shia environment and see how they react to that...
Frank never wrote about AI. Brian invented it whole cloth, enabling the grift of his father’s works.
Interesting to see this on HN! I've been beating the drum for quite a while that the Butlerian Jihad was never about terminator-style robot uprisings, but rather the intentionally-unexamined half of the themes the Dune ~trilogy ended up being about: long term fate of the species, the prospect of human speciation, the role of cold reason (read: aggressive utilitarianism) in leadership, and above all the concentration of power in light of modern technology.
All he mentioned explicitly was the very last clause, but the others are more important & uniquely-Dunean IMHO, and are central to the plot from the jump through the Bene Gesserit, the Mentats, and the Guild Navigators, all ostensibly Homo sapiens sapiens despite being completely unnatural. Even "Imperial Conditioning" --which obviously plays the biggest part in the first act of the first book-- is arguably the same topic. By refusing to include computers and robots at all, the reader is naturally guided to consider those more familiar tropes in light of the world-building snippets we get. Which is kinda what makes Dune so genius, I think!
Like, to make it explicit... I think having the aforementioned abhumans drive the plot without significant support from inhuman minds raises more subtle questions than we're used to in far-future tales of robots, like:
1. Can one lose their humanity through experience, be it intentional "conditioning" or just the acquisition of power? (see: Paul vis-a-vis Eichmann in Jerusalem)
2. Once one has lost their humanity superficially, can it be brought back/up/forward? Is love/greif the most plausible path, and thus the most human cognitive processes? (see: Dr. Yueh vis-a-vis Eichmann in Jerusalem)
3. If we try to maintain our humanity in the long term --the default goal of non-insane futurists, though debated in the 21st c.[1][2]-- how do we protect against the natural urge to pursue power, which is (arguably) inherently human?[3]
I'll cut it off there cause this threat will likely be deleted anyway and I'm heavily procrastinating. Long story short: the Butlerian Jihad is not a condemnation of artificial minds, and if even it were, the condemnation would apply equally as much if not more so to the humanist empire in which the story is set.
Also, hopefully obvious, but just in case: it likely began as an excuse to write the stories that he wanted to write, just like the shield thing giving an excuse for swords in space. But I think Herbert deserves credit for what was running underneath that and elaborated on as it became a series of novels rather than cute periodical stories. He was pretty smart!
Also also: we do not discuss the son's book about Butler lol. It is verboten.
[1]: https://en.wikipedia.org/wiki/He_Jiankui_affair
[2]: https://en.wikipedia.org/wiki/How_We_Became_Posthuman
[3]: This kind of structural conflict could be described as 'internal contradiction' https://en.wikipedia.org/wiki/Dialectical_materialism or 'endogenous contradiction' https://en.wikipedia.org/wiki/Endogeneity_(econometrics) depending on your preferred flavor of economics.
Two subtleties in Frank Herbert's original formulation that are often missed:
The first has been covered: "Once men turned their thinking over to machine ... that only permitted other men with machines to enslave them." Men did the enslaving. It describes a techbro oligarch takeover, not an AI takeover. The machines do not have agency.
Hebert was really into the whole idea now expressed as "Hard times create strong men. Strong men create good times. Good times create weak men. And, weak men create hard times." i.e. good times, people get lazy, then get weak, then get enslaved. (1)
And the second is that "Thou shalt not make a machine in the likeness of a human mind" is not the same thing as "Though shalt not make a machine mind". A drawing or statue of a person is "in the likeness" of that person. It's not a person. It doesn't have to actually be the thing in order to qualify as "likeness" of it.
People who scorn LLMs would agree that they produce something like the output of a human, in the sense that a parody, a caricature, a cartoon is a likeness. An uncanny valley mockery of the thing is "in the likeness" of a thing.
Ultimately, Herbert put this in his fiction because he didn't want to write about robots or machines, just humans and human potential (2). So he wrote it out with a few paragraphs of backstory in the novels' distant past. You can over-interpret it. Herbert wanted to sound deep, but he did not always succeed. And even when he succeeded in that, his views were not always the best.
1) https://acoup.blog/2020/01/17/collections-the-fremen-mirage-...
2) https://en.wikipedia.org/wiki/Human_potential_movement
It was a clever device to have a universe where high fantasy tropes can be freely mixed with advanced nigh-magical space flight and other sci-fi tropes, and is one of the things that makes Dune badass. (Heard it described once as "the Foundation trilogy for people who like metal.")
But history has already shown us that manipulation of belief and religion is a more effective and cheaper means of control than technocratic methods. A giant technocratic surveillance dystopia is possible, but it costs a lot more than old fashioned dogmatism with enforcement through herd behavior.
So we smashed religion, a kind of anti-Butlerian Jihad we call the Enlightenment, but the human lust for power and control just followed us into the machine age. If we smash the machines, we'll just end up back where we were. It will follow us there too.
Of course you could say this was one of the sub-themes of Dune, since (reading comprehension test!) Paul is not a hero. He's a religious manipulator, though a bit of a tragic villain in the sense that he feels dragged into it by fate.
IRL I don't think humanity will ever escape this cycle until we can define what freedom even means and reduce it to first principles. Does having thinking machines do my thinking for me increase or decrease my freedom? Does having a religious orthodoxy make my life choices for me increase or decrease my freedom? What's the third alternative, and do we really need to smash either religion or technology to get it?
The actual linked wikia article is short on the actual detail and background of exactly why the Butlarian Jihad actually happened. I’m an avid Dune fan. Sidebar/background: The son of the Dune writer, Brian Herbert, and his cowriter Kevin J. Anderson wrote the setting, events, and conclusion to the actual Butlarian Jihad conflictt. It’s largely derided by scifi fans, but is highly entertaining. It’s actually still ongoing almost 3 decades later. And frankly not as deserving of the harsh criticism it got when it came out. But I digress…
Here’s how it went down in the dune universe (I’m certainly messing up some details, but these are the broad strokes):
Humanity has spread to the stars. No FTL involved, just good old fashioned sublight engines. The dune universe takes place about 50-60,000 years in our future. Mankind has spread across a couple dozen solar systems and everything is going peachy. However, mankind gets complacent, fat, and happy with their success and society stagnates. A handful of eager, ambitious humans scheme about how they can take it all over. They do, and then eventually start turning themselves into cyborgs, or disembodies brains in a jars that swap bodies and are effectively immortal. One of the cyborgs gets lazy and makes an AI to manage things, which promptly takes over, overthrows the cyborg rulers of humanity, makes the cyborgs it’s slaves (who have great autonomy/are conspirators) and generally rules over everything. A small group of solar systems is left under human control, who fall back to a feudal-ish type system of government with lords and dukes and whatnot.
The AI does all sorts of nasty shit to the humans in it’s dominion. Unit 931 and Auschwitz type experiences, slavery until death, etc. Untold billions, if not trillions, of humans live in animal-like conditons worked to death. The human group of systems isn’t cool with this, but they can’t do much and are vastly overpowered by the cyborgs, so a fragile peace is held. One day, the son of one of the subservient cyborgs knocks up a human slave, and they have a kid. In the background, all sorts of rabble rousing and shenanigans are being done to stir unrest amoungst the slaves by the free human system. The human systems develop FTL, with spice as a catalyst, except it doesn’t work 10% of the time and kills the pilots. The AI’s main henchman (who does all of the nazi and Unit 931-type experiments) decides it would be cool to kill the baby of the cyborg son and the slave. The slaves last name was “Butler”/ This sparks a massive revolt by the slave worlds. The free human systems then go on to fight a prolonged war against the machines, using the FTL drive at great loss of life, to begin a “Butlarian jihad” to free the humans from thinking machines. Lots of other stuff happens, and humanity eventually wins, but this is why machines no more advanced than a desktop machine are allowed in the dune universe, and why they technologically hobble themselves.
>And frankly not as deserving of the harsh criticism it got when it came out.
Hard disagreement.
My issue is not with the work on its own merits (although I think its pretty generic mass-market SF), but rather that it is wearing the skin of something it is not and to which it has no honest claim. It is one thing for an author to take a well-regarded prior work and attempt to complete or expand it. For example, Brandon Sanderson's work completing Robert Jordan's epic Wheel of Time is very respectable; he tried to complete the series as Jordan would have done. It is quite another to take the name and setting of such a work, steal the surface detail and use it to dress up one's own story that shares nothing of the underlying body or soul of the work - such enterprises are an insult to the original author, the reader, and the self-respect of the new author.
> The [slave's] last name was "Butler"
Interesting backstory about this name. Of course, it is an "occupational surname" and means "bottler" or "cup-bearer". The Butler Dynasty was a notable and very influential noble family in Ireland, beginning a century after the Norman Invasion of England.
https://en.wikipedia.org/wiki/Butler_dynasty
Now Franklin Patrick Herbert, being of Irish and German descent, and naturally a student of history, would've been quite aware of the power and influence of the medieval Butlers in history. And some of this renown was brought to American shores, as well.
In learning of my birth family and tracing our origins back to the birth of America, I discovered that my paternal line had come from Butler County, PA. They arrived in the late 18th century and homesteaded large swaths of this region near Pittsburgh. They had prospered, especially from mineral rights on the land, and the Irish Catholic immigrants had intermarried with German Catholics who homesteaded nearby. (Often, entire counties in the area were pioneered by immigrants from a certain part of Europe. And the intermarriage was a natural result of sharing a common faith.)
So for several years before 2024, I was known as the guy in genealogical circles who was hunting down information about Butler County, and I was fairly proud of finding my family roots in that county, and I mentioned it all the time in the Irish pub and everywhere else. Well, imagine my shock when the little incident happened on July 13th, 2024. I have been more reticent to discuss Butler County since that time.
Anyway, I find it interesting that Herbert chose that particular name for the "Anti-AI crusade". And that is partly why I chose the moniker for my own, despite not being particularly anti-AI (remember, all my comments are generated by a neural net), nor having read any of the books, or seen the films. (I did attempt to watch the Lynch film, but it really did not hold my interest! So sue me!)
More interesting backstory into the name: it's actually a reference to https://en.wikipedia.org/wiki/Samuel_Butler_(novelist) and his 1872 proto-scifi satircal novel: https://en.wikipedia.org/wiki/Erewhon which talks about a society that had a machine intelligence uprising and so banned all complex machines.
I was born in Butler!
[dead]
[dead]
This gets mentioned on HN at least 5 times a day.
Highly relevant then. HN tends to discuss the things that are either meaningful today or what they are building in their garage that is often meaningful in 5-10 years.