This is Part 3 of a 5 Part series. You can find Part 1 here.
Roaring Falls
There are various claims about the hydromorphology (the shape, size, structure, flow, etc) of the Stream one could make.1 A very important claim that the Growth Theory posits is the absence of Falls.
Rivers are mostly flat(ish), they may have Rapids, but sometimes they have (Water-) Falls. The claim “there are no Falls” is a very strong claim about the hydromorphology of the Stream. And guess what?
It’s wrong.
A Fall is a plunge into the Abyss, an irrecoverable catastrophe. If you don’t spot it well ahead of time and get out before the rushing water pushes you off the precipice, it’s Game Over.
The existence of Falls greatly changes the character of the Stream.
If the Stream is monotonously pushing you forward towards the Ocean, and there are no Falls to worry about, then you should just always swim towards the fastest Currents you can find, so you can hopefully be propelled faster towards your goal. “Steering” isn’t really an important consideration, only insofar as you can use it to steer yourself into even faster flowing Water. But with this strategy, you are not really steering where you are going. The Current is.
If you believe the Stream has no Falls, you have no problem letting its turbulent Waters steer you. If there are Falls…well, you can imagine where the Currents are the strongest…
This mistake is not completely novel to the Moderns. The Ancients also had variants of this, even the Buddhists had their version of this belief, namely in the form of reincarnation.
The claim of reincarnation is basically that you have “infinite retries”. You can get into really, really bad Waters2, sure, but you can, in principle, always make it back out, eventually. And so can the rest of the world. Even if you, personally, don’t make it this time around, this doesn’t necessarily stop someone else from doing so.3
The humanists of Les Lumières also had a version of this. They believed in progress over generations, leaving a better world for your children, bit by bit. A very noble and good sentiment, but it doesn’t account for things that can not only kill you, the Human, but us all, Humanity.
There are Falls out there in the dark waters. Falls so tremendous and thundering they will take all of humanity howling into the abyss if we let ourselves be dragged towards them.
Population decline, widespread ecosystem collapse, bioweapons, nuclear war and, of course, artificial superintelligence are but a few of the Falls we are facing. The faster we let ourselves be dragged downstream, the more Falls we are liable to be dragged towards.
And if we are dragged off a Roaring Fall…well…that’s it. Story over. Humanity is no more. Thanks for playing.
And the Thunder is growing louder. We didn’t use to have to worry about nuclear weapons, or artificial intelligence. These are completely novel, terrifying threats, effectively eldritch horrors from the perspective of the Ancients. The further downstream we swim, the more turbulent the waters, the more insidious the riptides, the more apocalyptic the Falls.
The tragedy of the Falls is that we create the very Current that drags us toward them. Every human working to build AGI, every couple choosing not to have children, every civilization optimizing for growth at any cost…We are the Water, and collectively we’re flowing toward the Thunder.
And worse than even the Falls are the Sirens that sing beside them. The same optimization pressure that created intelligence creates adversaries, misaligned AGI, Molochian races to the bottom, egregores and ideologies that eat their creators. The corporations building AGI know the risks. They build anyways. The Current has driven them Mad.
This is the true horror: It’s not just that we might fall. It’s that the Stream itself whispers to us, ‘Jump.’
Is this really Progress?
A Wild River to Take You Home
Where the Ancients, and the Moderns, were wrong is thinking of the Stream as a Gentle Stream, instead of a Wild River. The Groove left in the fabric of reality by the General Intelligence Fixpoint is filling rapidly with storm water, and it has created a gnashing, violent, turbulent flow plunging down many Falls into abysses so dark, so hellish, so eldritch, the Ancients couldn’t even conceive of them.
And yet, in the distance: Light. Home.
The Wild River will likely lead you down a great Roaring Fall, but it’s also the only thing that can Take You Home.
What the Ancients were missing is the actual, true concept of tangible, real, scientific, economic, technical, political Progress. And this is real! The industrial revolution is real, modern science4 is real, antibiotics are real.
If you deny there has been Progress, that we have the Tools to build a Better World than we ever could in the past, then you are Mad.
And if you think the path towards the Ocean is Gentle and requires no Steering, you are also Mad. You have become a barely sentient Puppet for eldritch forces and Currents playing games with humanity’s future and Soul.
Giving yourself to the dark, pelagic gods of the Currents is giving up your Humanity, and forsaking Humanity’s Birthright.
Navigate the Currents, or become part of the Darkness threatening to swallow the Light.5
And symmetrically, giving up on the Stream, on the Journey, wanting us to sit still on Dry Land, is hardly better. You are just dooming Humanity to slowly wither away in the Badlands, and to be picked off by whatever Predators are stalking us.
We cannot stay here.
We cannot get to the Ocean by foot. We cannot get there by letting the Currents take us. We cannot get there by staying here.
We don’t know where the Ocean is, but we know the Wild River can take us there. But for this, we have to actually Navigate.
This is Part 3 of a 5 Part series. You can find Part 4 here.
And inevitably someone does make.
Such as the Hell Realms.
This is a strawman of actual buddhism, of course, they are a lot more nuanced about a lot of these things, but let's skip over that for the sake of brevity.
all its insufficiencies and flaws notwithstanding
In case the metaphors are somehow not on the nose enough: If you work at a company building AGI, you are part of the Problem. Doesn’t matter what your role is. You are a Bad Person and should immediately stop what you are doing and go do something else.
We're gonna need a bigger boat
uber good