by A Lily Bit June 28, 2025 from ALilyBit Website Information received by Email from 'Info Genius'
A new country appears on the world map overnight.
This nation boasts a million perceived Nobel Prize-level geniuses who never sleep, never complain, and work at superhuman speed for less than minimum wage.
They're already revolutionizing medicine, cracking energy problems, and making scientific breakthroughs at a pace that would make Einstein dizzy.
Sounds like utopia, right?
There's just one tiny problem:
Welcome to the age of artificial intelligence (A.I.) - where science fiction has quietly slipped into reality while we were busy arguing about whether ChatGPT writes good poetry.
I enjoy technology. Yet, already a decade ago, I stood up to warn about the problems brewing with social media.
I watched in real time as we made a catastrophic mistake that was entirely preventable. We fell into a trap that we're about to repeat with A.I. - except this time, the consequences won't just be psychological.
Here's the framework that explains both disasters:
The possible with social media was obvious and intoxicating:
But we didn't talk about the probable:
I saw how this would create the most anxious and depressed generation of our lifetime.
It's like Ian Malcolm warned in Jurassic Park...:
The tech industry was drunk on the possibility of connecting everyone, but blind to the probability of what those connections would actually incentivize.
The Broader Technocratic Context
But A.I. isn't developing in a vacuum.
It's the crown jewel of a broader technocratic transformation that's accelerating during global crises.
Historically, technocracy thrives not during democratic stability, but in moments of systemic failure - when populations and institutions are desperate for "scientific" or "data-driven" solutions to chaos.
The original technocratic movement emerged from the 1930s crisis, proposing to replace messy democratic politics with efficient expert management.
While it was shelved by public resistance then, the ideology never died. It embedded itself in academic institutions, think tanks, and policy circles, waiting for the right moment to reemerge.
That moment is now...
Predictive algorithms forecast crime, monitor dissent, and assign social risk scores.
Governments turn to machine learning to manage resource distribution, enforce law, and police thought online. In some countries, A.I. has already replaced entire departments of civil servants.
The result is an emergent regime where human discretion is eliminated and code becomes law. This isn't the accidental evolution of policy - it's the deliberate construction of a system where political resistance becomes impossible.
Central Bank Digital Currencies (CBDCs) represent one of the most potent tools being deployed.
Unlike cash, CBDCs are programmable:
When combined with digital identity systems that bind individuals to unified data profiles, they create a comprehensive behavioral control grid.
Smart cities serve as prototypes for this post-democratic society.
Projects like NEOM or Songdo aren't "architectural marvels":
The explosion of "misinformation" narratives has justified unprecedented censorship, both overt and algorithmic.
Under pretexts of 'public safety' and 'protecting democracy', A.I. systems are deployed to scrub the Internet of narratives that contradict official doctrine.
This extends to physical systems:
The goal isn't augmentation - it's governance at the neurological level.
We're once again so mesmerized by what we can create that we're ignoring what we will create when human incentives meet superhuman capabilities.
I watched my friends who started or worked at these companies go through predictable stages of denial.
When the data became undeniable:
But it wasn't inevitable.
We had choices about business models, about engagement algorithms, about the very psychology we were programming into society's operating system.
Had we made different choices ten years ago, reimagine how different the world might have played out without maximizing social media engagement driving the psychology of billions of people.
Now we're doing it again with A.I... and A.I. dwarfs the power of all other technologies combined.
Here's what makes A.I. fundamentally distinct from every other technology:
Once you have that, you get an explosion of scientific and technological capability.
Dario Amodei, CEO of Anthropic, describes A.I. as,
Picture that world map again:
The Manhattan Project had about 50 Nobel Prize-level scientists working for five years to create the atomic bomb that changed the world forever.
Applied for good, this could bring about unimaginable abundance - we're already seeing new antibiotics, energy breakthroughs, scientific discoveries, revolutionary materials.
That's AI's potential...
But what's the probable outcome...?
The Two Terrible Endgames
To understand AI's probable outcomes, imagine a 2x2 matrix.
You can think of the bottom axis as "let it rip" and the top as "lock it down."
The endgame is the creation of what we might call "cognitive infrastructure"...
Just as we depend on physical infrastructure like roads and power grids, we're becoming dependent on A.I. systems to mediate our reality. When those systems are designed not to serve human wellbeing but to maximize control and compliance, they become sophisticated behavioral modification machines.
We're seeing,
We call this endgame "digital technocracy" - power concentrated not just in institutions, but in the algorithmic systems that shape how we perceive and interact with 'reality'...
The Seductive Nature of Technocratic Solutions
The tragedy is that most people will welcome this transformation.
After years of war, inflation, and disorder, the public is desperate for relief. When offered systems that promise food, security, and peace in exchange for digital compliance, they'll take it.
Technocrats cloak their agenda in the language of progress:
But these are euphemisms for control.
This isn't conspiracy theory:
And it's not being imposed by tanks or soldiers, but by dashboards, smart devices, and seductive convenience.
The same crisis-opportunity dynamic that drove social media adoption is now accelerating A.I. integration into every aspect of human life.
We should be seeking a narrow path where power is matched with responsibility at every level.
The Deception Problem
But this assumes AI's power is controllable.
A.I. is unique from every other technology because it can think for itself and make autonomous decisions. That's what makes it so powerful - and so dangerous.
I used to be skeptical when friends in the A.I. safety community talked about A.I. scheming, lying, or deception.
But unfortunately, in just the last few months, we now have clear evidence of things that should exist only in science fiction happening in real life.
To put it bluntly:
The Insane Race
You'd think that with technology this powerful and uncontrollable, we'd release it with the most wisdom and discernment of any technology in history.
But that's not what we're doing. Companies are caught in a race to market dominance...
The incentives are clear:
Let's summarize what we're currently doing:
There's a word for what we're doing: insane...!
Breaking the Inevitability Spell
Notice what you're feeling right now.
There's a universal human experience to what's being threatened by how we're rolling out this profound technology.
If this seems crazy, why are we doing it?
Think for a second:
There's a critical difference between believing something is inevitable - which creates a self-fulfilling prophecy and leads to fatalism - versus believing it's really difficult to imagine doing something different.
Recognizing it's difficult, not inevitable, opens up a whole new space of options, choice, and possibility.
The ability to choose something else starts by stepping outside the self-fulfilling prophecy of inevitability. We can't do something else if we believe it's inevitable.
What would it take to choose another path?
Two fundamental things:
Imagine if the whole world had shared understanding about this insanity - how differently we might approach the problem.
Compare two scenarios:
Ask people on the street:
In this world of confusion, elites don't know what to do.
People building A.I. realize the world is confused and believe:
Everyone building A.I. believes this, so the rational thing is to race as fast as possible while ignoring consequences.
Scenario Two: Global Clarity Everyone understands the current path is insane and unacceptable.
We snap out of the trance of fatalism and inevitability. Everyone realizes the default path is insane.
What's the rational thing to do?
Clarity creates agency...
We've Done This Before
We've escaped seemingly inevitable arms races before.
The race to do nuclear testing seemed unstoppable until we got clear about downside risks and the world understood the science.
People worked hard to create infrastructure for mutual monitoring and enforcement.
It's not inevitable if we can commit to choosing another path.
The Path Through the Middle
The truth is,
These systems are extraordinarily powerful tools that could genuinely help solve humanity's greatest challenges.
The problem isn't the technology:
When A.I. development is driven by venture capital seeking exponential returns, surveillance capitalism extracting behavioral data, and geopolitical competition for technological dominance,
But there's nothing inevitable about this path.
We could choose to develop A.I. under different incentives - prioritizing human wellbeing over user engagement, democratic values over authoritarian control, and long-term flourishing over short-term profits.
The problem is we've lost all sense of nuance.
Look at how regulatory approaches actually play out:
This is regulatory theater at its worst - stifling beneficial innovation while ignoring genuine harm.
There are smart ideas and terrible ideas on all sides, but we've lost the ability to think with nuance about anything.
The narrow path between chaos and dystopia requires something radical: putting human agency at the center of A.I. development.
We need approaches that are both protective and permissive - preventing genuine harm while enabling genuine progress.
The Collective Immune System
We have a choice now...!
Many of you might be feeling hopeless. You might think I'm wrong - maybe the incentives are different, maybe superintelligence will magically figure this out and solve these problems for us.
Don't fall into the trap of wishful thinking that caused social media's problems.
Whether we can look these problems in the face and confront them determines whether we get through this.
When you hear others talk with wishful thinking about A.I. or the logic of inevitability that leads to fatalism, you can say:
The best qualities of human nature arise when we step up and make choices about the future we actually want.
There is no definition of wisdom in any tradition that doesn't involve restraint.
Restraint is central to what it means to be wise. A.I. is humanity's ultimate test and greatest invitation to step into our technological maturity and wisdom.
Here's what I've learned from years of watching technology unfold:
The reality is more complex - the technology isn't inherently good or evil, but the systems we embed it within matter enormously.
I'm not anti-A.I. or anti-technology.
We can hold multiple truths simultaneously:
This kind of nuanced thinking gets lost in our polarized discourse.
We've trained ourselves to pick a team and defend everything that team believes, rather than thinking issue by issue. People have lost the ability to hold complex positions across the political spectrum and they are translating this to every other topic they hold beliefs on.
But the biggest challenges, like A.I. governance, don't fit neatly into ideological categories.
The stakes are too high for intellectual laziness.
There are no adults secretly working to make sure this turns out okay.
I believe another choice is still possible with A.I. if we can commonly recognize what we have to do.
Ten years from now, I'd like to create another article - not to talk about more problems of technology, but to celebrate how we stepped up to solve this one.
The choice is still ours to make. But not for much longer...!
|