The Fight Against Superintelligence: Who’s Really in Control?

In Campbell, California, an unlikely coalition of AI scientists, policymakers, faith leaders, actors, and more have come together. They’re putting their foot down on the race towards superintelligence, declaring it must be stopped until we can guarantee it's safe and controllable. The initiative, backed by big names like Geoffrey Hinton and Steve Wozniak, demands public buy-in and scientific consensus before unleashing such potentially uncontrollable power.

What's the Real Issue?

Superintelligence isn’t just a technical challenge, it’s a societal one. We’re talking about artificial intelligence that could outperform humans in pretty much everything. The stakes? Huge. We're promised solutions to global issues but also warned about risks like economic displacement and even human extinction. It’s not just a tech problem, it’s a story about power and who holds it.

Yoshua Bengio, a top AI scientist, emphasizes the need for AI systems that can’t harm us, either accidentally or intentionally. But ask yourself: whose data? Whose labor? Whose benefit? AI development without public input feels like tech companies are playing God, but who benefits?

Public Sentiment: A Wake-Up Call

The Future of Life Institute released a poll: 95% of Americans don’t want unregulated AI’s current trajectory. A whopping 73% advocate for strong regulation. Yet, big tech seems deaf to these calls. Anthony Aguirre, co-founder of the institute, says, “Nobody developing these AI systems has been asking humanity if this is OK.” And he’s right.

The benchmark doesn’t capture what matters most. We need to prioritize public consent over unchecked technological progress. Are we really ready to trade off societal controls for faster algorithms?

Who’s Really in Charge?

Signatories like Prince Harry and Joseph Gordon-Levitt argue for AI that serves humanity, not replaces it. But the real question remains: who’s steering this ship? The paper buries the most important finding in the appendix, focusing on big names rather than the fact that tech giants are racing forward without a safety net.

It’s time to ask harder questions. Why are we pushing toward a future we can’t control? Regulators, take note. The call isn’t for slowing down innovation but for innovation that’s controllable and accountable. This is more than just a tech issue. it’s about who gets to decide the future we all have to live in.