Item — HN Reader

Everyone discussing some AGI superbeing a la Skynet is falling for the hype pushed hard by AI companies hook, line and sinker.

These things are dangerous not because of some sci-fi event that might or might not happen X years from now, they're dangerous now for perfectly predictable reasons stemming primarily from executive and VC greed. They won't have to be hyperintelligent systems that are actually good or better at everything a human is, you just need to sell enough CEOs on the idea that they're good enough now to reach a problematic state of the world. Hell, the current "agents" they're shoving out are terrible, but the danger here stems from idiots hooking these things up to actual real world production systems.

We already have AI systems deciding who does or doesn't get a job, or who gets fines and tickets from blurry imagery where they fill in the gaps, or who gets banned off monopolistic digital platforms. Hell, all the grifters and scammers are already using these systems because what they care about is quantity and not quality. Yet instead of discussing the actual real dangers happening right now and what we can do about it, we're instead focusing on some amusing but ultimately irrelevant sci-fi scenarios that exist purely as a form of viral marketing from AI CEOs that have gigantic vested interests in making it seem as if the black boxes they're pushing out into the world are anything like the impressive hyperintelligences you see in sci-fi media.

I'm as big of a fan as Philip K. Dick as anyone else, and maybe there is some validity to worrying a bit about this hypothetical Skynet/Bladerunner/Butlerian Jihad future, but how about we shift more of our focus on the here and now, where real dangers already exist?

2 comments

“Everyone discussing some fission superbomb is falling for the hype.

Nuclear reactors are dangerous not because of some sci-fi chain reaction that might or might not happen, they're dangerous now for perfectly predictable reasons stemming primarily from radiation and radioactive waste.”

The straight forward mitigation for the hypothetical situation is to halt development; this is not what the ai companies are pushing for, so I'm not convinced that this line of thinking can be meaningfully attributed to the marketing strategy of ai companies.

Analogy doesn't work because the bomb came first and the reactor later, that is, regarding fission the reactor isn't even here yet. And it was clear from the beginning that the chain reaction is real not hypothetical.

No, there was a functioning reactor built as part of making the bomb.

And it is not a hypothetical concern that a new type of agent might be fundamentally uncontrollable; we're already barely surviving corporations persuing their inhuman goals.

> Yet instead of discussing the actual real dangers happening right now and what we can do about it

This is almost the definition of "being short-sighted".

You're not wrong that there are pressing matters due to the rise of AI in various forms, but a (let's say) 'military-grade'/state-level AGI system is very much something to worry about hard and early. You can scream "AI hype" as much as you want, but if China gets to AGI first and abuses it to gain world dominance we're still royally fucked (in a variety of situations, including where they lose control of it). If you start thinking about that when it happens, you're already way too late.

You are entirely free to focus on short-term problems, but don't shit on the people who put an effort in looking a bit farther into the future.