Item — HN Reader

> It is the period between sentience and "advanced" artificial intelligence that should be worrying for two reasons: First, it is close at hand. More importantly, the unintended consequences of tech are never well thought out in the initial stages of adoption.

Pretty much every academic that I'm aware of who works in the field of AI has similar views to Ng, i.e., suggestions of an impending AI cataclysm are wildly overblown and don't fit the facts. The people predicting doom are those who for the most part aren't involved with current research and as such don't have a good perspective on the limitations of current techniques.

There is no magic in modern AI advances and there is no reason to expect that they'll yield anything like generalized intelligence. By the nature of the methods they work well in the specific domains they're trained on and poorly everywhere else.

1 comments

The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball. Dangerous autonomous software is very easy to imagine. What if someone hid code in flight control software that activated at a given time and sent all planes flying into the nearest building. Software is only safe because we program it to be safe. The more capable and widespread software becomes the more dangerous it becomes.

>The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball.

And a very precise detonation device strapped around said ball of uranium.

Software is not by itself dangerous. It only exists in the most abstract sense. What is dangerous is creating physical devices that have the ability to cause physical damage if manipulated in a certain way. Allowing those devices to be controlled autonomously through a software-hardware interface is where the danger from "AI" comes in.

Or if you plug the software into major financial trading hubs... oh...

No, they'll just roll back selected transactions arbitrarily like they did in past flash crashes.

We're much closer to runaway biological threats than we are to AI-inspired ones. Or maybe that's my knowledge gap in the two domains speaking and we're equally far away in each.

Market should take care of it, no?

Yes, but no physicist will tell you its safe to leave uranium lying around. Virtually everyone working in AI, including the biggest names in the field, will tell you that the methods are very domain specific (they generalize, but you have to train them specifically for a given domain), and bear no resemblance to sentience as it is typically understood other than that they give rise to seemingly intelligent behavior. Which actually isn't so much intelligence, as discovering a function mapping inputs to outputs.

And no one is arguing that software can't be dangerous. It clearly can, just like poorly constructed bridges or cars can be dangerous. But the idea of machines thinking for themselves is science fiction. There is no indication that we're anywhere close to that happening.