Item — HN Reader

Quote:

The philosopher Nick Bostrom, who heads the Future of Humanity Institute at the University of Oxford, says that humans trying to build AI are “like children playing with a bomb”, and that the prospect of machine sentience is a greater threat to humanity than global heating. His 2014 book Superintelligence is seminal. A real AI, it suggests, might secretly manufacture nerve gas or nanobots to destroy its inferior, meat-based makers. Or it might just keep us in a planetary zoo while it gets on with whatever its real business is.

3 comments

Some sibling comments either are not familiar with the book or are mixing up today's AI activities with the goal of general AI.

Bostrom is talking about the results of efforts to create super intelligence / general artificial intelligence (GAI).

I see AI being referred to four things these days:

1. Things that professionals would never refer to as AI. I.e. hyped gadgets and software. The word "smart" has been overused this way for a long time.

2. Today's machine learning applied to business or artistic problems. Often using third party tensor flow and gradient optimization libraries.

3. Next year's machine learning. I.e. improvements in library algorithms or creative uses of them: such as improved tensor flow, gradient optimization, novel data, layer or target definitions, etc.

4. Progress toward GAI, which on a day to day basis often looks a lot like 2 or 3, but where the leadership and budget are there for sustained progress wherever it leads.

GAI isn't going to suddenly pop out of a bottle today. But now is the time to be accessing the risks because in the time frame of decades we are going to cross that line (lots of lines really).

Some risks are existential so best overmanaged and overthought than under. Despite the understandable (in the short term) reaction of some that threats will always be fearmongering or hype.

As far as I can tell, the children are playing with a box that is painted to look like a bomb. Machine learning is not real AI. But that's where all the money is going.

Researchers and engineers are by and large trying to solve concrete real-world problems and don't care what label the public puts on them. Media darlings and lay people are the ones concerned with whether something is "real" AI or not.

Lol, how do you head an institute at Oxford while being so clueless about the relevant subject matter? This stuff is good emotion-grabbing speculation, but it has no bearing on developments in the real-world field of artificial intelligence. Replace "AI" with "fancy regression" and see how serious these claims sound.

To be clear, there are serious ethical concerns and threats to human life associated with modern AI, but they have to do with technology shifting the balances of power between groups of humans (country vs. country, in-group vs. out-group), not superintelligent Terminators hiding nerve gas in your shoes.

That's what you get when you listen to philosophers. They seem to have a dislike of evidence for backing up their claims

That's probably a bit broad - there are good philosophers doing good work in the world, this just might not be one of those cases.