264
people found this helpful, as of 2023
ranked #104,991 most helpful
out of 571,544,897 reviews
★★★★★
It's... complicated
Not surprisingly, 200+ pages later, the author can't answer the 'what is to be done' question concerning the likely emergence of non-human (machine-based) super-intelligence, sometime, possibly soon. This is expected because, as a species, we've always been the smartest ones around and never had to even think about the possibility of coexistence alongside something or someone impossibly smart and smart in ways well beyond our comprehension, possibly driven by goals we can't understand and acting in ways that may cause our extinction.
Building his arguments on available data and extrapolating from there, Bostrom is confident that:
- some form of self-aware, machine super-intelligence is likely to emerge
- we may be unable to stop it, even if we wanted to, no matter how hard we tried
- while we may be unable to stop the emergence of super-intelligence, we could prepare ourselves to manage it and possibly survive it
- us not taking this seriously and not being prepared may result in our extinction while serious pre-emergence debate and preparation may result in some form of co-existence
It's radical and perhaps frightening but our failure to comprehend the magnitude of the risks we are about to confront would be a grave error given that, once super-intelligence begins to manifest itself and act, the change may be extremely quick and we may not be afforded a second chance.
Most of the book concerns itself with the several types of super-intelligence that may develop, the ways in which we may be able to control or at least co-exist with such entities or entity, what the world and literally the Universe may turn into depending on how we plant the initial super-intelligent seed. The author also suggests that it may be possible for us to survive and even benefit if we manage to do everything just about right. Of course, the odds of that happening given human nature are extremely small but some optimism is needed or we'd just give up and allow ourselves to go extinct or perhaps all turn into maintenance workers, serving our all-knowing, all-powerful master.
I am not going to go into any further detail. Bostrom makes his case with competence and humor and this well-researched, original and important work deserves to be read and understood and, hopefully, taken seriously enough for some of us to expand upon his research and act upon our conclusions. I will end my little review here but not before I quote from Bostrom's book, his eloquent warning: "we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time"
How did he come to such a radical and pessimistic conclusion? You better read the book. It's not exactly fifth grade level material but it can be a fascinating read for anyone sufficiently motivated, patient and open-minded.
July 2014 · Books