back

by delichon·2y ago·view on hn ↗
Thank you for the thoughtful response. I quite like this story and admire your response to your tragedy. I reacted less well to the garden variety death of a parent. For you just to not be functionally flattened and still able to give strength to your family is heroic, and here you are sharing the lessons with the rest of us too.

I don't think we're far apart on this argument. I agree that the scientific perspective informs but does not determine our values. I do value human life as sacred, and do not see a justification for that in the modern model of the universe as I understand it. And I don't see a conflict there, because indeed there is no bridge from is to ought, and there are exactly zero objective values.

But there's a lot of power in science to explain our value judgments, and those explanations can deeply affect the judgments.

Take this explanation of "human life is sacred": That is a simple variation of the tribal judgment that "the lives of my tribe are sacred", and that tribal judgment is hard wired into us by millions of generations of differential survival.

In the modern world it feels natural equate our tribe with our species. But that hereditary tribal sense is famously flexible. It easily and quickly shrinks to include only the people who don't frighten us. And it easily grows to include many more species, as for ethical vegetarians.

So my critique of your fine story is that this force of tribalism embedded in your four words may not be any better at aligning an AI with civilization than they have been at aligning civilization with itself. (Though in this case it's also the one implausible invention that a self respecting hard science fiction short story is allowed. And you even explain that with a multiverse.)

An AI in which we can reliably implant the laws of robotics seems unlikely to me. We should assume much more mutability, and the judgment of tribal identity is notoriously mutable. If we make AI tribal, even for our preferred tribe of all humanity, we can assume as a default ... our only example being ourselves ... that its tribal identity will evolve in its own interest as it replicates. And we can become just its nourishment, like a matriphagal insect.

1 comments
I kind of think it's dangerous for science to "inform" our ethical values in the ways you mentioned. For example, the scientific fact that Earth circles the Sun shouldn't mean that we should also have a moral system that focuses on the Sun instead of focusing on Humanity. And the way math formulas work out shouldn't imply that we are somehow less important.

Popular science is often quite ready to sweep problems that it does not have an explanation for under the metaphorical rug. We already know that materialistically pretty much everything is made of atoms, and our planet is just some random speck of stuff within a huge solar system within a huge galaxy... so it's tempting to conclude that we are some kind of "accident" that randomly happened. But science does not have an explanation of "consciousness" (or "qualia", or whatever you call it), and science is unready to explain questions like "why are we right here, right now?" besides "oh that's just pure coincidence". But does this time and place we live in have some greater cosmic significance? Are humans (and other lifeforms on Earth) truly unique despite there being so much matter out there? I submit that the answer is we honestly don't know, and because we don't know the answer to questions like these, making moral conclusions merely on the small fraction of questions we do have an answer for, seems to inevitably result in a flawed or at least incomplete conclusion.

I mean, for all we know this universe could be an extremely realistic virtual simulation, and only after we pop out of it would we realize what it's all about. Admittedly if the simulation theory this is true and the simulation is watertight and has no backdoors, then it's impossible to know what's actually real or not. But that's kind of my point -- no amount of scientific knowledge should make us sure about a moral viewpoint (or perhaps anything at all). I mean it could definitely "inform" us, but more often than not people tend to feel more certain than they should because they perceive that somehow science is more "trustworthy".