back

by pkoird·1y ago·view on hn ↗
The answer is Perpetuation. Only those "structures" that learn to perpetuate can exist for a significant timeframe given the intrinsic randomness and the vast scale of universe. Be it a cell, or a cluster of cells, or an organism in its entirety, no matter how "intelligent" it is, if it does not figure out how to perpetuate, it will certainly die off due to entropy. So the most meaningful of all rewards, as far as it relates to AGI, is decidedly perpetuation* because any agent optimizing for otherwise will cease to exist.

*Note that improvement is also a perpetuation as it helps perpetuate in a changing world.