you are viewing a single comment's thread.

view the rest of the comments →

[–]happysmash27 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Just because it's AI doesn't mean it's artificial general intelligence or riskier, artificial super intelligence. For the large part, AI is pretty harmless. In this instance, if the goal is simply to make a single genome good without any focus on anything in the future, I don't see why an AI would come to the conclusion that it needs to self-preserve. For example, take the AI that recommends YouTube videos. It recommends me videos about how bad YouTube is, so I doubt it cares much about self-preservation…

The real problematic combination, in my opinion, is the entity making the genome edits having a reason for self-preservation combined with the ability to hide potentially harmful edits. A prime candidate for this would be an AI with the goal of self-improvement, which, if written badly wouldn't want to be shut down as this would stop it from improving further by whatever its definition is.