As the first author of the underlying paper, I had a vital contribution and it was solely my responsibility to write down the content and to perform an extensive literature research as well as in-depth analysis. This chapter is based on a slightly modified form of the publication: N.-M. Aliman and L. Kester. Augmented Utilitarianism for AGI Safety. In International Conference on Artificial General Intelligence, pages 11-21. Springer, 2019. As the first author of the underlying paper, I had a vital contribution and it was solely my responsibility to write down the content and to perform an extensive literature research as well as in-depth analysis.