Some researchers have speculated that capable reinforcement learning (RL) agents pursuing misspecified objectives are often incentivized to seek resources and power in pursuit of those objectives. An agent seeking power is incentivized to behave in undesirable ways, including rationally preventing deactivation and correction. Others have voiced skepticism: humans seem idiosyncratic in their urges to power, which need not be present in the agents we design. We formalize a notion of power within the context of finite deterministic Markov decision processes (MDPs). We prove that, with respect to a neutral class of reward function distributions, optimal policies tend to seek power over the environment.1 Informal definition suggested by Cohen et al. [2019]. 2 https://www.alignmentforum.org/posts/WxW6Gc6f2z3mzmqKs/debate-on-instrumental-convergence-between-lecun-russell
Ethical hacking education prepares future information security professionals with the tools and skills to fight and prevent cybersecurity threats. Businesses, schools, governments, and individuals have an increasing concern to keep their systems, networks, and data secure from outside threats. Most information security technologies use a defensive approach to keep client’s data safe; however, ethical hacking provides one of the only proactive/aggressive methods for information security professionals to utilize. Teaching inexperienced information security professionals these aggressive tactics can be viewed as a double- edged sword. Since they are the same methods used by malicious hackers, educating new security professionals will undoubtedly educate more malicious hackers.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.