Artificial Intelligence (AI) has become an integral part of our lives, prompting crucial questions about ensuring AI systems align with human intentions. Michael Zhang, a PhD student in computer science at the University of Toronto and a graduate fellow at the Schwartz Reisman Institute for Technology and Society, delves into the complexities of AI safety and discusses ongoing efforts to keep AI on the right track.
In a conversation with U of T News, Zhang sheds light on the concept of AI alignment, highlighting challenges such as reward misspecification and bias. He emphasizes the need to ensure AI systems follow intended objectives, especially as they become more sophisticated.
“In the research sense, it means trying to make sure that AI does what we intended it to do – so it follows the objectives that we try to give it,” says Zhang. He points out the challenges arising from reward misspecification, where defining a precise reward function can lead to unintended consequences. Additionally, bias in training data can result in AI systems making decisions that perpetuate existing biases.
Zhang explains that AI models, particularly large language models like ChatGPT, learn from diverse datasets without specific hard-coded rules. This can result in emergent behaviors or abilities in larger models that were not anticipated in smaller ones. Hallucinations, where models generate plausible but false claims, are cited as an example of these unforeseen behaviors.
The discussion extends to Artificial General Intelligence (AGI), the potential for AI systems to outperform humans in most tasks requiring intelligence. Zhang explores concerns about AGI aligning with human values and the associated risks, cautioning against potential scenarios where highly intelligent AI systems may not prioritize human well-being.
Zhang outlines five key areas of AI alignment research: specification, interpretability, monitoring, robustness, and governance. The Schwartz Reisman Institute plays a pivotal role in interdisciplinary collaboration to address these challenges. Zhang discusses ongoing efforts, including encoding human principles for AI models, improving interpretability, systematic monitoring of model capabilities, ensuring robustness against unusual events, and establishing effective governance.
As the debate on the future of AI intensifies, U of T researchers strive to navigate short- and long-term risks, contributing valuable insights and technical solutions to guide the responsible development of AI technologies.
