Natasha Jaques PhD Thesis Defense

The speaker, Natasha, introduces herself and sets the stage for the presentation of her thesis. She mentions the significance of the occasion, which includes the defense of her thesis, and acknowledges the presence of her supervisor, Rosa, and other key individuals. Natasha also briefly touches on the scope of her work, which revolves around machine learning and its progress, highlighting the challenges and potential solutions in the field.

Natasha delves into the issue of generalization in machine learning models, particularly in reinforcement learning. She provides examples of how models can fail in new scenarios, such as slight changes in video game environments. Natasha emphasizes the importance of addressing this problem, especially in light of the rapid advancements in deep learning, and introduces the concept of intrinsic motivation as a potential solution.

Natasha discusses the concept of intrinsic motivation in AI agents, exploring how it can drive learning across different tasks. She draws parallels to human motivations, suggesting that social learning is a crucial aspect of human intelligence. Natasha presents her thesis that social learning can be incorporated into AI agents to improve their learning capabilities.

Natasha explains her first project on multi-agent reinforcement learning, focusing on the development of intrinsic social motivation. She describes the methodology of equipping agents with a model to predict other agents' actions and the concept of causal influence as a reward. Natasha presents the idea that by focusing on causal influence, agents can learn to coordinate better.

Natasha uses the example of autonomous vehicles to illustrate the application of her research on causal influence rewards. She discusses the challenges of different vehicles learning from each other without sharing proprietary reward functions. Natasha presents her findings that agents can learn to coordinate better by using causal influence rewards, even in non-cooperative environments.

Natasha moves on to discuss the development of communication protocols among agents. She explains the concept of 'cheap talk' and how it can be used to train agents to communicate effectively. Natasha presents her results, showing that agents can learn meaningful communication strategies that lead to higher collective rewards, suggesting improved coordination.

Natasha shifts focus to the idea of AI agents learning from human preferences, using sentiment analysis as a means to understand and respond to human feedback. She presents a project where AI agents are trained to sense human preferences through text-based interactions, aiming to improve the quality of AI interactions with humans.

Natasha discusses the challenges of training AI agents for natural language generation and the use of sentiment analysis as a reward signal. She explains the methodology of using sentiment detection to guide the training of conversational AI, resulting in agents that can sense and respond to human sentiment, leading to more engaging and satisfying interactions.

Natasha acknowledges the challenges faced in her research, such as the complexity of learning from human interaction data and the difficulties in off-policy reinforcement learning. She also shares her future research interests, including the integration of social learning with human feedback and the development of personalized AI agents that can adapt to individual user preferences.

Natasha concludes her presentation by summarizing her research achievements and expressing gratitude to her advisor, collaborators, and friends. She reflects on the journey of her PhD and the support she received throughout. Natasha ends on a positive note, celebrating the completion of her thesis and opening the floor for questions.