A Trust-Aware Reinforcement Learning Approach to Enhance Human-Agent Teaming: An Overcooked-AI Study
Embedding a real-time trust prediction model into a reinforcement learning agent's reward function, so that shaped rewards and a bonus mechanism respond to predicted trust dynamics. The trust-aware agent outperformed non-trust-aware baselines when collaborating with a human proxy.
This study integrates a previously developed trust prediction model into a reinforcement learning framework to enhance human-agent collaboration. The ensemble trust prediction model, consisting of long short-term memory and neural network architectures, analyzes gameplay trajectories to predict dynamic trust levels in real-time. We embed this model within the Overcooked-AI environment, modifying the agent's reward strategy to account for trust dynamics. Our integration employs shaped and sparse rewards as well as a bonus point mechanism. Rewards are adjusted based on fluctuations in the predicted trust score to promote trust-building actions and discourage those that erode trust. The bonus point mechanism was intended to incentivize the maintenance of trust above a specified threshold. This novel approach was designed to ensure that the RL agent prioritized trust and cooperation within its decision-making. Experiments demonstrated that our trust-aware RL agent outperformed standard non-trust-aware agents when collaborating with a human-proxy agent. This was evidenced by higher scores achieved in human-proxy gaming scenarios, which validated the benefits of incorporating trust dynamics into agent learning. This work establishes a foundation for developing more adaptive and trustworthy artificial agents in complex collaborative environments.
Presented at the 5th IEEE International Conference on Human-Machine Systems in Abu Dhabi. Joint work with Matthew L. Bolton at the University of Virginia and Peter A. Beling at the Virginia Tech National Security Institute.