TY - GEN
T1 - Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
AU - Gerstgrasser, Matthias
AU - Danino, Tom
AU - Keren, Sarah
N1 - Publisher Copyright:
© 2023 Neural information processing systems foundation. All rights reserved.
PY - 2023/1/1
Y1 - 2023/1/1
N2 - We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training.The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn.Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents.We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms.Further, sharing only a small number of highly relevant experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters and DQN variants.
AB - We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training.The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn.Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents.We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms.Further, sharing only a small number of highly relevant experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters and DQN variants.
UR - https://www.scopus.com/pages/publications/85191155483
M3 - Conference contribution
AN - SCOPUS:85191155483
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 36 - 37th Conference on Neural Information Processing Systems, NeurIPS 2023
A2 - Oh, A.
A2 - Neumann, T.
A2 - Globerson, A.
A2 - Saenko, K.
A2 - Hardt, M.
A2 - Levine, S.
PB - Neural information processing systems foundation
T2 - 37th Conference on Neural Information Processing Systems, NeurIPS 2023
Y2 - 10 December 2023 through 16 December 2023
ER -