Reinforcement Learning Benchmarks for Traffic Signal Control

James Ault; Guni Sharon

Reinforcement Learning Benchmarks for Traffic Signal Control

James Ault
, Guni Sharon

Research output: Contribution to journal › Conference article › peer-review

95 Scopus citations

Abstract

We propose a toolkit for developing and comparing reinforcement learning (RL)based traffic signal controllers. The toolkit includes implementation of state-of-the-art deep-RL algorithms for signal control along with benchmark control problems that are based on realistic traffic scenarios. Importantly, the toolkit allows a first-of-its-kind comparison between state-of-the-art RL-based signal controllers while providing benchmarks for future comparisons. Consequently, we compare and report the relative performance of current RL algorithms. The experimental results suggest that previous algorithms are not robust to varying sensing assumptions and non-stylized intersection layouts. When more realistic signal layouts and advanced sensing capabilities are considered, a distributed deep Q-learning approach is shown to outperform previously reported state-of-the-art algorithms in many cases.

Original language	English
Journal	Advances in Neural Information Processing Systems
State	Published - 1 Jan 2021
Externally published	Yes
Event	35th Conference on Neural Information Processing Systems - Track on Datasets and Benchmarks, NeurIPS Datasets and Benchmarks 2021 - Virtual, Online Duration: 6 Dec 2021 → 14 Dec 2021

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 11 Sustainable Cities and Communities

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

@article{32dc6df89b5842eca3902ca42f6b50f7,

title = "Reinforcement Learning Benchmarks for Traffic Signal Control",

abstract = "We propose a toolkit for developing and comparing reinforcement learning (RL)based traffic signal controllers. The toolkit includes implementation of state-of-the-art deep-RL algorithms for signal control along with benchmark control problems that are based on realistic traffic scenarios. Importantly, the toolkit allows a first-of-its-kind comparison between state-of-the-art RL-based signal controllers while providing benchmarks for future comparisons. Consequently, we compare and report the relative performance of current RL algorithms. The experimental results suggest that previous algorithms are not robust to varying sensing assumptions and non-stylized intersection layouts. When more realistic signal layouts and advanced sensing capabilities are considered, a distributed deep Q-learning approach is shown to outperform previously reported state-of-the-art algorithms in many cases.",

author = "James Ault and Guni Sharon",

note = "Publisher Copyright: {\textcopyright} 2021 Neural information processing systems foundation. All rights reserved.; 35th Conference on Neural Information Processing Systems - Track on Datasets and Benchmarks, NeurIPS Datasets and Benchmarks 2021 ; Conference date: 06-12-2021 Through 14-12-2021",

year = "2021",

month = jan,

day = "1",

language = "English",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

publisher = "Neural information processing systems foundation",

}

TY - JOUR

T1 - Reinforcement Learning Benchmarks for Traffic Signal Control

AU - Ault, James

AU - Sharon, Guni

PY - 2021/1/1

Y1 - 2021/1/1

N2 - We propose a toolkit for developing and comparing reinforcement learning (RL)based traffic signal controllers. The toolkit includes implementation of state-of-the-art deep-RL algorithms for signal control along with benchmark control problems that are based on realistic traffic scenarios. Importantly, the toolkit allows a first-of-its-kind comparison between state-of-the-art RL-based signal controllers while providing benchmarks for future comparisons. Consequently, we compare and report the relative performance of current RL algorithms. The experimental results suggest that previous algorithms are not robust to varying sensing assumptions and non-stylized intersection layouts. When more realistic signal layouts and advanced sensing capabilities are considered, a distributed deep Q-learning approach is shown to outperform previously reported state-of-the-art algorithms in many cases.

AB - We propose a toolkit for developing and comparing reinforcement learning (RL)based traffic signal controllers. The toolkit includes implementation of state-of-the-art deep-RL algorithms for signal control along with benchmark control problems that are based on realistic traffic scenarios. Importantly, the toolkit allows a first-of-its-kind comparison between state-of-the-art RL-based signal controllers while providing benchmarks for future comparisons. Consequently, we compare and report the relative performance of current RL algorithms. The experimental results suggest that previous algorithms are not robust to varying sensing assumptions and non-stylized intersection layouts. When more realistic signal layouts and advanced sensing capabilities are considered, a distributed deep Q-learning approach is shown to outperform previously reported state-of-the-art algorithms in many cases.

UR - https://www.scopus.com/pages/publications/105000254296

M3 - Conference article

AN - SCOPUS:105000254296

SN - 1049-5258

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - 35th Conference on Neural Information Processing Systems - Track on Datasets and Benchmarks, NeurIPS Datasets and Benchmarks 2021

Y2 - 6 December 2021 through 14 December 2021

ER -

Reinforcement Learning Benchmarks for Traffic Signal Control

Abstract

UN SDGs

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this