Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Simon Iosti; Doron Peled; Khen Aharon; Saddek Bensalem; Yoav Goldberg

doi:10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Simon Iosti, Doron Peled, Khen Aharon, Saddek Bensalem, Yoav Goldberg

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

Original language	English
Title of host publication	Leveraging Applications of Formal Methods, Verification and Validation
Subtitle of host publication	Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings
Editors	Tiziana Margaria, Bernhard Steffen
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	457-472
Number of pages	16
ISBN (Print)	9783030614690
DOIs	https://doi.org/10.1007/978-3-030-61470-6_27
State	Published - 1 Jan 2020
Externally published	Yes
Event	9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020 - Rhodes, Greece Duration: 20 Oct 2020 → 30 Oct 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12477 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020
Country/Territory	Greece
City	Rhodes
Period	20/10/20 → 30/10/20

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-61470-6_27

Cite this

Iosti, S., Peled, D., Aharon, K., Bensalem, S., & Goldberg, Y. (2020). Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. In T. Margaria, & B. Steffen (Eds.), Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings (pp. 457-472). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12477 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-61470-6_27

Iosti, Simon ; Peled, Doron ; Aharon, Khen et al. / Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. editor / Tiziana Margaria ; Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2020. pp. 457-472 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3141425566bb4925ba22e64d4bd9f515,

title = "Synthesizing Control for a System with Black Box Environment, Based on Deep Learning",

abstract = "We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.",

author = "Simon Iosti and Doron Peled and Khen Aharon and Saddek Bensalem and Yoav Goldberg",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020 ; Conference date: 20-10-2020 Through 30-10-2020",

year = "2020",

month = jan,

day = "1",

doi = "10.1007/978-3-030-61470-6_27",

language = "English",

isbn = "9783030614690",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "457--472",

editor = "Tiziana Margaria and Bernhard Steffen",

booktitle = "Leveraging Applications of Formal Methods, Verification and Validation",

address = "Germany",

}

Iosti, S, Peled, D, Aharon, K, Bensalem, S & Goldberg, Y 2020, Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. in T Margaria & B Steffen (eds), Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12477 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 457-472, 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020, Rhodes, Greece, 20/10/20. https://doi.org/10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. / Iosti, Simon; Peled, Doron; Aharon, Khen et al.
Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. ed. / Tiziana Margaria; Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2020. p. 457-472 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12477 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

AU - Iosti, Simon

AU - Peled, Doron

AU - Aharon, Khen

AU - Bensalem, Saddek

AU - Goldberg, Yoav

PY - 2020/1/1

Y1 - 2020/1/1

N2 - We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

AB - We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

UR - http://www.scopus.com/inward/record.url?scp=85096593874&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-61470-6_27

DO - 10.1007/978-3-030-61470-6_27

M3 - Conference contribution

AN - SCOPUS:85096593874

SN - 9783030614690

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 457

EP - 472

BT - Leveraging Applications of Formal Methods, Verification and Validation

A2 - Margaria, Tiziana

A2 - Steffen, Bernhard

PB - Springer Science and Business Media Deutschland GmbH

T2 - 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020

Y2 - 20 October 2020 through 30 October 2020

ER -

Iosti S, Peled D, Aharon K, Bensalem S, Goldberg Y. Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. In Margaria T, Steffen B, editors, Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 457-472. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this