TY - JOUR
T1 - Human-in-the-loop active learning via brain computer interface
AU - Netzer, Eitan
AU - Geva, Amir B.
N1 - Funding Information:
We would like to thank the Israel Innovation Authority of the Ministry of Economy and Industry for supporting this study. We would also like to thank InnerEye Ltd. for funding and running the experiments and for providing the EEG classification system used in this study. Eitan Netzer’s work was supported by InnerEye Ltd., Ben Gurion University of the Negev, and the Israel Innovation Authority of the Ministry of Economy and Industry.
Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020/12/1
Y1 - 2020/12/1
N2 - This paper develops and examines an innovative methodology for training an artificial neural network to identify and tag target visual objects in a given database. While the field of Artificial Intelligence in general, and computer vision in particular, has greatly advanced in recent years, fast and efficient methods for tagging (i.e., labeling) visual targets are still lacking. Tagging data is important to train, as it allow to train supervised learning models. However, this is a tiresome task that often creates bottlenecks in academic and industrial research projects. In order to develop an algorithm that improves data tagging processes, this study utilizes the advantages of human cognition and machine learning by combining Brain Computer Interface, Human-In-The-Loop, and Deep Learning. Combining these three fields into one algorithm could enable the rapid annotation of large visual databases that have no prior references and cannot be described as a mathematical optimization function. Human-In-The-Loop is an increasingly researched area that refers to the integration of human feedback in computation processes. At present, computer-based deep learning can only be incorporated in the process of identifying and tagging target objects of interest if a predefined database exists – one that has already been defined by a human user. To reduce the scope of this timely and costly process, our algorithm uses machine learning techniques (i.e., active learning) to minimize the number of target objects a human user needs to identify before the computer can successfully carry out the task independently. In our method, users are connected to electroencephalograms electrodes and shown images using rapid serial visual presentation – a fast method for presenting users with images. Some images are target objects, while others are not. Based on users’ brainwave activity when target objects are shown, the computer learns to identify and tag target objects – already in the learning stage (unlike naïve uniform sampling methods that first require human input, and only then begin the learning stage). As such, our work is proof of concept for the effectiveness of involving humans in the computer’s learning stage, i.e., human-in-the-loop as opposed to the traditional method of humans first tagging the data and the machines then learning and creating a model.
AB - This paper develops and examines an innovative methodology for training an artificial neural network to identify and tag target visual objects in a given database. While the field of Artificial Intelligence in general, and computer vision in particular, has greatly advanced in recent years, fast and efficient methods for tagging (i.e., labeling) visual targets are still lacking. Tagging data is important to train, as it allow to train supervised learning models. However, this is a tiresome task that often creates bottlenecks in academic and industrial research projects. In order to develop an algorithm that improves data tagging processes, this study utilizes the advantages of human cognition and machine learning by combining Brain Computer Interface, Human-In-The-Loop, and Deep Learning. Combining these three fields into one algorithm could enable the rapid annotation of large visual databases that have no prior references and cannot be described as a mathematical optimization function. Human-In-The-Loop is an increasingly researched area that refers to the integration of human feedback in computation processes. At present, computer-based deep learning can only be incorporated in the process of identifying and tagging target objects of interest if a predefined database exists – one that has already been defined by a human user. To reduce the scope of this timely and costly process, our algorithm uses machine learning techniques (i.e., active learning) to minimize the number of target objects a human user needs to identify before the computer can successfully carry out the task independently. In our method, users are connected to electroencephalograms electrodes and shown images using rapid serial visual presentation – a fast method for presenting users with images. Some images are target objects, while others are not. Based on users’ brainwave activity when target objects are shown, the computer learns to identify and tag target objects – already in the learning stage (unlike naïve uniform sampling methods that first require human input, and only then begin the learning stage). As such, our work is proof of concept for the effectiveness of involving humans in the computer’s learning stage, i.e., human-in-the-loop as opposed to the traditional method of humans first tagging the data and the machines then learning and creating a model.
KW - Active Learning
KW - Brain Computer Interface
KW - Clustering
KW - Deep learning
KW - EEG
KW - Human-In-The-Loop
KW - Transfer Learning
UR - http://www.scopus.com/inward/record.url?scp=85081966919&partnerID=8YFLogxK
U2 - 10.1007/s10472-020-09689-0
DO - 10.1007/s10472-020-09689-0
M3 - Article
AN - SCOPUS:85081966919
VL - 88
SP - 1191
EP - 1205
JO - Annals of Mathematics and Artificial Intelligence
JF - Annals of Mathematics and Artificial Intelligence
SN - 1012-2443
IS - 11-12
ER -