A hybrid approach for automatic model recommendation

Roman Vainshtein; Asnat Greenstein-Messica; Gilad Katz; Bracha Shapira; Lior Rokach

doi:10.1145/3269206.3269299

A hybrid approach for automatic model recommendation

Roman Vainshtein, Asnat Greenstein-Messica, Gilad Katz, Bracha Shapira, Lior Rokach

Department of Software and Information Systems Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

23 Scopus citations

Abstract

One of the challenges of automating machine learning applications is the automatic selection of an algorithmic model for a given problem. We present AutoDi, a novel and resource-efficient approach for model selection. Our approach combines two sources of information: metafeatures extracted from the data itself and word-embedding features extracted from a large corpus of academic publications. This hybrid approach enables AutoDi to select top-performing algorithms both for widely and rarely used datasets by utilizing its two types of feature sets. We demonstrate the effectiveness of our proposed approach on a large dataset of 119 datasets and 179 classification algorithms grouped into 17 families. We show that AutoDi can reach an average of 98.8% of optimal accuracy and select the optimal classification algorithm in 49.5% of all cases.

Original language	English
Title of host publication	CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management
Editors	Norman Paton, Selcuk Candan, Haixun Wang, James Allan, Rakesh Agrawal, Alexandros Labrinidis, Alfredo Cuzzocrea, Mohammed Zaki, Divesh Srivastava, Andrei Broder, Assaf Schuster
Publisher	Association for Computing Machinery
Pages	1623-1626
Number of pages	4
ISBN (Electronic)	9781450360142
DOIs	https://doi.org/10.1145/3269206.3269299
State	Published - 17 Oct 2018
Event	27th ACM International Conference on Information and Knowledge Management, CIKM 2018 - Torino, Italy Duration: 22 Oct 2018 → 26 Oct 2018

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	27th ACM International Conference on Information and Knowledge Management, CIKM 2018
Country/Territory	Italy
City	Torino
Period	22/10/18 → 26/10/18

Keywords

Algorithm Recommendation
Classification
Classifier Families
Dataset Meta-features
Expert System
Meta-Learning
Scholarly Big Data
Word Embedding

ASJC Scopus subject areas

General Business, Management and Accounting
General Decision Sciences

Access to Document

10.1145/3269206.3269299

Cite this

Vainshtein, R., Greenstein-Messica, A., Katz, G., Shapira, B., & Rokach, L. (2018). A hybrid approach for automatic model recommendation. In N. Paton, S. Candan, H. Wang, J. Allan, R. Agrawal, A. Labrinidis, A. Cuzzocrea, M. Zaki, D. Srivastava, A. Broder, & A. Schuster (Eds.), CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 1623-1626). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3269206.3269299

Vainshtein, Roman ; Greenstein-Messica, Asnat ; Katz, Gilad et al. / A hybrid approach for automatic model recommendation. CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management. editor / Norman Paton ; Selcuk Candan ; Haixun Wang ; James Allan ; Rakesh Agrawal ; Alexandros Labrinidis ; Alfredo Cuzzocrea ; Mohammed Zaki ; Divesh Srivastava ; Andrei Broder ; Assaf Schuster. Association for Computing Machinery, 2018. pp. 1623-1626 (International Conference on Information and Knowledge Management, Proceedings).

@inproceedings{7b692f103e5847459a0ff2aa6619ae98,

title = "A hybrid approach for automatic model recommendation",

abstract = "One of the challenges of automating machine learning applications is the automatic selection of an algorithmic model for a given problem. We present AutoDi, a novel and resource-efficient approach for model selection. Our approach combines two sources of information: metafeatures extracted from the data itself and word-embedding features extracted from a large corpus of academic publications. This hybrid approach enables AutoDi to select top-performing algorithms both for widely and rarely used datasets by utilizing its two types of feature sets. We demonstrate the effectiveness of our proposed approach on a large dataset of 119 datasets and 179 classification algorithms grouped into 17 families. We show that AutoDi can reach an average of 98.8% of optimal accuracy and select the optimal classification algorithm in 49.5% of all cases.",

keywords = "Algorithm Recommendation, Classification, Classifier Families, Dataset Meta-features, Expert System, Meta-Learning, Scholarly Big Data, Word Embedding",

author = "Roman Vainshtein and Asnat Greenstein-Messica and Gilad Katz and Bracha Shapira and Lior Rokach",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computing Machinery.; 27th ACM International Conference on Information and Knowledge Management, CIKM 2018 ; Conference date: 22-10-2018 Through 26-10-2018",

year = "2018",

month = oct,

day = "17",

doi = "10.1145/3269206.3269299",

language = "English",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "1623--1626",

editor = "Norman Paton and Selcuk Candan and Haixun Wang and James Allan and Rakesh Agrawal and Alexandros Labrinidis and Alfredo Cuzzocrea and Mohammed Zaki and Divesh Srivastava and Andrei Broder and Assaf Schuster",

booktitle = "CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management",

}

Vainshtein, R, Greenstein-Messica, A, Katz, G , Shapira, B & Rokach, L 2018, A hybrid approach for automatic model recommendation. in N Paton, S Candan, H Wang, J Allan, R Agrawal, A Labrinidis, A Cuzzocrea, M Zaki, D Srivastava, A Broder & A Schuster (eds), CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 1623-1626, 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, 22/10/18. https://doi.org/10.1145/3269206.3269299

A hybrid approach for automatic model recommendation. / Vainshtein, Roman; Greenstein-Messica, Asnat; Katz, Gilad et al.
CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ed. / Norman Paton; Selcuk Candan; Haixun Wang; James Allan; Rakesh Agrawal; Alexandros Labrinidis; Alfredo Cuzzocrea; Mohammed Zaki; Divesh Srivastava; Andrei Broder; Assaf Schuster. Association for Computing Machinery, 2018. p. 1623-1626 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A hybrid approach for automatic model recommendation

AU - Vainshtein, Roman

AU - Greenstein-Messica, Asnat

AU - Katz, Gilad

AU - Shapira, Bracha

AU - Rokach, Lior

PY - 2018/10/17

Y1 - 2018/10/17

N2 - One of the challenges of automating machine learning applications is the automatic selection of an algorithmic model for a given problem. We present AutoDi, a novel and resource-efficient approach for model selection. Our approach combines two sources of information: metafeatures extracted from the data itself and word-embedding features extracted from a large corpus of academic publications. This hybrid approach enables AutoDi to select top-performing algorithms both for widely and rarely used datasets by utilizing its two types of feature sets. We demonstrate the effectiveness of our proposed approach on a large dataset of 119 datasets and 179 classification algorithms grouped into 17 families. We show that AutoDi can reach an average of 98.8% of optimal accuracy and select the optimal classification algorithm in 49.5% of all cases.

AB - One of the challenges of automating machine learning applications is the automatic selection of an algorithmic model for a given problem. We present AutoDi, a novel and resource-efficient approach for model selection. Our approach combines two sources of information: metafeatures extracted from the data itself and word-embedding features extracted from a large corpus of academic publications. This hybrid approach enables AutoDi to select top-performing algorithms both for widely and rarely used datasets by utilizing its two types of feature sets. We demonstrate the effectiveness of our proposed approach on a large dataset of 119 datasets and 179 classification algorithms grouped into 17 families. We show that AutoDi can reach an average of 98.8% of optimal accuracy and select the optimal classification algorithm in 49.5% of all cases.

KW - Algorithm Recommendation

KW - Classification

KW - Classifier Families

KW - Dataset Meta-features

KW - Expert System

KW - Meta-Learning

KW - Scholarly Big Data

KW - Word Embedding

UR - http://www.scopus.com/inward/record.url?scp=85058009450&partnerID=8YFLogxK

U2 - 10.1145/3269206.3269299

DO - 10.1145/3269206.3269299

M3 - Conference contribution

AN - SCOPUS:85058009450

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 1623

EP - 1626

BT - CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management

A2 - Paton, Norman

A2 - Candan, Selcuk

A2 - Wang, Haixun

A2 - Allan, James

A2 - Agrawal, Rakesh

A2 - Labrinidis, Alexandros

A2 - Cuzzocrea, Alfredo

A2 - Zaki, Mohammed

A2 - Srivastava, Divesh

A2 - Broder, Andrei

A2 - Schuster, Assaf

PB - Association for Computing Machinery

T2 - 27th ACM International Conference on Information and Knowledge Management, CIKM 2018

Y2 - 22 October 2018 through 26 October 2018

ER -

Vainshtein R, Greenstein-Messica A, Katz G , Shapira B , Rokach L. A hybrid approach for automatic model recommendation. In Paton N, Candan S, Wang H, Allan J, Agrawal R, Labrinidis A, Cuzzocrea A, Zaki M, Srivastava D, Broder A, Schuster A, editors, CIKM 2018 - Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2018. p. 1623-1626. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3269206.3269299

A hybrid approach for automatic model recommendation

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this