Classifier evaluation under limited resources

Reuven Arbel; Lior Rokach

doi:10.1016/j.patrec.2006.03.008

Classifier evaluation under limited resources

Reuven Arbel, Lior Rokach

Department of Software and Information Systems Engineering

Research output: Contribution to journal › Article › peer-review

15 Scopus citations

Abstract

Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.

Original language	English
Pages (from-to)	1619-1631
Number of pages	13
Journal	Pattern Recognition Letters
Volume	27
Issue number	14
DOIs	https://doi.org/10.1016/j.patrec.2006.03.008
State	Published - 15 Oct 2006

Keywords

Classification
Evaluation measures
Hit-rate
Recall
Receiver operating characteristic

ASJC Scopus subject areas

Software
Signal Processing
Computer Vision and Pattern Recognition
Artificial Intelligence

Access to Document

10.1016/j.patrec.2006.03.008

Cite this

@article{1075f583440c4839a62fa5136f55e946,

title = "Classifier evaluation under limited resources",

abstract = "Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.",

keywords = "Classification, Evaluation measures, Hit-rate, Recall, Receiver operating characteristic",

author = "Reuven Arbel and Lior Rokach",

note = "Funding Information: Jo Marsden and Hugo Pedder on behalf of The British Menopause Society with acknowledgement to Professor Richard Santen, Department of Endocrinology and Metabolism, University of Virginia, VA, USA.",

year = "2006",

month = oct,

day = "15",

doi = "10.1016/j.patrec.2006.03.008",

language = "English",

volume = "27",

pages = "1619--1631",

journal = "Pattern Recognition Letters",

issn = "0167-8655",

publisher = "Elsevier B.V.",

number = "14",

}

TY - JOUR

T1 - Classifier evaluation under limited resources

AU - Arbel, Reuven

AU - Rokach, Lior

N1 - Funding Information: Jo Marsden and Hugo Pedder on behalf of The British Menopause Society with acknowledgement to Professor Richard Santen, Department of Endocrinology and Metabolism, University of Virginia, VA, USA.

PY - 2006/10/15

Y1 - 2006/10/15

N2 - Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.

AB - Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.

KW - Classification

KW - Evaluation measures

KW - Hit-rate

KW - Recall

KW - Receiver operating characteristic

UR - http://www.scopus.com/inward/record.url?scp=33747871347&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2006.03.008

DO - 10.1016/j.patrec.2006.03.008

M3 - Article

AN - SCOPUS:33747871347

SN - 0167-8655

VL - 27

SP - 1619

EP - 1631

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

IS - 14

ER -

Classifier evaluation under limited resources

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this