Unknown malcode detection and the imbalance problem

Robert Moskovitch, Dima Stopel, Clint Feher, Nir Nissim, Nathalie Japkowicz, Yuval Elovici

Research output: Contribution to journalArticlepeer-review

57 Scopus citations

Abstract

The recent growth in network usage has motivated the creation of new malicious code for various purposes. Today's signature-based antiviruses are very accurate for known malicious code, but can not detect new malicious code. Recently, classification algorithms were used successfully for the detection of unknown malicious code. But, these studies involved a test collection with a limited size and the same malicious: benign file ratio in both the training and test sets, a situation which does not reflect real-life conditions. We present a methodology for the detection of unknown malicious code, which examines concepts from text categorization, based on n-grams extraction from the binary code and feature selection. We performed an extensive evaluation, consisting of a test collection of more than 30,000 files, in which we investigated the class imbalance problem. In real-life scenarios, the malicious file content is expected to be low, about 10% of the total files. For practical purposes, it is unclear as to what the corresponding percentage in the training set should be. Our results indicate that greater than 95% accuracy can be achieved through the use of a training set that has a malicious file content of less than 33.3%.

Original languageEnglish
Pages (from-to)295-308
Number of pages14
JournalJournal in Computer Virology
Volume5
Issue number4
DOIs
StatePublished - 1 Oct 2009

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Unknown malcode detection and the imbalance problem'. Together they form a unique fingerprint.

Cite this