Unknown malcode detection via text categorization and the imbalance problem

Robert Moskovitch, Dima Stopel, Clint Feher, Nir Nissim, Yuval Elovici

Research output: Contribution to conferencePaperpeer-review

74 Scopus citations

Abstract

Today's signature-based anti-viruses are very accurate, but are limited in detecting new malicious code. Currently, dozens of new malicious codes are created every day, and this number is expected to increase in the coming years. Recently, classification algorithms were used successfully for the detection of unknown malicious code. These studies used a test collection with a limited size where the same malicious-benign-file ratio in both the training and test sets, which does not reflect real-life conditions. In this paper we present a methodology for the detection of unknown malicious code, based on text categorization concepts. We performed an extensive evaluation using a test collection that contains more than 30,000 malicious and benign flies, in which we investigated the imbalance problem. In real-life scenarios, the malicious file content is expected to be low, about 10% of the total files. For practical purposes, it is unclear as to what the corresponding percentage in the training set should be. Our results indicate that greater than 95% accuracy can be achieved through the use of a training set that contains below 20% malicious file content.

Original languageEnglish
Pages156-161
Number of pages6
DOIs
StatePublished - 22 Sep 2008
EventIEEE International Conference on Intelligence and Security Informatics, 2008, IEEE ISI 2008 - Taipei, Taiwan, Province of China
Duration: 17 Jun 200820 Jun 2008

Conference

ConferenceIEEE International Conference on Intelligence and Security Informatics, 2008, IEEE ISI 2008
Country/TerritoryTaiwan, Province of China
CityTaipei
Period17/06/0820/06/08

Keywords

  • Classification algorithms
  • Malicious code detection

Fingerprint

Dive into the research topics of 'Unknown malcode detection via text categorization and the imbalance problem'. Together they form a unique fingerprint.

Cite this