A fast and scalable method for threat detection in large-scale DNS logs

Ron Begleiter, Yuval Elovici, Yona Hollander, Ori Mendelson, Lior Rokach, Roi Saltzman

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

This paper presents a fast and scalable method for detecting threats in large-scale DNS logs. In such logs, queries about 'abnormal' domain strings are often correlated with malicious behavior. With our method, a language model algorithm learns 'normal' domain-names from a large dataset to rate the extent of domain-name 'abnormality' within a big data stream of DNS queries in the organization. Variable-order Markov Models (VMMs) serve as out underlying algorithmic tool since their running time is linear in the input sequence while their memory requirements are constantly bounded from above, both very appealing characteristics. Our experimental study indicates that the proposed method can detect domain names generated by a genuine Domain Generation Algorithm, used in Advanced Persistent Threat attack scenarios, with less than 5% false-negative and 1% false-positive rates. This detection rate is similar to more computationally intensive methods that are not scalable for big data environments.

Original languageEnglish
Title of host publicationProceedings - 2013 IEEE International Conference on Big Data, Big Data 2013
PublisherIEEE Computer Society
Pages738-741
Number of pages4
ISBN (Print)9781479912926
DOIs
StatePublished - 1 Jan 2013
Event2013 IEEE International Conference on Big Data, Big Data 2013 - Santa Clara, CA, United States
Duration: 6 Oct 20139 Oct 2013

Publication series

NameProceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Conference

Conference2013 IEEE International Conference on Big Data, Big Data 2013
Country/TerritoryUnited States
CitySanta Clara, CA
Period6/10/139/10/13

Fingerprint

Dive into the research topics of 'A fast and scalable method for threat detection in large-scale DNS logs'. Together they form a unique fingerprint.

Cite this