Analyst intuition based Hidden Markov Model on high speed, temporal cyber security big data

T. T. Teoh, Y. Y. Nguwi, Yuval Elovici, N. M. Cheung, W. L. Ng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

11 Scopus citations

Abstract

Hidden Markov Models (HMM) are probabilistic models that can be used for forecasting time series data. It has seen success in various domains like finance [1-5], bioinformatics [6-8], healthcare [9-11], agriculture [12-14], artificial intelligence[15-17]. However, the use of HMM in cyber security found to date is numbered. We believe the properties of HMM being predictive, probabilistic, and its ability to model different naturally occurring states form a good basis to model cyber security data. It is hence the motivation of this work to provide the initial results of our attempts to predict security attacks using HMM. A large network datasets representing cyber security attacks have been used in this work to establish an expert system. The characteristics of attacker's IP addresses can be extracted from our integrated datasets to generate statistical data. The cyber security expert provides the weight of each attribute and forms a scoring system by annotating the log history. We applied HMM to distinguish between a cyber security attack, unsure and no attack by first breaking the data into 3 cluster using Fuzzy K mean (FKM), then manually label a small data (Analyst Intuition) and finally use HMM state-based approach. By doing so, our results are very encouraging as compare to finding anomaly in a cyber security log, which generally results in creating huge amount of false detection.

Original languageEnglish
Title of host publicationICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery
EditorsLiang Zhao, Lipo Wang, Guoyong Cai, Kenli Li, Yong Liu, Guoqing Xiao
PublisherInstitute of Electrical and Electronics Engineers
Pages2080-2083
Number of pages4
ISBN (Electronic)9781538621653
DOIs
StatePublished - 21 Jun 2018
Externally publishedYes
Event13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2017 - Guilin, Guangxi, China
Duration: 29 Jul 201731 Jul 2017

Publication series

NameICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery

Conference

Conference13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2017
Country/TerritoryChina
CityGuilin, Guangxi
Period29/07/1731/07/17

Keywords

  • Analyst Intuition
  • Big Data
  • Cyber security
  • Expectation Regulated
  • Fuzzy k-means (FKM)
  • Hidden Markov Model (HMM)
  • High Velocity
  • Multi-layer Perceptron (MLP)
  • Network Protocols
  • Principal Component Analysis (PCA)
  • Virus

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems
  • Information Systems and Management
  • Logic
  • Modeling and Simulation
  • Statistics and Probability

Fingerprint

Dive into the research topics of 'Analyst intuition based Hidden Markov Model on high speed, temporal cyber security big data'. Together they form a unique fingerprint.

Cite this