Anti-forensic = suspicious: Detection of stealthy malware that hides its network traffic

Mayank Agarwal, Rami Puzis, Jawad Haj-Yahya, Polina Zilberman, Yuval Elovici

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Stealthy malware hides its presence from the users of a system by hooking the relevant libraries, drivers, system calls or manipulating the services commonly used to monitor system behaviour. Tampering the network sensors of host-based intrusion detection systems (HIDS) may impair their ability to detect malware and significantly hinders subsequent forensic investigations. Nevertheless, the mere attempt to hide the traffic indicates malicious intentions. In this paper we show how comparison of the data collected by multiple sensors at different levels of resilience may reveal these intentions. At the lowest level of resilience, information from untrusted sensors such as netstat and process lists are used. At the highest resilience level, we analyse mirrored traffic using a secured hardware device. This technique can be considered as fully trusted. The detection of a discrepancy between what is reported by these common tools and what is observed on a trusted system operating at a different level is a good way to force a dilemma on malware writers: Either apply hiding techniques, with the risk that the discrepancy is detected, or keep the status of network connections untouched, with a greater ability for the administrator to recognize the presence and to understand the behaviour of malware. The proposed method was implemented on an evaluation testbed and is able to detect stealthy malware that hides its communication from the HIDS. The false positive rate is 0.01% of the total traffic analysed, and barring a few exceptions that can easily be white-listed, there are no legitimate processes which raise false alerts.

Original languageEnglish
Title of host publicationICT Systems Security and Privacy Protection - 33rd IFIP TC 11 International Conference, SEC 2018, Held at the 24th IFIP World Computer Congress, WCC 2018, Proceedings
EditorsLech Jan Janczewski, Mirosław Kutyłowski
PublisherSpringer
Pages216-230
Number of pages15
ISBN (Print)9783319998275
DOIs
StatePublished - 1 Jan 2018

Publication series

NameIFIP Advances in Information and Communication Technology
Volume529
ISSN (Print)1868-4238
ISSN (Electronic)1868-422X

Keywords

  • Command & control
  • Malware
  • Security
  • Stealthy
  • Trusted network monitor

ASJC Scopus subject areas

  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Anti-forensic = suspicious: Detection of stealthy malware that hides its network traffic'. Together they form a unique fingerprint.

Cite this