Content-based methodology for anomaly detection on the web

Mark Last, Bracha Shapira, Yuval Elovici, Omer Zaafrany, Abraham Kandel

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

18 Scopus citations


As became apparent after the tragic events of September 11, 2001, terrorist organizations and other criminal groups are increasingly using the legitimate ways of Internet access to conduct their malicious activities. Such actions cannot be detected by existing intrusion detection systems that are generally aimed at protecting computer systems and networks from some kind of "cyber attacks". Preparation of an attack against the human society itself can only be detected through analysis of the content accessed by the users. The proposed study aims at developing an innovative methodology for abnormal activity detection, which uses web content as the audit information provided to the detection system. The new behavior-based detection method learns the normal behavior by applying an unsupervised clustering algorithm to the contents of publicly available web pages viewed by a group of similar users. In this paper, we represent page content by the well-known vector space model. The content models of normal behavior are used in real-time to reveal deviation from normal behavior at a specific location on the net. The detection algorithm sensitivity is controlled by a threshold parameter. The method is evaluated by the tradeoff between the detection rate (TP) and the false positive rate (FP).

Original languageEnglish
Title of host publicationAdvances in Web Intelligence
EditorsErnestina Menasalvas, Javier Segovia, Piotr S. Szczepaniak
PublisherSpringer Verlag
Number of pages11
ISBN (Print)3540401245, 9783540401247
StatePublished - 1 Jan 2003
Event1st International Atlantic Web Intelligence Conference, AWIC 2003 - Madrid, Spain
Duration: 5 May 20036 May 2003

Publication series

NameLecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)
ISSN (Print)0302-9743


Conference1st International Atlantic Web Intelligence Conference, AWIC 2003


  • Activity monitoring
  • Anomaly detection
  • Information retrieval
  • Unsupervised clustering
  • User modeling
  • Web security


Dive into the research topics of 'Content-based methodology for anomaly detection on the web'. Together they form a unique fingerprint.

Cite this