How to (better) find a perpetrator in a haystack

Yair Neuman, Yochai Cohen, Yiftach Neuman

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


In many real-world contexts, there is a pressing need to automatically screen for potential perpetrators, such as school shooters, whose prevalence in the population is extremely low. We first explain one possible obstacle in addressing this challenge, which is the confusion between “recognition” and “localization” during a search process. Next, we present a pragmatic screening methodology to the problem along Jaynes Bayesian hypothesis testing procedure. According to this approach, we should first focus our efforts on reducing the size of the haystack rather than on the identification of the needle. The third and major methodological contribution of the paper is in proposing that we may reduce the size of the haystack through the identification and use of unique data cues we describe as “impostors’ cues”. An experiment performed on an artificial data set of 7000 texts, shows that when incorporating these cues in the hypothesis testing procedure, they significantly improve the automatic screening of objects characterized by an attribute of a low prevalence (i.e. a psychopathic signature). The relevance of the proposed approach for Big Data and Homeland security is explained and discussed.

Original languageEnglish
Article number9
JournalJournal of Big Data
Issue number1
StatePublished - 1 Dec 2019


  • Bayes Factor
  • Homeland security
  • Jaynes
  • Lone wolf perpetrators
  • Needle in a haystack
  • Screening
  • Terrorism

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications
  • Information Systems and Management


Dive into the research topics of 'How to (better) find a perpetrator in a haystack'. Together they form a unique fingerprint.

Cite this