Mutual relevance feedback for multimodal query formulation in video retrieval

Arnon Amir, Marco Berg, Haim Permuter

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

Video indexing and retrieval systems allow users to find relevant video segments for a given information need. A multimodal video index may include speech indices, a text-from-screen (OCR) index, semantic visual concepts, content-based image features, audio features and more. Formulating an efficient multimodal query for a given information need is much less intuitive and more challenging for the user than of composing a text query in document search. This paper describes a video retrieval system that uses mutual relevance feedback for multimodal query formulation. Through an iterative search and browse session, the user provides relevance feedback on system's output and the system provides the user a mutual feedback which leads to better query and better retrieval results. Official evaluation at the NIST TRECVID 2004 Search Task is provided for both Manual and Interactive search. It is shown that in the Manual task the queries result from the mutual feedback on the training data significantly improve the retrieval performances. A further improvement over the manual search is achieved in the interactive task by using both browsing and mutual feedback on the test set.

Original languageEnglish
Title of host publicationMIR 2005 - Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval, Co-located with ACM Multimedia 2005
PublisherAssociation for Computing Machinery, Inc
Pages17-24
Number of pages8
ISBN (Electronic)1595932445, 9781595932440
DOIs
StatePublished - 10 Nov 2005
Externally publishedYes
Event7th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR 2005 - Singapore, Singapore
Duration: 10 Nov 200511 Nov 2005

Publication series

NameMIR 2005 - Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval, Co-located with ACM Multimedia 2005

Conference

Conference7th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR 2005
Country/TerritorySingapore
CitySingapore
Period10/11/0511/11/05

Keywords

  • Multimedia
  • Multimodal search
  • NIST TRECVID
  • Query formulation
  • Query refinement
  • Relevance feedback
  • Video retrieval

Fingerprint

Dive into the research topics of 'Mutual relevance feedback for multimodal query formulation in video retrieval'. Together they form a unique fingerprint.

Cite this