Shop by image: characterizing visual search in e-commerce

Arnon Dagan, Ido Guy, Slava Novgorodov

Research output: Contribution to journalArticlepeer-review


Visual search has become more popular in recent years, allowing users to search by an image they are taking using their mobile device or uploading from their photo library. One domain in which visual search is especially valuable is electronic commerce, where users seek for items to purchase. Despite the increasing popularity of visual search in e-commerce, no comprehensive study has inspected its characteristics compared to traditional search using a text query. In this work, we present an in-depth comprehensive study of visual e-commerce search. We perform query log analysis of one of the largest e-commerce platforms’ mobile search application. We compare visual and textual search by a variety of characteristics, with special focus on the retrieved results and user interaction with them. We also examine image query characteristics, refinement by attributes, and segmentation by user types. Additionally, we examine, for the first time, a wide variety of visual pre- and post-retrieval query performance predictors, several of which showing strong results. Our study points out a variety of differences between visual and textual e-commerce search. We discuss the implications of these differences for the design of future e-commerce search systems.

Original languageEnglish
Article number2
JournalInformation Retrieval Journal
Issue number1
StatePublished - 1 Jun 2023


  • E-commerce search
  • Product search
  • Query log analysis
  • Query performance prediction
  • Search by image
  • Visual search

ASJC Scopus subject areas

  • Information Systems
  • Library and Information Sciences


Dive into the research topics of 'Shop by image: characterizing visual search in e-commerce'. Together they form a unique fingerprint.

Cite this