Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Kamila M. Jozwik, Jonathan O’Keeffe, Katherine R. Storrs, Wenxuan Guo, Tal Golan, Nikolaus Kriegeskorte

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

Original languageEnglish
Article numbere2115047119
JournalProceedings of the National Academy of Sciences of the United States of America
Volume119
Issue number27
DOIs
StatePublished - 5 Jul 2022
Externally publishedYes

Keywords

  • Basel face model
  • deep neural networks
  • face identification
  • face perception
  • face similarity

ASJC Scopus subject areas

  • General

Fingerprint

Dive into the research topics of 'Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models'. Together they form a unique fingerprint.

Cite this