Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning models

Background: Perceptions of the built environment, such as nature quality, beauty, relaxation, and safety, may be key factors linking the built environment to human health. However, few studies have examined these types of perceptions due to the difficulty in quantifying them objectively in large populations. Objective: To measure and predict perceptions of the built environment from street-view images using crowd-sourced methods and deep learning models for application in epidemiologic studies. Methods: We used the Amazon Mechanical-Turk crowdsourcing platform where participants compared two street-view images and quantified perceptions of nature quality, beauty, relaxation, and safety. We optimized street-view image sampling methods to improve the quality and resulting perception data specific to participants enrolled in the Washington State Twin Registry (WSTR) health study. We used a transfer learning approach to train deep learning models by leveraging existing image perception data from the PlacePulse 2.0 dataset, which includes 1.1 million image comparisons, and refining based on new WSTR perception data. Resulting models were applied to WSTR addresses to estimate exposures and evaluate associations with traditional built environment measures. Results: We collected over 36,000 image comparisons and calculated perception measures for each image. Our final deep learning models explained 77.6% of nature quality, 68.1% of beauty, 72.0% of relaxation, and 64.7% of safety in pairwise image comparisons. Applying transfer learning with the new perception labels specific to the WSTR yielded an average improvement of 3.8% for model performance. Perception measures were weakly to moderately correlated with traditional built environment exposures for WSTR participant addresses; for example, nature quality and NDVI (r = 0.55), neighborhood area deprivation (r = −0.16), and walkability (r = −0.20), respectively. Significance: We were able to measure and model perceptions of the built environment optimized for a specific health study. Future applications will examine associations between these exposure measures and mental health in the WSTR. Impact statement: Built environments influence health through complex pathways. Perceptions of nature quality, beauty, relaxation and safety may be particularly import for understanding these linkages, but few studies to-date have examined these perceptions objectively for large populations. For quantitative research, an exposure measure must be reproducible, accurate, and precise––here we work to develop such measures for perceptions of the urban environment. We created crowd-sourced and image-based deep learning methods that were able to measure and model these perceptions. Future applications will apply these models to examine associations with mental health in the Washington State Twin Registry.