Habitat heterogeneity limits prey colour polymorphism maintained via negative frequency-dependent selection

The persistence of non-neutral trait polymorphism is enigmatic because stabilizing selection is expected to deplete variation. In cryptically coloured prey, negative frequency-dependent selection due to search image formation by predators has been proposed to favour rare variants, promoting polymorphism. However, in a heterogeneous environment, locally varying disruptive selection favours patch type-specific optima, resulting in spatial segregation of colour variants. Here, we address whether negative frequency-dependent selection can overcome selection posed by habitat heterogeneity to promote local polymorphism using an individual-based model. In addition, we compare how prey and predator mobility may modify the outcome. Our model revealed that frequency-dependent predation could strongly promote local prey polymorphism, but only when differences between morphs in patch-specific fitness were small. The effect of frequency-dependent predation depended on the predator adjustment of search image and was hampered by the prey population structure. Gene flow due to prey movement counteracted local selection, promoted local polymorphism to some extent, and relaxed the conditions for polymorphism due to frequency-dependent predation. Importantly, abrupt spatial changes in morph frequencies decreased the probability that mobile frequency-dependent predators could maintain local prey polymorphism. Overall, our study suggests that in a spatially heterogeneous environment, negative frequency-dependent selection may help maintain local polymorphism but only under a limited range of conditions.