Outliers in the LIGO black hole mass function from coagulation in dense clusters

The advanced LIGO O3a run catalogue has been recently published, and it includes several events with unexpected mass properties, including mergers with individual masses in the lower and upper mass gaps, as well as mergers with unusually small mass ratios between the binary components. Here, we entertain the possibility that these outliers are the outcome of hierarchical mergers of black holes or neutron stars in the dense environments of globular clusters. We use the coagulation equation to study the evolution of the black hole mass function within a typical cluster. Our prescription allows us to monitor how various global quantities change with time, such as the total mass and number of compact objects in the cluster, its overall merger rate, and the probability to form intermediate-mass black holes via a runaway process. By accounting for the LIGO observational bias, we predict the merger event distributions with respect to various variables such as the individual masses M1 and M2, their ratio q, and redshift z, and we compare our predictions with the published O3a data. We study how these distributions depend on the merger-rate and ejections parameters and produce forecasts for the (tight) constraints that can be placed on our model parameters using the future data set of the O5 run. Finally, we also consider the presence of a static channel with no coagulation producing merger events alongside the dynamic channel, finding that the two can be distinguished based solely on the merger mass distribution with future O5 data.