Evaluating threshold models as prediction models

Understanding the motivation for an individual’s decision to adopt and the consequence diffusion has been the subject of much research and motivated model developments. Rogers’ Diffusion of Innovation suggests an equal effect of all members regardless of their connections. Alternatively, in Granovetter’s or Valente’s cohesion threshold models a node decides whether to adopt a phenomenon based on his or her personal threshold and the adoption of his or her direct connection. Otherwise, Burt proposes examining nodes’ structural equivalence as the point of influence. These fundamental models put light on the different effects on ones’ decision to adopt, and raises the questions, do these effects suffice for prediction. The research reported here investigates the core contagion threshold models in social networks as time of adoption (TOA) predictors, comparing and evaluating their performance over simulated data. To carry out the simulation, the models were coded in R programming language so that in every step of each simulated model, all non-adopting nodes could either adopt or not based on each model. The simulation was performed 1000 time in each setting to account for specific sample variance. The TOA was simulated by applying the simulation code to three well-known datasets: (1) medical innovation, (2) hybrid corn seed by Brazilian farmers, and (3) contraceptives in Korean family planning. The first dataset was used in the original models’ presentation and therefore there is a bigger interest in examining it in this different viewpoint. Moreover, these datasets vary in many aspects such as density, reciprocity, percentage of seed nodes, field, population, TOA distribution etc. which makes the results general and not specific. For model evaluation and comparison, we used acceptable measures of both goodness-of-fit and relative error measures: Mean Absolute Error (MAE), Root Mean Square Error(MSE), Relative Root Mean Square Error (RRMSE), Nash-Sutcliffe Efficiency (EF), Pearson’s Correlation Coefficient and Coefficient of Determination (R2). In each simulation, the different measures were calculated and averaged across the 1000 runs. The results showed that the random Rogers’ diffusion of innovation model took the lead in terms of the distance from the real TOA measures (goodness-of-fit). However, all models were far from the real TOA, with a minimum of 80% RRMSE. The threshold models using cohesion and structural equivalence took the lead in terms of the correlation measures (relative error), which represent a similarity in the trend. Nevertheless, the baseline of steady prediction of the average TOA yielded better results than the existing threshold models tested even upon using the actual calculate cohesion threshold in the simulation suggesting that for prediction purposes a threshold is only one factor in determining the action of the individual. Finally, we discuss enhancements that can improve the predictability of the model, one of which even improved the actual calculated threshold simulation. These suggestions sharpen the perspective on threshold models, the factors that are at play in them and the attributes affecting them.