MND: A New Dataset and Benchmark of Movie Scenes Classified by Their Narrative Function

The success of Hollywood cinema is partially attributed to the notion that Hollywood film-making constitutes both an art and an industry: an artistic tradition based on a standardized approach to cinematic narration. Film theorists have explored the narrative structure of movies and identified forms and paradigms that are common to many movies - a latent narrative structure. We raise the challenge of understanding and formulating the movie story structure and introduce a novel story-based labeled dataset-the Movie Narrative Dataset (MND). The dataset consists of 6,448 scenes taken from the manual annotation of 45 cinema movies, by 119 distinct annotators. The story-related function of each scene was manually labeled by at least six different human annotators as one of 15 possible key story elements (such as Set-Up, Debate, and Midpoint) defined in screenwriting guidelines. To benchmark the task of scene classification by their narrative function, we trained an XGBoost classifier that uses simple temporal features and character co-occurrence features to classify each movie scene into one of the story beats. With five-fold cross-validation over the movies, the XGBoost classifier produced an F1 measure of 0.31 which is statistically significant above a static baseline classifier. These initial results indicate the ability of machine learning approaches to detect the narrative structure in movies. Hence, the proposed dataset should contribute to the development of story-related video analytics tools, such as automatic video summarization and movie recommendation systems.