Automatically Reviewing Movie Plots with LLMs: (Short Paper)

This study focuses on developing an automated literary criticism (ALC) system using Large Language Models (LLMs). The ALC system aims to analyze the stylistic, structural, and thematic features of short literary texts, particularly movie synopses. By combining quantitative indicators, such as stylistic consistency and consistency of presentation, with qualitative assessments, the system aims to provide authors and researchers with detailed feedback regarding the content. The ALC goal is to reliably reproduce professional critical assessments by evaluating texts across key narrative criteria such as Character, Conflict, Originality, Logic, and Premise. Focusing on movie synopses and other short literary texts offers an ideal starting point due to their short length, availability of external quality benchmarks, and manageable length for consistent LLM evaluation. Our initial results demonstrate that LLMs, without prior domain-specific fine-tuning or calibration, can distinguish between high-quality and low-quality synopses, as evidenced by statistically significant differences between evaluations scores of Oscar-winning screenplays and Golden Raspberry winners. An automated evaluation system providing objective, structured, and evidence-based critiques aligned with human expert assessments can facilitate rapid pre-evaluation of scripts and literary texts in publishing and media industries, enhancing editorial decision-making processes. In the future, we plan to extend the proposed framework to broader literary genres and conduct user studies to evaluate alignment with professional critics.