CineSteady: A Real-Time Hybrid Video Stabilization and Distortion Correction Method

The rapid development of the consumer electronics industry has highlighted the significance of video stabilization technologies for devices like smartphones. Nevertheless, State-of-the-Art (SOTA) solutions face significant challenges in real-world deployment, primarily due to limitations in stability, image distortion control, black border elimination, and real-time processing capabilities. To address these challenges, we propose CineSteady, a novel and comprehensive video stabilization algorithm framework. First, by using gyroscope data and image sequences, CineSteady achieves high-order smoothing of video sequences under complex motion scenarios, delivering a top-tier stabilization experience for users. In addition, we introduce a forward-backward filtering algorithm that rigorously enforces boundary constraints. This approach guarantees black-border-free output within a fixed Field of View (FOV) while maximizing video stability. Moreover, to address practical deployment challenges on mobile devices, we present Optical Lens-based Rolling Shutter Correction (OLRC), a comprehensive camera and distortion correction model. By leveraging physical imaging models and 3D motion estimation, OLRC effectively eliminates the jello effect. Finally, the proposed algorithm demonstrates exceptional computational efficiency, achieving real-time performance on the Xiaomi 15 Ultra powered by the Qualcomm Snapdragon 8 Elite platform, supporting 4K resolution at 60 frames per second. Evaluations in diverse real-world scenarios confirm that CineSteady surpasses SOTA video stabilization methods in user experience, demonstrating significant advantages of the proposed solution.