Constructing a skeleton database and enriching it using a Generative Adversarial Network (GAN) simulator to assess human movement

This Ph.D. thesis develops a neural network simulator for quantifying, tagging, and inferring human gestures using an anonymized patient gesture database. Deep Learning (DL) applications require a sufficient data set for training. In this work, we propose enriching a database that contains a limited number of videos of human physiotherapy exercises by generating synthetic data. Our pose generator produces human movement in the form of skeletal vectors. We use OpenPose (OP) to convert videos and images containing multiple individuals into human skeletal. Within every video frame, OP represents each pose of the human skeleton as a vector in three-dimensional Euclidean space. We employ the Generative Adversarial Network (GAN) to generate new samples and control the motion parameters. We rearrange the joints in our skeletal model to emphasize the connections between them by using depth-first search (DFS), a tree structure search algorithm. Moreover, this research examines common challenges associated with capturing human gesture data, including synchronizing activities, temporal, and spatial relations, and how to address them. We intend to build an innovative simulator that will generate a set of human virtual choreography movements from a textual script.