Parametric Human Movements: Learning, Synthesis, Recognition, and Tracking

The thesis aims at the learning of action primitives and their application on the perceptive side (tracking/recognition) and the generative side (synthesizing for robot control). A motivation is to use a unified primitive representation applicable on both sides. The thesis considers arm actions in a table-top scenario (e.g., pointing to, or relocating an object), which are are highly context-dependent (parametric) as they depend on object locations. Suitable models in such a context are generative stochastic models such as the parametric hidden Markov model (PHMM). --- The thesis begins with an investigation of PHMM training methods and structures to utilize the PHMM as a unified representation of parametric primitives, which is adequate for recognition and for synthesis. This is evaluated on a large motion data set. Main contributions of the thesis are the development and evaluation of approaches to imitation and intertwined tracking/recognition. Both approaches utilize PHMMs. In the imitation application, a humanoid robot is enabled to relocate objects. It is important to synthesize movements such that the robot can reach for arbitrarily located objects, which requires the adaption of the PHMM parameters. Since actions and parameters are learned from demonstration of another (human) embodiment, it is necessary to map both (actions and parameters) consistently on the robot. A rule-learning experiment shows the robot successfully handling several objects. In the tracking and recognition application, action primitives are used to define a space of possible actions and action sequences. The tracking is performed in this space, which reduces the dimensionality of the tracking problem and enables recognition. From the tracking perspective, it is crucial that the parameters of the action primitives can adapt the primitives to the actual appearance of the tracked motion, since the appearance of actions depends on the object locations. From the recognition perspective, it is necessary to recognize a performed action, but the understanding requires also the recovery of the action parameters, which can identify, e.g., the pointed to object. The implemented framework utilizes several actions/models. It runs in real time and can recover body pose, actions, and their parameters online. The experiments show that different actions can be distinguished and that the primitive actions of complex actions can be recognized even from a single view in an view invariant manner.