Vision-Based Human Interface System for Hand Manipulative Gestures

Abstract

This paper presents a system recognizing manipulative gestures like grasping, moving, holding an object(s) with both hands, and extending or shortening of the object(s) in the virtual world using contextual information. Contextual information is represented by a state transition diagram, each state of which indicates possible gestures at the next moment. Image features obtained from extracted hand regions are used to judge state transition. When we use a gesture recognition system, we sometimes move our hands unintentionally. To solve this problem, the proposed system can recognize collaborative gestures with both hands. They are expressed in a single state so that the complexity in combination of gestures of each hand can be avoided. We have realized an experimental human interface system. Operational experiments show promising results.

This paper presents a system recognizing manipulative gestures like grasping, moving, holding an object(s) with both hands, and extending or shortening of the object(s) in the virtual world using contextual information. Contextual information is represented by a state transition diagram, each state of which indicates possible gestures at the next moment. Image features obtained from extracted hand regions are used to judge state transition. When we use a gesture recognition system, we sometimes move our hands unintentionally. To solve this problem, the proposed system can recognize collaborative gestures with both hands. They are expressed in a single state so that the complexity in combination of gestures of each hand can be avoided. We have realized an experimental human interface system. Operational experiments show promising results.