Multi-camera finger tracking for 3D touch screen usability experiments

Abstract

Three-dimensional Human-Computer Interaction (HCI) has the potential to form the next generation of user interfaces and to replace the current 2D touch displays. The recent progress in computer vision and object tracking has made it possible to get rid of data gloves or other sensors that constrain user experience and to accurately measure hand movements using a camera-based system. The objective of this thesis was to develop a framework for recording human-computer interaction in 3D which is expected to form the basis for the future study of touch and gesture based user interfaces. The research included building of a multi-camera measurement setup, evaluating the eleven state-of-art object trackers for the task of sustainable finger tracking, selecting a post-processing method, combining data from a high-speed and regular frame rate cameras, and obtaining a 3D trajectory. The developed framework was successfully evaluated in the application where 3D touch screen usability is studied with 3D stimuli. The most sustainable performance was achieved by the SCT visual tracker complemented with the LOESS smoothing, and thus, this combination formed the core of the proposed framework.