Evaluation and study of user interface with wearable devices based on computer vision : model based on VR keyboard and hand interaction

Abstract

This project was initiated with the goal of significantly enhancing the user experience in virtual reality (VR) environments through the development of an advanced mathematical model for camera-based fingertip tracking. The inception of this project involved an exhaustive literature review, which not only established the feasibility of the concept but also provided critical insights into existing methodologies and technological limitations. Building upon this foundational knowledge, the model was meticulously developed using MATLAB, a choice driven by its robust computational capabilities and suitability for algorithmic modeling.The core of the model centered around the precise measurement of distance between the camera and the fingertips, employing various triangular theorems to accurately triangulate the position of the fingertips in the virtual space. This innovative approach enabled the algorithm to effectively map and position fingertips in the VR environment, thereby creating a more interactive and immersive experience for users. The algorithm's efficiency in tracking and interpreting complex finger movements marked a notable advancement in the field of interactive VR technology. Despite its success, the project identified key limitations that posed challenges to its effectiveness. One of the primary issues was the resolution of the cameras used; the fidelity of fingertip tracking is heavily dependent on the camera's ability to capture fine details and subtle movements, which is constrained by its resolution. Additionally, the accuracy of the hand model used in the algorithm was a limiting factor. The model’s ability to precisely interpret and replicate finger movements was intrinsically tied to the realism and detail of the hand model. These limitations, while highlighting the current constraints of the technology, also pave the way for future research and development. Addressing these issues will be crucial in advancing the technology further, allowing for greater precision in tracking and an even more immersive VR experience. Future enhancements could involve integrating higher resolution cameras, refining the hand model for increased accuracy, and exploring the use of machine learning techniques to improve the algorithm's adaptability and responsiveness to a wider range of hand shapes and movements. The long-term implications of these improvements could be substantial, potentially revolutionizing the way we interact with virtual environments and expanding the applications of VR technology in various fields such as education, medicine, and entertainment.