Enhancing interactive virtual environments for UR robotic arm testing and learning

Abstract

A digital twin (DT) is a virtual model of a physical system that enables real-time data exchange and interaction, facilitating precise, flexible, and efficient system control. In robotics, particularly in collaborative systems like the UR5e arm, digital twins have emerged as pivotal tools for improving real-time teleoperation and training. This paper evaluates the latency challenges in VR-driven DT frameworks using Unity and ROS 2 as the primary platforms for simulation and real-time communication. The system integrates MoveIt for motion planning and execution, utilizing Unity as a VR-based interface for controlling both virtual and physical UR5e robotic arms. After analyzing key latency metrics, it was found that the Unity and Gazebo simulations had an average response time delay of 0.49 seconds, with execution, planning, and subscription latency accounting for 0.45 seconds of the total delay. With a component delay of 0.45 seconds, the average response time delay in Unity-real robot interactions was 0.52 seconds. The results highlight how DTs can improve robotic operations' accuracy and efficiency while tackling latency, a significant problem. The study highlights the framework's applicability in teleoperation and training scenarios, setting the stage for future advancements in VR-robotics integration.