Hand-tracked 3D data selection of point clouds in XR

Abstract

Three-dimensional data is crucial in fields like scientific visualization, medical imaging, and astronomy, yet effective 3D selection remains a challenging and unsolved problem. Traditional methods, primarily limited by 2D interfaces, often lack the precision and flexibility required for accurate 3D selection. Additionally, existing 3D techniques are limited to simple selection volumes and must be optimized for point clouds, further complicating the process. This project addresses these challenges by developing a GPU-based selection approach using Signed Distance Fields (SDFs). This approach enables efficient, real-time selection of near-arbitrary subsets of point clouds in XR environments. The solution is implemented in Unity and integrates novel and existing selection techniques, validated through a user experiment. Results revealed significant performance differences between selection techniques, task complexities, and point clouds. Participants favoured brushing modes over direct selection methods, a preference confirmed by quantitative data. This research bridges the gap between traditional 2D methods and XR, offering a robust and efficient approach to 3D data selection.