Mixed Reality Laptops: A Protocol to Evaluate Enhanced Viewing Experiences on Consumer Laptops Using Parallax Engine

Research into Fish Tank Virtual Reality (FTVR) techniques commonly uses specific wearable sensors such as infrared cameras, glasses, or helmets to estimate the position of the user’s eyes. However, using only existing RGB cameras on laptops is more affordable for users as they are already widely available on devices. In this work, we continue and increase previous results using face feature detection software to implement the FTVR technique in 3D applications using standard laptops without needing extra devices. The main contribution is to enhanced the Parallax Engine solution to: 1)This work also defines and tests a protocol to evaluate the user experience and the sense of the presence of the Parallax Engine; 2) adding a new dimension in parallax mode (Parallax3DoF), with freedom of horizontal, vertical and depth movement with cameras in 3D scenes; 3) we eliminate the need for Python servers and the intermediation of network communication between Unity and these servers; and 4) improve integration, performance, and user experience of monoscopic FTVR mode (FishTank), with the virtual camera locked in the 3D environment of laptop screens. Parallax Engine uses Google Media Pipe for face mesh detection via an open-source implementation called the Media Pipe Plugin, available for Unity. We evaluated the proposed solution and almost doubled the FPS of the tool. In addition, we did not find fatigue problems, the interface was considered very good on the SUS scale, and the immersion was considered acceptable. We also compared the immersion and user experience of the FishTank and Parallax3DoF visualization modes, and the users evaluated better the Parallax3DoF in both. We aim in future work to explore this technique further and test it with more users.