Realism, Storytelling and User Experience in HMD-based eXtended Reality for Holocaust Museum

Due to the COVID-19 lockdowns and travel restrictions, the demand for remote museumvisiting experiences has increased. Fortunately, technologies like Head Mounted Display (HMD)-based Virtual Reality (VR) and Augmented Reality (AR) have made HMD-based eXtended Reality Museum (HXRM) experiences possible. HXRMs can be one of or a combination of the following: an HMD-based AR museum for on-site experience, or an HMDbased VR museum and an HMD-based Augmented Virtuality (AV) museum for remote online access. HXRM is a new approach for museums to enhance user experience while increasing learning outcomes and accessibility. Though there has been some previous research for HXRM, gaps still exist in the interactive narrative and user experience of HXRM. Thus, this study proposes following three Research Questions (RQ): (1) What is the difference between the impact of NUI and GUI on user experience in the HMD-based AR museum? (2) What is the user experience difference between HMD VR and HMD AV as the medium for XR remotesite museums? (3) How is the user acceptance of HMD-based remote-site XR museums?

Based the Technology Acceptance Model (TAM) and several user experience theories, the author proposed a user experience model for HXRM, an uncanny valley framework for realistic CG character, and an interactive narrative model. Then, in collaboration with National Holocaust Centre and Museum, The Extended Journey project was initiated. The project included an AR HoloLens application, The AR Journey, and a VR application, The Virtual Journey, that can be deployed on AR headsets like HoloLens and VR headsets like HTC Vive, respectively. The Extended Journey is an interactive narrative experience that presents the story of a fictional Jewish boy named Leo using virtual CG characters and environments, allowing the audience to participate in his story from the second-person-view. The audience can not only decide the direction of the storyline by helping Leo make choices, but they could also inspect the environments and objects within them to learn the stories behind them.

Three experiments were then conducted using The Extended Journey, and a mixed approach of quantitative and qualitative methods were used for analysis. In experiment 1, a between-subjects design was conducted to answer RQ1, and the results showed that the influence of interaction mapping on presences and narrative engagement for an HMD-based AR museum experience is moderated by prior game experience. In experiments 2 and 3, a between-subjects design and a within-subjects design were performed together to answer RQ 2 and RQ 3. The results showed that HMD VR can produce better narrative immersion, presence, and enjoyment, while also increasing CG characters’ affinities compared to HMD AV in XR remote-site museums. The data analysis also showed narrative-based HXRM had high user acceptance, within which HMD VR demonstrated significantly higher user acceptance levels than HMD AV for remote-site HXRM. Experiments 2 and 3 verified all the hypotheses for the mechanism behind the extended TAM via regression analysis, confirming the influence of the four external factors of narrative engagement, presence, interactivity, and CG characters’ affinity. In addition, the analysis also revealed two other potential external factors with influence over the extended TAM: use environment and device ergonomics. Two independent variables, learning interest and prior game experience, were found to have an impact on these external factors. Finally, the author summarised the design guidelines for HXRM and provide an outlook on the limitations and potential future work of this study.

Access to this this thesis is restricted. Please contact UAL Research Online for more information.