Interface design and interaction optimization for spatial computing 3D content creation and immersive environment generation using Apple Vision Pro

Traditional 3D content creation paradigms present significant barriers to meaningful creative expression in XR environments, limiting designers’ ability to iterate fluidly between conceptual thinking and spatial implementation. Current tools often disconnect the designer’s creative thought process from the immersive context where their work will be experienced, creating a gap between design intention and spatial realization. This disconnect particularly impacts the iterative cycles fundamental to effective design thinking, where creators need to rapidly externalize, test, and refine concepts within their intended spatial context. This research addresses the need for more intuitive, context-aware creation systems that support the iterative nature of creative cognition in immersive environments. We developed Dream Space, a spatial computing system that bridges this gap by enabling designers to think, create, and iterate directly within XR contexts. The system leverages generative AI for rapid prototyping of 3D content and environments, allowing designers to externalize and test creative concepts without breaking their cognitive flow. Through multimodal interaction design utilizing Vision Pro’s spatial computing capabilities, creators can manipulate virtual artifacts through natural gestures and gaze, supporting the fluid iteration cycles characteristic of established design thinking frameworks. A mixed-methods evaluation with 20 participants from diverse creative backgrounds demonstrated that spatial computing-based creation paradigms significantly reduce cognitive load in the design process. The system enabled even novice users to complete complex creative tasks within 20-30 minutes, with real-time feedback mechanisms supporting rapid iteration between ideation and implementation. Participants reported enhanced creative flow and reduced technical barriers compared to traditional 3D creation tools. This research contributes to understanding how XR interfaces can better support creative cognition and iterative design processes, offering insights for developing tools that enhance rather than hinder the natural flow of creative thinking in immersive environments.