Digital Dérive was a project developed for the SmartGeometry 2018: Machine Minds Conference which took place in Toronto, Canada. The project explores a novel method for reconstructing urban environments through the lens of human experience. Inspired by the Situationist International movement, which emphasized subjective urban experiences over purely physical ones, this project employs a low-cost 360-degree camera to capture video and audio during natural walks through city environments.

Using a custom workflow based on the open-source Structure from Motion (SfM) library, the data is processed to generate dense 3D point clouds. These point clouds are integrated with audio data to create an immersive virtual reality (VR) environment that allows users to experience urban spaces as perceived by individual occupants.

The innovative approach utilized in Digital Dérive questions the role of precision and fidelity in our digital representations of physical environments. Unlike traditional 3D models, which prioritize accuracy and completeness, this project embraces the "imperfections" that come with subjective experience. The reconstructed VR models, though less precise, capture more of the noise, imprecision, and uniqueness that characterize our real-world experiences. The result is an immersive yet "uncanny" environment that feels both more real and less structured than traditional VR spaces.

Methodology

The workflow begins with video capture using a head-mounted 360-degree camera, which allows for natural movement and minimizes user interference. The raw footage is processed using stitching software, and images are extracted to generate a 3D point cloud using SfM principles. The method also incorporates ambisonic audio to create a layered soundscape, further immersing users in the multisensory experience. The VR model allows users to explore the city from different perspectives, switching between top-down views and first-person immersion.

The raw camera footage is stitched into a 360 deg video and then split into four sets of directional images

The reconstruction process finds unique features in the images and then matches images based on similar features

Sparse reconstruction model visualized in COLMAP GUI

Comparing auto-generated depth and vector maps

Conclusion

This project represents a significant step forward in urban psychogeography, blending emerging technologies with a deeper understanding of how we experience cities. The final VR environment not only captures the visual and auditory elements of urban spaces but also highlights the subjective nature of human interaction with the built environment. As the project demonstrates, by focusing on individual perspectives, we can gain richer insights into the emotional and experiential dimensions of cities. Digital Dérive serves as both an artistic and technological exploration of how we can better understand the spaces we inhabit.

User orbiting and exploring the model (top) Still image of VR experience showing "first-person mode" (bottom)

Publication

This project was published in the proceedings of the ACADIA 2018 Conference in Mexico City, Mexico.

Digital Dérive. Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Mexico City, Mexico. (2018) [link] [pdf]