Cocktail Party

Virtual reality research project exploring the “cocktail party” phenomenon and audiovisual speech integration.

6 months

Languages: C#

Platforms: VR Headset

Technologies: Unity

Roles: Developer

Domains: VR

Description

This project was carried out as part of Raphaël Thézé's research (University of Geneva/Campus Biotech) on auditory and audiovisual perception of speech in complex environments. The aim was to create an immersive virtual reality environment to explore the “cocktail party” phenomenon, where the brain manages to focus on one voice among several sound sources. The device used the McGurk effect to enhance audiovisual integration and study how interactions between vision and hearing influence perception and selective attention.

Key features

  • Development of an immersive VR scene for studying the cocktail party phenomenon
  • Automation of lip-audio synchronization from vocal tracks
  • Multilingual audio-text alignment using SPPAS software
  • Generation and processing of sound stimuli (Google TTS, Audacity)
  • 3D modeling and animation of avatars (Blender, Mixamo, motion capture)
  • Implementation in Unity with SteamVR and LipSync Pro
  • Performance optimization and compatibility with the HTC Vive Pro headset and an EEG system

Feedback

This project was a formative experience combining 3D development, software engineering, and cognitive research. It allowed me to strengthen my skills in virtual reality and audio-visual processing, while discovering the specific challenges related to audio-visual synchronization and experimental rigor in scientific research. The interdisciplinary nature of the project, spanning computer science, neuroscience, and psychology, pushed me to adopt an analytical and adaptable approach. I learned to collaborate effectively with researchers, understand their experimental needs, and transform theoretical issues into concrete technical solutions. Beyond the technical aspect, this experience consolidated my interest in applied VR research and gave me a clear vision of the role that technology can play in the study of human perception and attention.