jingyang (leo) liu

polyphony

Polyphony is an interactive audio-visual installation that establishes a dynamic feedback loop between performance, visual imagery, and sound, allowing for real-time engagement and creative exploration. The installation is built around two distinct yet complementary systems. The first employs the principles of cymatics to dematerialize physical media, enabling viewers to disrupt a field of voxels that abstractly simulate fluid dynamics. This visual disturbance, triggered by human motion, modulates pre-existing musical tones by altering their frequency and amplitude, thereby transforming visual inputs into auditory outputs. The second system adopts a bottom-up approach, utilizing a multi-agent swarm model in which the participant’s movements generate target points for agents to interact with. These interactions produce complex visual mappings that are intricately linked to generative soundscapes, creating a mysterious and emergent audio-visual experience.

Both interactive systems are designed to be open-ended, encouraging participants to create unique experiential pathways. The installation utilizes Kinect sensors to capture motion data, detecting and mapping body pixels in real-time. This data is processed through custom algorithms developed in Processing and SuperCollider, generating simultaneous graphical and audio outputs that respond directly to the viewer’s actions. Installed at Hartell Gallery, Polyphony transforms the space into a dynamic performance platform that invites public participation, stimulating social interaction and collective movement.

The project aims to challenge conventional understandings of public space by inviting visitors to co-create a constantly evolving sensory environment. Through their movements and interactions, participants shape an endless array of graphical and auditory outputs, making each experience distinct and personal. Polyphony blurs the boundaries between visual art, sound design, and interactive technology, ultimately questioning the relationship between human perception, digital media, and spatial dynamics.