Into Music

When sound turns into image – uncovering the visual beauty of classical music

INTO MUSIC invites you into a virtual concert hall with sparking timpani and trumpets. The 360° visualization of George Bizet’s “Les Toréadors” combines the spatial listening pleasure of classical music with its simultaneous translation into dynamic sound images. In two different worlds of experience, visitors can enjoy the audiovisual abundance of the Carmen Overture and explore it for themselves.

The recording by the Symphony Orchestra of the Central German Broadcasting Corporation (MDR) provides thirty audio tracks in which the individual instruments stand out dominantly. Height, volume, tempo, and intensity are the parameters used for their visual translation. Fused in virtual space, along with Bizet’s composition they form two different sound patterns, depending on the mode.

In the connoisseur mode, the visitor finds himself in the midst of a frenetic firework of music. The explorer mode, on the other hand, takes the visitor into a sound garden of stalagmite-like plants. They sprout up on sound tracks stored next to each other and shine in a dazzling bright light. Using hand gestures the individual audio tracks can be isolated, or the entire piece can be fast-forwarded and rewound. Short applause is enough to swing back and forth between the two modes.

The mobile set-up consists of a VR headset and high-quality headphones. It is supplemented by a depth camera with a sensor for digital recording of the hands, as natural gestures replace the controller. Finally, Spatial Audio traces the visitor’s movement in the virtual setting and simulates his spatial distance to the individual audio sources.

The result of a cooperation with Drefabrik and Prof. Anette Scholz from Burg Giebichenstein Kunsthochschule Halle, INTO MUSIC, opens up new fields of application for VR concepts. The pilot project is proving to be a promising intermezzo between analog and digital cultural enjoyment, which should promote the generational change of concert visitors, but above all intensifies and enriches the sensory perception of classical music.