Picturing the ephemeral

an audio-visual syntax for visualizing the unspeakable in musical open notation

by Clémence Martel, Sylvain Reynal and Pierry Jacquillard

We propose a performance-installation that aims at exploring a new audio-visual syntax for rendering the subjectivity that pertains to interpretation choices in the framework of open music notations. With the emergence of Sprechgesang at the beginning of the 20th century and the increasingly widespread use of open notations after World War II, a space for freedom and personal embodiment never seen before has been offered to the performer, as is exemplified in the works of Cage, Berio, Globokar and Lachenmann. However, just because much more freedom has been given to the performer than was the case a century ago, does not mean the interpretation is a perfect mirror of the performer's free will [Castoriadis]. Those fragile and ephemeral variations from one performance to the next are indeed remnants of the unconscious of the subject. However, although these "actes manqués" may directly translate into acts that have all the features of genuine randomness, they differ from computational randomness --- as in pseudo-random number generators --- in their ability to capture and render an historicity, a singularity, an unthought-of of the subject [Reynal]. This hiatus between both perspectives on randomness is at the core of our proposal: what we aim at figuring here --- in a sort of reverse sonification [Casteloes] --- is the part in the interpretation which the performer has no conscious control on and that makes every performance unique, in spite of continuous attempts at over-controlling and hours of tedious work.

The installation relies on LED panels --- as used for massive advertisements --- to display the animations. Used as a primary scenographic light source, the device reacts to the voice of the singer, echoing the uniqueness of analog sound performance. As the performance unfolds, we carry out a moving audio signal analysis to detect interpretation fluctuations using a database fed with past performances. The device --  a visual counterpart of pitch-tracking algorithms developed at IRCAM in the 80-90's -- mostly focuses on texture variations, pitch, note lengths and dynamics,. Displayed is an abstract animation that coalesces shapes and colours taken from a dictionary of key-value pairs, where keys represent variational interpretation parameters and values are visual shapes. Simultaneously, a user manual containing a listing of the meaning of every shape and colour displayed on the screen is handed out to every participant. In this way, what is being displayed is a self-mediated visualisation --- although extremely complex, seeing the huge number of measured audio parameters --- of a unique fluctuation "screenshot" in the interpretation.

We aim at investigating how one can bodily reclaim digital devices in order to feed a real life experience, by diverting the emotional evocative power of these devices, from massive advertisement displays to algorithms that measures every part of our lives, by giving life to the unspeakable.