Lighting people have been trying to interpret sound for decades. Lately, they’ve been getting better at it. Visual artist Quayola, working with record producer and remixer Jamie xx, developed a software language that “translates” music into what they call Structures — computer-generated artwork, projected onto a pair of 56-inch projection screens, that modulate to the real-time music signals input to their host computers and described by Wired UK as “a unique immersive experience.”
Partitura and Rynth
The software, dubbed Partitura, can interpret sounds and transform them into visuals. It can also be “played” like an instrument. If it sounds like the algorithmic-based music Visualizer that accompanies iTunes on your Mac laptop, it’s at least the same concept, though Quayola asserts that Partitura is significantly more precise and detailed.
Another British visual visionary, Paul Prudence, is also melding music and projected images. Created using a combination of coding and algorithms based on math and geometry to turn sound data into images in real time, Prudence’s latest project, Rynth, underscores the inventor’s intention that, while looking at static versions of the images is all well and good, the system was intended to be viewed in conjunction with live sound, whether it be music or noise. As Prudence explains (again, via Wired UK), “The high frequencies, for example…modulate the small squares which cover the surface of the image. The deeper drone sounds trigger the changes in the image’s structure…Because of how the algorithm works, each performance is unique: different sounds change the output.”
In fact, Rynth changes the relationship between projection lighting and sound by having the former determine the latter, rather than the other way around, as is typical at concerts. As Prudence puts it, “The sounds I use are informed by the way things look. What I do is, I have the software running, then I feed sound into it, and if I get an interesting result, I use that sound. Composition is informed by both looks as well as how it sounds.”
Liquid Light Shows
You can trace this evolving relationship back to the liquid light shows of the 1960s that formed the protean backdrops of shows at venues like the Fillmores East and West and the Electric Circus in New York City. Notable among them were the Joshua Light Show at the Fillmore East and Tony Martin’s displays at the Electric Circus on St. Mark’s Place.
Some of these shows could be as mobile as the layers of colored mineral oil and alcohol that were stimulated by the heat of the projection lamp to produce changing color patterns, like Headlights, Glen McKay’s invention that traveled with the Jefferson Airplane, and Mike Leonard’s light shows for Pink Floyd’s concerts. But you could also just as easily reference bored members of the AV squad in high school, messing with the overheard projectors used to explain algebra equations to mystified students.
The concept of pairing projection lighting with sound has been around for while. What’s changing is the sophistication of the technology and the fact that, increasingly, projection lighting and other media arts are themselves converging on digital platforms that change the relationship between them and dim their differences. (Prudence, by the way, used to perform his work in clubs, doubling as the DJ). And that convergence is creating new art forms in the process. Quayola revels in that ambiguity, saying, “The connection I’d like the audience to feel is…that they don’t know if the visuals are reacting to the sound, or the other way round.”
Commercial Applications
The potentially significant business applications of converged projection lighting and sound go beyond the world of experimental art. A more conservative but commercially proven application of converged image and sound is found in Atmosphere Media, a Germany-based company whose Atmosphere AV hardware and software uses four channels of HD video and audio on four separate portrait-oriented screens that can be occupied by virtual musicians performing on a stage to a synchronized AV immersive experience that sets moods around particular product or event. While not as unpredictable and interactive as, for example, Partitura, which is an artistic representation of voltage-based serendipity, Atmosphere is a commercial proposition from the start, but one that does it artfully, using what Atmosphere Media representative Jeff Van Duyn says is the first plug-and-play media server able to play simultaneously four full HD channels that are frame-accurate synchronized from an HDME quad server. And it has endurance: the server has a storage capacity of up to 40 hours of full multi-channel HD content encoded in H264 at 40MBit/s per channel, and it has flexibility — Atmosphere has been designed to be displayed on HD plasma or LCD displays, LED walls or high-performance HD projectors.
The number of companies specializing in large-scale image advertising and marketing projection onto buildings and other outdoor spaces also has steadily increased, and they’ve been adding audio to their repertoires. Tools like the ones mentioned here are convergent rather than simply additive — they underscore the notion that sound and lighting can be a unified proposition when they’re on a digital platform. We’ve already seen the convergence of lighting and video/graphical projection over the last decade, spurred by the uptake of the LED. Perhaps soon we’ll soon be watching additional hybrids of sound and illumination move from the artful to the commercial.
