Features and Columns

Culture Warrior: Synesthesia at the Movies

By  · Published on August 24th, 2010

Synesthesia (syn-es-the-sia, Brit. syn-aes-the-sia): “The production of a sense impression relating to one sense or part of the body by stimulation of another sense or part of the body.”

Synesthesia is a neurological disorder in which the experience of one sense motivates an involuntary association with another sense. Those who experience synesthesia, known as synesthetes, are able to either perceive letters or numbers as inherently colored, hear movement, or – in probably the best-known cases of the disorder – see music in the form of colors and/or associative shapes.

Now, cognitive sciences seem, on the surface, to have little to do with the study of cinema, but the topic of synesthesia can be particularly helpful in understanding the way in which we interpret the interaction of the two senses most available in watching movies: the aural and the visual.

A Symphony for the Eyes

While cinema is predominantly referred to and conceived of as a visual medium, even since the silent era it has been, in fact, almost always audiovisual. But in watching films, the prioritization of our senses automatically privileges the images over the sound. The sound, in fact, seems to serve the image, and hardly the other way around: we see actors’ lips move, so we hear them speak in synchronization, and film music typically operates invisibly, seemingly existing only to serve in a way that accentuates the images and elicit corresponding emotion. In the most typical experiences of mainstream film, this interpretation is, for the most part, true. Even those of us who have seen a great many films struggle if we try to focus on the sound alone – the separation of sound and foley effects, the intonations of dialogue, the function of music in providing motif, emotion, and theme – while the image itself is readily available to (and thus, more easily interpretable for) our senses.

But there are some exceptions to this, and some cases in which film sound – and, for the purposes of this post, specifically film music – resides equally at the forefront of our senses alongside the image if not surpassing it. This operates most often, it seems, in cases where previously existing music is used (be it classical or pop) rather than the original score (as scores by film composers most often provide an undercurrent accentuating the image rather than act as a partner alongside it). In such cases, the images are often manifested from the music itself rather than using the music for the purpose of the image. Music in such cases acts for the filmmaker as the stimulation of an additional sense, it inspires the nature of the corresponding image. Think, for example, of Johann Strauss’ “Blue Danube” inspiring an elegant waltz between a spaceship and a docking ship in 2001: A Space Odyssey, or each and every sequence in Walt Disney’s Fantasia, as they provide something akin to a synesthetic symphony for the eyes rather than abide by music’s traditional filmic function; the images here are motivated, structured, and paced by the dictates of the specific piece of music.

A Cultural Definition

Rather than abiding strictly by the cognitive-scientific definition of the term, a looser perceptual understanding of synesthesia allows us to decipher how, as filmgoers, we all in one way or another become complicit synesthetes. What I’m concerned with here is a cultural definition of synesthesia, an understanding of how, through the conditioning processes of both watching and hearing films, we not only experience sound and image in unison (without one sense substituting for or privileged over another), but also experience the ways in which, through this juxtaposition, we associate specific sounds or pieces of music with their associative images as we’ve experienced through film. It is through this process that an image becomes forever associated with a sound within the vernacular of popular culture.

Try listening, for instance, to Wagner’s “Ride of the Valkyries” without thinking of helicopters dropping bombs on Vietnamese civilians in Apocalypse Now (or, for that matter, any parody of that sequence). Listen to Richard Strauss’ “Also Sprach Zarathustra” without your mind referring the opening and/or closing of 2001, or Paul Dukas’ “The Sorcerer’s Apprentice” without imagining Mickey Mouse summoning brooms to do his bidding in Fantasia, or even something as simple as an electronic five-tone musical procession on a synthesizer without thinking of humans attempting communication with aliens in Close Encounters of the Third Kind (the association of light and tone alone, utilized here (literally) as a universal form of communication in the film, is itself a perfect example of how synesthesia operates in an audiovisual artform).

Admittedly, there are many examples of this sound-image association within original scores for films (Close Encounters is a special example of this as it operates within the plot of the film (diegetic), but other John Williams/Steven Spielberg collaborations, like Jaws, contain many motifs that are near impossible to dissociate from their visual counterparts), but what’s unique about the use of existing classical or popular music in correspondence with moving images manufactured by the filmmaker is the birth of new sensory associations from a previously existing work of music.

Sure, several pieces of music already possess visual associations before being utilized in a film (an opera, ballet, or music video, for example), but the universality of cinema permits a rather fruitful avenue for new audiovisual associations, an artful and surprisingly fitting juxtaposition that combines the sounds and the image not simply into the intersecting art form of the audiovisual, but rather provides the impression of an ultimate, transcendent sensory experience altogether. The synesthetic use of music in film is not an intersection of the arts as much as it has the potential to be a gesamtkunstwerk, or an ambitious combination several forms of art and sensation, enabling an amalgamated sensory experience of gestalt in which the very combination of these senses creates a unique sensory experience that doesn’t reside in each form individually; the whole becomes greater than its parts.

This experience changes the cultural currency of both the images and the music used in their association with each other, thus altering the meaning of a piece of music which may have existed for centuries without a distinct visual counterpart and enabling a prolonged association with the images conjured by another artist with his or her own intent of meaning. (For example, Strauss’ “Zarathustra” existed for 72 years before being stamped in association with 2001. What results in not merely intents and structures of meaning and context of Strauss utilized for or integrated into intents and structures of meaning and context of Kubrick, but a combination of the two that manifests a third meaning separate from the intents of the individual artists.) Associating a specific image with a sound or piece of music forever alters its meaning and changes its context for understanding, and our ability to be conditioned to make such associations as viewers, especially as these associations are further legitimated within the larger cultural discourse and exchange of art, makes us all synesthetes in our consumption of cinema.

An Abridged History of Synesthetic Cinema

Associations between images and music have been attempted since experiments in the silent era. In works like Fernand Léger’s Ballet mecanique or Viking Eggeling’s Symphonie diagonale (both 1924), a non-narrative succession of geometric images is approached in terms of the musical art forms of the ballet and the symphony, thus attempting (even when viewed without added sound) to express music through images alone. René Clair’s famous Entr’acte (also 1924, the title referring to the interlude between two acts of a musical production), in its collaboration both on and off-screen with composer Erik Satie, is also a non-narrative association of images composed with intent in combination with sound, this time in the form of Satie’s original score for the film. The sound here neither motivates nor is motivated by the image, rather each work in accordance and combination with one another.

Experiments in synesthetic cinema took on a rather interesting term in 1940s and 1950s North American short filmmaking. German-American Oskar Fischinger’s Motion Painting No. 1 (1947) is the moving evolution of a painting as motivated by Bach’s Brandenberg Concerto No. 3. This and Fischinger’s subsequent motion paintings provide an interesting artistic intersection of paint, music, and cinema and manifested a unique work that could exist in neither individually. Canadian filmmaker Norman McLaren took a similar approach to Fischinger’s motion paintings, except the motivating music he used was jazz and the paint was directly applied to the 35mm film negative itself, like in his Begone Dull Care (1950). McLaren later experimented with a synesthetic approach to bodies and objects in motion also motivated by music, like the sitar-themed, whimsical A Chairy Tale (1950) or the hypnotic combination of ballet and the pan flute in Pas de Deux (1968). Additionally, previously existing popular music was used in American experimental film long before it became part and parcel of Hollywood filmmaking, and one of the better examples of this brand of juxtaposition is Kenneth Anger, whose Scorpio Rising (1963) and Rabbit’s Moon (1950/72/79) used popular radio-friendly doo-wop and rock tunes against their often challenging non-narratives.

While possessing its roots in experimental and avant-garde filmmaking, synesthetic cinema operates in many forms in Hollywood and experimental filmmaking, in part because of the more frequent use of previously existing classical and popular music in modern cinema. Stanley Kubrick’s films from 1962 to his death provide a wealth of examples of synesthetic associations between music and the moving image, as have unconventional nonverbal documentaries like Godfrey Reggio’s Koyaanisqatsi (1983) or Ron Fricke’s Baraka (1992).

Synesthetic appropriations popular music have been near-perfected by Martin Scorsese’s Anger-inspired sonic implementations (ex. the extended bar scene that shows Harvey Keitel’s drunken subjectivity in Mean Streets, or Scorsese’s associative ownership of The Rolling Stones’ “Gimme Shelter” in nearly any film he makes) following the frequent leitmotific pop-movie associations of New Hollywood cinema with films like The Graduate (1967, Simon & Garfunkel), Harold & Maude (1971, Cat Stevens), and Midnight Cowboy (1969, Harry Nilsson). These examples merely scratch the surface of the infinite number synesthetic associations of music and the moving image in modern cinema, and that purely synesthetic moments can suddenly show up in otherwise seemingly unremarkable films stands as evidence that synesthesia has fully permeated mainstream cinema and is here to stay.

Click here for more Culture Warrior

Related Topics: