Music is the most abstract art form. Its concrete value consists only of the impulsive compression of air. As opposed to painting, to poetry, to literature or to the performing arts music has no visual representation on itself.
      There are numerous attempts of adding a visual representation to music. The business of musical performance falls back on different forms of notation to support the memory with the service of the instrument. The relation of notation and musical sound has become dubious in the modern age and has been perfected at the same time more and more. Understanding sheet music is a lover's technology to aid in virtuoso reproduction of harmonious connections. For the purpose of mechanical reproduction of music the track of the sound is used. If it is in the musical clock, the barrel organ, the vinyl record or the CD - musical sound is fixed as its track to allow a mechanical apparatus the playback of music. The notation becomes the engraving in the material.
      The notation fixes audible streams for the purpose of the human or mechanical reproduction. The notation shows with it no artistic conversion of music for the purposes of a visualization. The notation is a means of the reproduction.
      In contrast to notation visualization takes the tone of the music as a base to provide illusionary scenarios which are linked with the music in a compelling and compassionate way.
      In this respect, different forms and degrees of audio-visual translations can be distinguished.

An instrument of mechanical representation is the oscilloscope. The oscilloscope takes up the sound as a regular oscillation and displays it as a curve form. No matter how much it appears to be a direct translation of the physical phenomenon, the visualization as a curve form also shows an interpretation. The musical phenomenon is reduced to its sinusoidal oscillation to be found again as the universality of the wave form. The music is an oscillation, the light is an oscillation - and all sensation is only a decoration of the synaptical impulses.
      E.F.F Chladni provided an inventive contribution to the physical interpretation and figuration of sound waves at the beginning of the 19th century. He strewed fine powder on metal plates. If a pure violin tone was played, the metal plates began to oscillate, and the powder formed up in regular geometrical shapes. The orbital shapes show a direct correspondence to the played tone. Provided that the same tone is played again, each time the same shape arises. The experiments of Chladni emphasize a principle which should be obeyed with the automatic visualization: the ability to reproduce a visual pattern by bringing in the same kind of acoustic impulses. Because the automatic visualization offers more possibilities of variation than the physical experiment, the results of a visualization can differ, even if the musical impulses are identical. The automatic visualization is steered by internal parameters which can produce a feedback between the visual and the acoustic domain. Also the temporal dimension contributes to the design, because what has already been shown is available as stored information to the current process and can be used as design parameter again. Not the precise form must be reproduceable, but the audio-visual tension expressed by the visualization.
      The analogy of music and shape is found even more clearly in the interpretation of the waveform and harmony as tone. The concept of waveform and harmony as tone gives way to the analogy of acoustical and optical phenomena. In synaesthesis the characteristic overtone is imagined as color tone. Still further the chromatics can be put in analogy: the scale of the spectral colors can be put in respect to the chromatics of the musical scale. So enticing these analogies may appear to the mind, so schematic they are, nevertheless, on the other hand. An exact correspondence solves the mystery of color sensation just as little as that of music psychology; instead, the open questions are expelled in the empire of the speculation about abstract patterns and correspondences of scale and tone. Only beyond the speculation around schematic solutions the relation of sound and color stimulates for creative contest.
      The light organ can be respected as a popular form of engineered music illumination. The light organ does not pretend to be a direct representation of the physical reality, but shows a free and entertaining interpretation of the tonal intensity. Often using the three primary colors red, green, and blue, the sound impulses are divided into frequency bands in an easy manner, so that a flickering arises if the music becomes dramatic, while in the quiet passages a mild glowing accompanies the music. Partially metrical regularities like the beat can be emphasized optically that way. The performing possibilities of the light organ reach further than those of the oscilloscope, although they are primitive. A part of the fascination about the light organ arises from the simplicity of its means.
      While the oscilloscope would like to eliminate the interpretation from the perception of the musical sound, the light organ stimulates interpretation. The light organ opens a window of opportunity for interpretation of the audio-visual communication of the vibrating and sensory elements. By watching the light impulses the connection to the music emerges over and over again in a new way.
      The most part of known forms of music visualization presumably acts between these two poles of representation.

As a prominent example of music visualization the video clip has evolved. Since its establishment as an independent TV channel and event stream at the end of the 80s meanwhile the music video clip has lost a lot of its appeal. The blueprints of innovative multimedia communication have been partially overtaken by a reality whose principles of efficiency degrade the icons of avant-garde to window-dressing. With their commercialization the broadcasters of music videos rashly submitted to the putative needs of primitive identification. In the beginning - and largely also even today - open questions about the connection between sound and patterns have given way to a culture of confirmation. Today the clips are introduced by presenters. At the beginning of music television the music clips were packed into a flow of pictures which was unpresented to a great extent and only was interrupted by skillfully produced short clips which varied the channel logo. The synaesthetic harmony of the pioneer's days no longer develops. Instead, the system of labor division provides for the fact that the music can be taken as a recognition pattern and as an entrance fee for cultural stimulation and distinction. Music is a ready-to-wear product, the pictures are bare colors of the labels.
      The music video clip relies on the tradition of the avant-garde. In the age of the machine the abstract combination of music, rhythm and picture appeared as vision (Entr'cte and Ballet Méchanique). As well as the camera could become the eye, and with it the stiff observer's perspective was dissolved, the young film put abstract pictures in respect with rhythm and sound (W. Ruttmann).
      On the other side the modern music video also refers to the physical translation of music in dance. The combination of music and dance is worked out in the ballet theater. Today the mainstreamed presentation of the music video offers less avant-garde interpretations of audio streams, rather it is a staged choreography.
      In the recent times the VJs see themselves as artistic successors of the DJ culture. While the DJ practice led to an enhanced concept of music, the VJs start with the multimedia experience of dance and trance events to alienate it, to extend it and to make it able to experience as itself. The VJs go back to the original fascination of the music video, by aiming synaesthesis and integration of music and picture (cf Eye Scratch in Neid magazine #4). The technologies of the montage, as they are practiced in modern music, should find their correspondence at the visual level.
      But also in contemporary music video production attempts with artistic claim can be found. So different musicians and image strategists like David Bowie, Pet Shop Boys, Kylie Minogue, Madonna or Missy Elliot link their music production with the production of visual art work, which applies from sleeve design about stage show set up to the music video clip. The visual art work is linked with the musical art work in a way which goes beyond the promotion of the product.
      While the creative interpretations of music are personally bound and are recognized as an expression of the artist's personality, new forms of visual expression arise in connection with the mechanical processing of music. With his concept of ambient Brian Eno has outlined an early understanding for the depersonalized sound experience.
      If one closes the eyes, pictures arise in front of the inner mind's eye. If one in addition at the same time listens to music, the pictures of the inner mind's eye are automatically associated with the music. With "Fantasia" in the 30s Walt Disney (together with Oskar Fischinger and others) has gone back to the tradition of the modern avant-garde and has delivered a popularized version of the "inner mind's eye vision". Other artistic musicial visions have been worked out in the area of the applied graphics arts in the form of the main title of film. With the main title design typography and moving text meet music. Maurice Binder (James Bond films) and Saul Bass (some films of Hitchcock and Scorsese) have produced in this genre artistic works whose flow shows the aesthetic blueprint for the algorithmic visualization of music.
      The visualization is intensified by the algorithmic interpretation of musical data. The visualization is no more personally bound, but also not bare display of measured data. Between music and presentation mediates a spiritual apparatus.

In the area of the psychoacoustics during the last years considerable efforts were started describing and understanding the music perception and sensation. Thereby the point of view of a deaf person is taken. How can music be explained to a person whose acoustic perception has dried up? Psychoacoustics deliver non-psychological paradigms with which every music experience categorially can be described. In the main issue psychoacoustics concentrate upon the metrics and the dynamic of music. The spectrogram is no more respected as a result of an analysis, but as a starting point for further questions.
      The music in itself is a phantasma which is too amorphous in its original form to be perceived as a mental experience. Perception of music means the production of patterns and groupings out of the audible. The music originates only by structuring the audible as sound and construction. Frequency, melody, harmony, dynamics and also cultural genre are categorial concepts which develop in the human mind successively in the course of the listening experience. The temporal component of the music is essential for the cognitive capture of audiostreams.
      Psychoacoustic science uses the advanced means of algorithmic logic to reconstruct in auditory scene analysis the mechanical processes of music perception. The human perception is understood as a conditioned process, whose conditions to question or to dissolve is not seen as a task of the discipline, instead, the mechanical and cognitive production processes are made usable for the algorithmic analysis and reproduction.
      In the common music perception time-bound impulses are translated into high-quality information. The stream of acoustic events is bundled and is perceived as rhythm, tempo, timbre etc. Beside the technical attributes psychological attributes are applied to the music: wild, quiet, demanding, begging etc. The psychological description of music can itself be respected as a high-quality categorization of music, out of the combination of the "hard" data (rhythm, dynamics, pitch etc.). The "hard" data is classified in features and grouped in feature sets to earn from it the high-quality description models.
      The psychoacoustics try to explain the listening experience to a machine. One hears at first with the ear and only secondly with the help of the whole neural system. The mechanics of the ear can be reconstructed algorithmically. The tiny hairs of the inner ear offer, on account of their mechanics, a certain sluggishness of the perception which can be observed, for instance, by the perception of frequency differences. Thus we can perceive pitch differences in the deeper area by far better than in the high frequency range. Also the volume sensation does not run linear but logarithmically. While the physical pressure intensity of audioevents is measured in Pascal or also decibel, different other scales were invented for the psychologically adequate description, like phone or sone. The intensity of the pressure is not described linear, but with the degrees of the felt loudness, because the sensitivity rises in the loud area. Besides, the loudness sensation is different with different frequencies.
      If one collects the features which can be extracted from an audiostream with the means of psychoacoustic, algorithmic intelligence, the high-quality characteristics of the music can be evaluated without the aid of manually collected meta data.

At present we are able to generate moving pictures with the computer, engineered in parallel to real-time processed music, which are no more statically bound to an artistic blueprint, but are generated anew with each call and still at the same time protect the association with the music.
      The computer-generated picture worlds have an iconography on their own which still is often, on account of the young age of the technology, narrowly bound to their engineering production process. Thus in the 3D graphics arts there are certain patterns like image warping, particle systems or plasma tunneling which can be transformed to picture sequences with the help of proofed methods. A big part of the screen savers in circulation helps itself of the reservoir of canonized methods to generate moving pictures.
      The hard- and software conditions for music visualization have reached since the middle of the 90s a state which enables to process audiodata on consumer devices in parallel to the picture production. One of the first software-engineered applications is the program "Cthugha" which presented itself the epithet "an Oscilloscope on Acid". The software drew fire colored surfaces in the 256 colors of the screen in fluent speed. The music data was delivered by the sound card of the computer which could play back commercial music CDs for the first time then. The connection between audio and video was produced in a pragmatic way: the sinusoidal form of the currently played frequency wave was taken as an input for the production of the picture forms. A pragmatic, but also mechanical solution which is often used even today. In this way an obvious connection at a dynamic level can be produced with easy means. Also configuration files were integrated into the software; the files were read in by the start of the program and were used for the parametric and alleatoric styling of the screen action. Also this principle is in circulation even today with music visualization software.
      The compression of audiodata by the end of the 90s led to the fact that large audio collections can be put into archives on hard disk. Different jukebox players exist for the management of the collections. WinAmp is one of the most popular programs on the Windows platform. It offers via an openly declared plug-in interface to third parties the possibility to access the processed music data and to colorize the screen. With the interface it lies in the responsibility of the developers of plug-ins in which way the visual level reacts to the provided audiodata. With the definition of the plug-in interface a clear separation was found between the digital signal processing of the player software and the visualization of the plug-in. By the clear separation the relation between the audiodata and the visualization could become an independent subject of the visualization. One of the most original plug-ins for WinAmp is "Geiss", which uses advanced transformation matrix calculations for the production of the visual effects.
      On the Macintosh platform SoundJam evolved as a player software which provided an interface for plug-ins. Andy O'Meara later has disclosed the source files for his visualizations "WhiteCap" and "G-Force". When Apple computer in the beginning of 2001 brought out the jukebox software iTunes for MacOS, the default visualizer showed the forms introduced by "G-Force" on the screen. Apple published an interface definition for the integration of visual plug-ins whose concept followed the defaults of G-Force and WinAmp.
      Since the end of 2003 an iTunes counterpart exists for the Windows platform whose cross-platform interface for plug-ins demands the effect programmer only to exchange a CGrafPort for a HWND. Possibly with it iTunes presents itself as an attractive player for music visualization also for the Windows platform, because WinAmp proceded a version upgrade which decreased the quality of the software, and WindowsMediaPlayer or the player of RealNetworks are judged as instruments of dominance and commerce restriction.
      Today the visualization of music can be performed by every (multimedia) PC. Algorithmic picture production that uses the parameters of the psychoacoustics with the translation of the music in visual space can challenge the watching mind in a special way. Because the psychoacoustic rules of music perception can be built in the algorithmic set of rules, one will perceive in the pictures not only an effigy of the music, but also a mirror world in which the active interpretation is a part of the performance.
      However, it is most important that the visualizers deliver original contributions to the music, indifferently of which intermediational techniques they use for their picture-generating algorithms. With their reference on the music, visualizers must distinguish themselves from screen savers like from oscillographs. Then visualizers once could possibly show up as a software genre on their own.

Writings