Sound is all around but is odourless, tasteless nor visible. Other senses are normally stimulated simultaneously however, with sound this can come separately and individually. We perceive auditory rhythm by detecting patterns of events in time (Dobrian,1993) Sound in todays digital environment is added to by the use of visual elements to further stimulate the audiences association to the rhythm and beat of the audio.
“A hierarchically ordered network of movements, incidents, episodes, epochs, eras. Time-spans, ‘boundaries’ are determined by the nature of the events or processes occurring within them. Similarly for the musician, a piece of music does not consist of time-spans whose perceptual boundaries are largely determined by the nature of the sounds and sound-configurations occurring within them. Time-spans immediately proceeding and following it. Such time-spans will here be called temporal gestalt-units” (Tenney and Polansky 1980, p.205)
As Tenney and Polansky describe the gestalt-units are the distinctions of time i.e. the rhythm or beats of audio, repartitions of similar sounds and similarities of pitches. All of these distinct units of time in a sound medium are similar to the features of Gestalt in a visual medium. Reybrouck in Leman suggest higher functions of the brain could be linked and use a similar classification in relation to sound as Gestalt does to arrangement.
“ The grouping phenomena of Gestalt psychology therefore can be formulated in terms of organizational principles, knowledge acquisition, categorization, schematization and abstraction, and this in a dynamic rather than a static way, since music is a dynamic system characterized by totality” (Reybrouck in Leman, 1997, 59)
These features of the brain are similar to that of Gestalt and work in the same fashion. It could be considered that this can there fore with the association between Gestalt and brain psychology of sound perception. Visual medium can be associated with that of the sound medium through the systems and principles of Gestalt.
In a complex media sound environment sounds that don’t fit together and will seem different. These distinct sounds can be distinct and but the relationship to the environment is key to the proximity and order of the environments sounds. For example a bird can chirp in a forest environment would be distinct but not unexplainable however, if a phone ring was herd that is be intangible and have a greater prominence.
This non conformity of the system of sounds within the environment also doesn’t conform to Gestalt rules of proximity and order as its protrudes itself as McGookin explains “The more similar (or different) two auditory sources are along a number of different dimensions, such as familiarity, continuation, and so on, the more likely it will be that they will be merged to be into a single sound (or will be separated and perceived as different sounds), that is, they will be placed in the same (or different) stream” (McGookin and Brewster,2004, 135).
In this example the stream is the environment of which the different sounds don’t conform. This in terms of media ambience and environment ambience its the irregular or mismatch of environment or context of the sound of which stands out to an audiences auditory system, of which is separated into meaningful representations of auditory perception. This auditory perception is perceived by the brain in relation to the Gestalts principles of Similarity, Continuation, Closure, Proximity, Figure/Ground, Symmetry and order
“in a collection of sound-elements, those which are simultaneous or contiguous will tend to form clusters, while relatively greater separations in time will produce segregation, other factors being equal” (Tenney and Polansky, 1980, 207).
This proximity as Tenney considers is the auditory version of the distance between shapes or objects. As he considers the further apart the sound elements due to time, note or real world relevance the more the feeling of separation on individuality. If a shape animation or motion graphic was considered as an example elements closer together may appear of lager significant. Darwin considers Gestalt Proximity in music as a process of “embedding intervals in a kind of auditory scene analysis; tones close in frequency will group together, so as to minimize frequency jumps. Sounds from different frequency registers are harder to group together across time than those from the same location” (Darwin, 2002).
Visual elements and shapes can be used to translate and visualise the motion and patterns of sound through the use of Gestalt principles, using a narrative in relation to time. This is considered as Motion Gestalt “The extension from two-dimensional considerations to time and motion moves to build a model of Motion Gestalt. It is proposed that the model of Motion Gestalt will account for varieties of motion understanding that can underpin and guide its application through visual communication design on screen.”(Jinsook 2007)
This similar to Gestalt Auditory has been given a set of principles of which makes the motion of visual elements of Gestalt foundation. In short, the following definitions were formed:
- Motion Proximity, which represents grouping by frequencies and/or regularities of movement,
- Motion Similarity, which represents a grouping by speed and/ or direction of movement,
- Motion Common-fate, which represents grouping by replications of movement,
- Motion Good-continuation, which represents grouping by syntax of motion behaviours of movement in the unfolding process,
- Motion Closure, which represents grouping by semantic summarizations of movement
(Kim and Poggenpohl, 2004, 2005).
This motion of imagery in a digital environment is defined as motion graphics. In reference to this a form of Gestalt perceived audio is used to interoperated the imager or the Motion Gestalt is used to illustrate the audio. This can be seen here in two examples.
- Audio is used to interoperated that of the Gestalt Imagery & Motion
In this example the use of movement of the visual imagery and symmetry of elements are the added and made more apparent by the use of audio. Its clear that this is so because the imagery and motion has more correlation to the use Gestalt with principles of which are clear to see. The audio on the other hand is spontaneous and irregular in the context of itself but, put together it makes more sense in a psychology perception of visual elements motion.
- Motion is used to illustrate the Gestalt Auditory
This on the other hand use the font and movement is used in relation to gestalt units of time. This unit of time is that of the spoken words which correspond to the imagery of which depict this. Unlike the previous example these elements of sound and imagery could work apart with however their connection give a far grater in depth and powerful meaning to both. It express and sinuates the audio more through the motion of words including how they move in relation to other elements.
In relation to my piece of installation which will present the idea of ambient sound of an environment using visual graphics and audio representation Gestalt will be key to the effectiveness of all elements of the design, motion and audio feedback. These will all have to similtaniously effect eachother creating an intertwined feedback of which can represent that of the changing enviroment that its in.
Dobrian, C., 1993. Music and Artificial Intelligence. [online] Music.arts.uci.edu. Available from: http://music.arts.uci.edu/dobrian/CD.music.ai.htm [Accessed 15 Jan. 2015].
Jinsook, K., 2007. Motion Gestalt For Screen Design: Applied Theroy of Grouping Principles for Visual Motion Integrity. Ph.D. Graduate College of the Illinois Institute of Technology.
Kim, H. and Francis, G., 2000. Perceived motion in complementary afterimages: verification of a neural network theory. Spatial Vision, 13 (1), 67-86.
Leman, M., 1997. Music, gestalt, and computing. Berlin: Springer.
McGookin, D. and Brewster, S., 2004. Understanding concurrent earcons: Applying Auditory Scene Analysis Principles to Concurrent Earcon Recognition. ACM Transactions on Applied Perceptions, 1 (2), 130-155.
Tenney, J. and Polansky, L., 1980. Temporal Gestalt Perception in Music. Journal of Music Theory, 24 (2), 205.