===       ===     =============        ====
             ===       ===           ==            ==   ==
            == ==    ====           ==           ==      =
           ==   ==== ===           ==           ==      ==
          ==     ==  ==           ==            =      ==
         ==         ==           ==             ==   == 
        ==         ==           ==               ====
       M U S I C          T H E O R Y         O N L I N E
                     A Publication of the
                   Society for Music Theory
          Copyright (c) 1995 Society for Music Theory
+-------------------------------------------------------------+
| Volume 1, Number 3        May, 1995      ISSN:  1067-3040   |
+-------------------------------------------------------------+
  All queries to: mto-editor@boethius.music.ucsb.edu or to
                  mto-manager@boethius.music.ucsb.edu
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
AUTHOR:  Smoliar, Stephen W.
TITLE:  Book Review:  Robert Cogan, *New Images of Musical Sound*, 
Harvard University Press, 1984.
KEYWORDS:  Fourier analysis, music analysis, phonology, sonology, 
auditory perception
Stephen W. Smoliar
National University of Singapore
Institute of Systems Science
Heng Mui Keng Terrace
Kent Ridge, SINGAPORE  0511
smoliar@iss.nus.sg
ABSTRACT:  While Robert Cogan's *New Images of Musical Sound* is 
now over ten years old, the applicability of Fourier analysis as a 
basis for music theory is still a relevant issue.  This review 
attempts to put Cogan's work into both a technical perspective, 
regarding what sound analysis technology now supports, and a 
theoretical one, regarding his attempt to build a theory on the 
foundations of phonology.  From a technical point of view, it is 
now far easier to approach analysis as Cogan has done;  but his 
attempt to build a theory suffers from some significant 
weakenesses.
ACCOMPANYING FILES:  mto.95.1.3.smoliar1.gif
                     mto.95.1.3.smoliar2.gif      
THE NEED TO LOOK AT SOUNDS
[1] This book is now over ten years old.  However, a recent discussion
thread on mto-talk was addressing the applicability of Fourier
analysis as a basis for music theory; and, in the course of this
discussion, David Lewin was kind enough to observe that Robert Cogan
had already discussed some of these issues.  It therefore seemed
appropriate to return to this book and see how it has withstood the
past decade.
[2] Cogan's primary interest is in music analysis.  Given how much
disagreement there has been over just what music analysis is all
about, he clarifies his own position in his final chapter: "To analyze
is to create a map or model--a model that reveals certain functions
and relationships" (p. 153).  He then develops this point along a line
very similar to one which John Roeder recently presented in *Music
Theory Online* (1): "A model itself can be verbal, numerical, or
graphic, and the preceding chapters have employed (to varying degrees)
all of these means: commentaries, tables of oppositions, and spectral
photos" (Cogan, p. 153).  Roeder's own interpretation of the graphic
had more to do with the use of diagrams to explicate mathematical
relationships (2); and, of course, just about every form of music
notation is also graphic.  However, Cogan's approach is far more
direct: He is interested in graphics to the extent that they satisfy
his need to *look at sounds*, and his thesis is that this need may be
best satisfied by spectrographic traces of those sounds.  In this way
what the eye sees in such traces can supplement what the ear hears,
perhaps even informing the mind of structural details which are not
immediately apparent in the course of listening.
================================================
1.  Roeder, J.  1993.  "Toward a Semiotic Evaluation of Music 
Analysis," *Music Theory Online* 0.5:  4.
2.  Ibid, 8.
================================================
[3] Before examining the specifics of this approach, it is worth
reviewing some of its general virtues.  Most important is what may be
called Cogan's attempt to "conquer time."  Sound cannot exist without
the passage of time; but, during that passage, the sound goes as soon
as it comes, so to speak.  An instant cannot be scrutinized because,
once scrutiny begins, the instant is gone.  Cogan's images, on the
other hand, are *traces* which remain in the present long after the
sound has faded into the past; and, unlike the sounds themselves,
those traces *can* be scrutinized in as much or as little time as the
mind chooses to allocate.  Music notation, of course, has the same
advantage of timelessness; but notation is, at best, a *prescription*
for sound.  Cogan has attempted, for purposes of *description*, to
capture *the sound itself*.
[4] Another advantage to Cogan's approach is its scalability.  By
suitably compressing the time scale, one can take in the entirety of
any sound event in a single glance.  Of course if that event happens
to be all of *Goetterdaemmerung*, that single glance is not likely to
take in very much detail; but one can then adjust the time scale in
order to examine greater detail.  In other words if the data are
appropriately captured, the observer can control what that single
glance encompasses, moving between coarse and fine detail at will.
The variable time scale provides an *implicitly hierarchical* view of
the sounds of any musical experience, which gets away entirely from the
symbolic representations of hierarchy found in the approaches of
Heinrich Schenker (3) and Eugene Narmour (4).
==============================================
3.  Schenker, H.  1956.  *Der Freie Satz*, O. Jonas, editor, 
Vienna:  Universal Edition.
4.  Narmour, E.  1977.  *Beyond Schenkerism:  The Need for 
Alternatives in Music Analysis*.  Chicago:  The University of 
Chicago Press.
==============================================
[5] However, spectrographic traces are only one of many ways in which
sound events may be represented visually (5).  There is also the
waveform itself: a display of how, for example, a loudspeaker cone
physically vibrates with the passage of time.  This is again a display
with the advantage of scalability; but, in this case, we have to be
more careful about the scales at which we choose to examine the
signal.  When we go down to the millisecond level, we can see the
actual periodic waveforms associated with isolated pitches.  However,
the *shape* of such a waveform depends not only on its *harmonic*
content (as revealed by a spectrogram) but also on the *phase*
associated with each of the component frequencies.  The problem is
that significant differences in phase do not necessarily imply
differences in what is heard (6).  Figure 1 illustrates two waveforms,
each of which is represented by a single cycle.  Both waveforms have
the same harmonic content in identical proportions; but, in the second
waveform, the first and second harmonics are cosines, rather than
sines, which means they have been phase-shifted by 90 degrees.
However, in spite of the obvious differences in appearances, these
waveforms are indistinguishable to the ear.
==============================================
5.  Aigrain, P., *et al.*  1995.  Representation-Based User 
Interfaces for the Audiovisual Library of the Year 2000, 
*Proceedings:  Multimedia Computing and Networking 1995*, A. A. 
Rodriguez and J. Maitan, editors, SPIE, pp. 35-45.
6.  Risset, J.-C.  1991.  Timbre Analysis by Synthesis:  
Representations, Imitations, and Variants for Musical Composition, 
*Representations of Musical Signals*, G. DePoli, A. Piccialli, and 
C. Roads, editors, Cambridge:  The MIT Press, pp. 7-43.
==============================================
[6] On the other hand if we view these waveforms at a *macroscopic*,
rather than *microscopic*, level, they have at least the potential of
being more informative.  Figure 2, for example, is the entirety of the
Aloys Kontarsky recording of Karlheinz Stockhausen's "Klavierstueck
III" (7).  This display tells us nothing about notes or the serial
structure of the pitches, but it *does* show how those notes are
grouped into *gestures* and how the intensities of those gestures are
modulated.
==============================================
7.  The sound was digitized from the vinyl Columbia recording, 32 
31 0008;  these recording sessions were supervised by Stockhausen.
==============================================
[7] Before examining *any* such visual approach in greater detail, 
however, we should also be sober enough to recognize that 
*biological* support is not encouraging.  One thing we know for 
certain is that the path from ear to brain is decidedly different 
from that from eye to brain (8).  Thus, if the biological 
substrates differ, it is unlikely that principles which dictate 
how we identify objects and structures in what we see are 
necessarily going to carry over into what we hear.  Furthermore, 
much of this distinction has to do with the necessity to account 
for time in auditory perception.  It is all very well and good to 
synthesize visual traces which "freeze" the passage of time;  but 
once the stimuli are "frozen," they can no longer be auditory.  If 
time "stops" then so do the stimuli;  and any attempt to abstract 
away from the passage of time runs the risk of also abstracting 
away certain attributes and relations which may be most critical 
to how those stimuli are perceived and interpreted.
==============================================
8.  Gibson, J. J.  1983.  *The Senses Considered as Perceptual 
Systems*.  Westport:  Greenwood Press.
==============================================
COGAN'S APPROACH
[8] Probably the element which has changed the most in the ten 
years since Cogan published these results has been the supporting 
technology.  Cogan's data were collected during the years 1980 and 
1981 in the Sonic Analysis Laboratory at the New England 
Conservatory.  The very first sentence in the book acknowledges 
the support of Dale Teaney and Charles Potter from the IBM Watson 
Research Center:  "Without their professional and personal 
initiatives, the process would have become available to musicians 
much later than it did"  (p. v).
[9] Given the extent to which computers have become part of our 
day-to-day lives, it is hard to believe that, when these data were 
being collected, personal computing did not yet exist.  Much of 
what we now take for granted could not even be imagined in 1980, 
making it somewhat difficult to infer from the description in 
Appendix A just what instrumentation Cogan actually used to arrive 
at the spectrum photos which lie at the heart of this book.  He 
*does* tell us that the "spectrum analyzer was a thirty-three-
millisecond fast Fourier transform instrument, capable of 
analyzing sounds in five continguous octave registers 
simultaneously" (p. 155);  so we can conclude that at least *some* 
of the equipment was digital.  On the other hand the images appear 
as if they were photographed from a rather conventional (and 
probably temperamental) analog oscilloscope;  and there is no 
doubt that he had to use photography (rather than, for example, 
laser printing) to capture those images (not to mention physical 
acts of cutting and pasting his photographs, rather than composing 
his images with software assistance).  However, there are also 
references to dynamic controls which could be adjusted, with 
little reference to what exactly is being controlled.
[10] One thing is certain:  Anyone interested in undertaking a 
similar project today is going to have a far easier time of it.  
These days it is pretty difficult to find a personal computer 
which *lacks* some form of audio input, not to mention direct 
capture of data from audio compact disc recordings.  It is quite 
likely that Cogan could now carry all the examples from his book 
around in a laptop computer, examining and listening to his data 
while on a flight surrounded by other users who are buried deep in 
their spreadsheets and games (all of whom, collectively, are 
probably radiating enough frequency to give the pilot a whole new 
patch of gray hairs).  Put another way, this is no longer a big 
deal, which means that, putting a more positive slope on the 
situation, it is now the sort of thing we can expect any 
resourceful undergraduate to do.
[11] To be a bit more fair, all this means is that *collecting* 
data is no big deal.  The "real deal" is what happens next.  
Therefore, I would like to be relatively brief in reviewing the 
data which Cogan actually collected and focus more attention on 
how he then undertook to develop a theory from them.
THE EXAMPLES
[12] Part I of the book is essentially an exposition of seventeen 
spectrum photos.  These are collected into four chapters:  Voices, 
Instruments, Large Mixed Ensembles, and Electronic and Tape Music.  
Each chapter, in turn, attempts to explore considerable variety in 
its subject matter.  Thus, the Voices chapter covers Gregorian 
chant, Tibetan Tantric chant, Billie Holiday singing "Strange 
Fruit," and Gyorgy Ligeti's "Lux Aeterna."  The instrumental 
examples include a Balinese gamelan, a Ludwig van Beethoven piano 
sonata recorded on both a forte-piano and a modern instrument, the 
second of Igor Stravinsky's "Three Pieces for String Quartet," the 
latter two of Anton Webern's Opus 7 ("Four Pieces for Violin and 
Piano"), and the third etude from Elliott Carter's "Eight Etudes 
and a Fantasy" (the only example of winds).  The mixed ensembles 
include both instruments and voices.  The first two examples 
compare the "Confutatis" from Wolfgang Amadeus Mozart's *Requiem* 
with the "Tibi Omnes Angeli" from Hector Berlioz' *Te Deum*.  This 
is followed by the first of Claude Debussy's orchestral 
"Nocturnes," the brief orchestral interlude which follows Marie's 
murder in Alban Berg's *Wozzeck*, and Edgard Varese's 
*Hyperprism*.  In contrast the final chapter is rather sparse in 
its examples:  the Introduction to Milton Babbitt's "Ensembles for 
Synthesizer," the "Fall" movement from Jean-Claude Risset's 
*Little Boy* suite, and Cogan's own "No Attack of Organic Metals."
Nevertheless, the overall variety of the entire collection is 
quite satisfying;  and it is gratifying to see that Cogan 
deliberately avoided concentrating on the music of Dead White 
European Males.
THE THEORY
[13] What is less satisfying about Part I of the book is that it 
is primarily anecdotal.  As one proceeds through the images, one 
gets the feeling that Cogan is saying, "Here is something 
interesting.  Here is something else interesting.  Here is yet 
another interesting thing."  After a while, even the most generous 
reader is likely to erupt:  "YES!  I agree!  There *are* 
interesting things here!  Do you also have an interesting 
*theory*?"
[14] Cogan attempts to confront this question with Part II of his 
book.  This is a decidedly shorter part of the entire volume;  
and, unfortunately, it is also noticeably weaker.  Some of its 
weaknesses may be more apparent because of what we have learned 
over the last ten years;  but, even in his own time, Cogan was 
failing to do justice to his subject matter in several significant 
ways.
[15] In order to develop his theory, Cogan turns to *phonology*, 
that division of linguistics which is concerned with the *sounds* 
of language, as opposed to either syntax or semantics.  This was 
not, even in its own time, a particularly new idea.  Gottfried-
Michael Koenig had organized an Institute for Sonology at the 
University of Utrecht in a deliberate attempt to generalize 
phonological theory to encompass other sources of sound (such as 
experiments in electronic and computer music which particularly 
interested Koenig);  and some of the earliest research concerned 
with developing a sonological theory was undertaken by Otto Laske 
(9).  It is therefore unfortunate that Cogan gives no indication 
of either Koenig or Laske, either the theory or practice of their 
work, or even the *idea* of generalizing from phonology to 
sonology.  Instead, his primary foundations seem to rest on the 
work of Roman Jakobson (10) and N. S. Trubetzkoy (11).  There 
seems to be little acknowledgment that there might be any *other* 
foundations, such as those which were pursued by Morris Halle and 
Noam Chomsky (12);  so the reader is left with the uneasy feeling 
that Cogan decided to pursue this particular approach because it 
looked good at the time.
============================================
9.  Laske, O.  1972.  "On Problems of a Performance Model for 
Music" (Technical Report, Institute of Sonology, Utrecht State 
University):  29.  For a critical review of this work, see 
Smoliar, S. W.  1976.  "Music Programs:  An Approach to Music 
Theory Through Computational Linguistics," Journal of Music Theory 
20.1:  105-131.
10.  Jakobson, R., and Waugh, L.  1979.  *The Sound Shape of 
Language*.  Bloomington:  Indiana University Press.
11.  Trubetzkoy, N. S.  1969.  *Principles of Phonology*, C. A. M. 
Baltaxe, translator, Berkeley:  University of California Press.
12.  Chomsky, N., and Halle, M.  1968.  *The Sound Pattern of 
English*.  New York:  Harper & Row.
============================================
[16] What interested Cogan most was Jakobson's desire to describe 
sounds in terms of *oppositions*.  This amounts to describing the 
quality of a sound in terms of where it is situated between two 
contrasting extremes.  For example, the grave/acute opposition 
distinguishes between concentration of sounds in low and high 
frequencies, respectively.  On the other hand the centered/extreme 
opposition addresses whether there is frequency activity at the 
extremities of the spectral range or only in some more limited 
middle range.  Cogan presents thirteen of these oppositions;  and, 
while he is very up front about the fact that this list may not be 
complete, the idea of building such a list in the first place lies 
at the heart of his theory.
[17] This approach is not unfamiliar in the technology of signal 
analysis.  However, what Jakobson called *oppositions* are now 
more commonly known as *features*;  and, when several of these are 
collected together, the results are called vectors in a *feature 
space* (13).  Feature vectors have become invaluable tools for the 
description and recognition of both visual and auditory signals;  
but, like most other tools, they religiously follow the GIGO 
(Garbage-In-Garbage-Out) Principle.  If they are not used 
properly, they are likely to yield results which are, at best, 
questionable.  Therefore, it is important to review some questions 
which need to be asked before feature vectors are invoked as a 
descriptive tool.
===============================================
13.  Gianotti, C.  1993.  Analysis of Economic and Business 
Information, *Handbook of Pattern Recognition and Computer 
Vision*, C. H. Chen, L. F. Pau, and P. S. P. Wang, editors, 
Singapore:  World Scientific, pp. 569-594.
===============================================
[18] Perhaps the most important question to be asked of any 
feature is:  *Can it be effectively calculated?*  This question 
has been deliberately formulated using the language of Alonzo 
Church (14);  so a positive answer implies that there is some 
*effective algorithm* which may be applied to the input, whether 
from spectra or waveforms, which will assign a value for that 
feature in a reproducible manner.  Cogan is quick to point out 
that his feature values are context-dependent;  but that does not 
preclude their being effectively computed.  Computation may just 
have to look at a broader span of time than, say, thirty-three 
milliseconds.  However, while Cogan is certainly *trying* to be 
specific in describing his features, effective computation appears 
to be beyond his scope.
===============================================
14.  Church A.  1965.  An Unsolvable Problem of Elementary Number 
Theory, *The Undecidable:  Basic Papers On Undecidable 
Propositions, Unsolvable Problems And Computable Functions*, M. 
Davis, editor, Hewlett:  Raven Press, pp. 88-107.
===============================================
[19] The opposite question is:  *What can be reconstructed from a 
feature vector?*  One of the interesting things about the spectrum 
is that it can be used to reconstruct the original sound (15).  
Given a feature vector, can it be used to construct *any* sound?  
The context-dependent nature of Cogan's approach to description 
definitely works against him on this count.  However, even if his 
value assignments were *less* context-dependent, it is unclear 
that one could ever look at a feature vector and hear anything 
remotely relevant in "the mind's ear."
===============================================
15.  Butler, D.  1992.  *The Musician's Guide to Perception and 
Cognition*.  New York:  Schirmer:  208.
===============================================
[20] This leads to yet another question:  *How do we compare 
feature vectors for similarities?*  This is a familiar problem in 
visual processing.  While any color can be easily described as a 
weighted sum of red, green, and blue sources of light, colors 
which are visually similar do not always have similar weight 
contributions.  Color similarity is usually better represented in 
terms of a different set of descriptors, commonly called 
luminance, hue, and saturation (16).  The two sets of descriptive 
vectors are equally effective in reconstructing a color, but the 
second representation facilitates identifying colors which are 
*perceptually similar*.  Thus, even if we have an effective set of 
features for describing sounds, until we know how to relate those 
features to auditory perception, we have little more than a 
mathematical abstraction.
===============================================
16.  Luong, Q.-T.  1993.  Color in Computer Vision, *Handbook of 
Pattern Recognition and Computer Vision*, C. H. Chen, L. F. Pau, 
and P. S. P. Wang, editors, Singapore:  World Scientific, pp. 311-
368.
===============================================
[21] The moral of the story is that the path Cogan has chosen is 
probably heading in the right direction, but he has not yet 
properly equipped himself for the journey.  Fortunately, our 
understanding of auditory perception has come a long way since 
Cogan's book appeared (17).  If Cogan is no longer interested in 
the trip, others can look where his finger is pointing and set off 
on their own.  Thus, the weakness of the theoretical portion of 
this book should not be seen as a condemnation of Cogan's approach 
to collecting data but as an incentive to apply our increased 
knowledge of both signal analysis and perception to go forth and 
do a better job.
===============================================
17.  One may gain a good appreciation of how long we have come 
from Butler, op. cit.
===============================================
CONCLUSIONS
[22] Is the trip going to be worth making, bearing in mind, for 
example, the biological conflict between what we hear and what we 
see?  If the purpose of the trip is to try to reduce all that is 
auditory to the visual, then, most likely, the trip will be doomed 
to failure.  However, as Cogan observed, images are but one of 
many approaches to description;  and no one approach will ever do 
all the work (18).  So we should not expect images of either 
spectra or waveforms to yield *all* the secrets of any musical 
experiences.  This should not be the purpose of the trip.  
Instead, one should undertake the trip to learn, in more specific 
ways, what these image data both *can* and *cannot* tell us.  If 
we undertake this task seriously, we are likely to find that those 
data can, indeed, tell us things which cannot be readily 
accommodated, if at all, by other modes of description.  Such a 
discovery will leave us better equipped than ever for future 
analyses of music and a firmer sense of the capabilities of music 
theory.
===============================================
18.  Smoliar, S.  1994.  "Comment on John Roeder's Article," *Music 
Theory Online* 0.6:  7.
===============================================


+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
Copyright Statement
[1] *Music Theory Online* (MTO) as a whole is Copyright (c) 1995,
all rights reserved, by the Society for Music Theory, which is
the owner of the journal.  Copyrights for individual items 
published in (MTO) are held by their authors.  Items appearing in 
MTO may be saved and stored in electronic or paper form, and may be 
shared among individuals for purposes of scholarly research or 
discussion, but may *not* be republished in any form, electronic or 
print, without prior, written permission from the author(s), and 
advance notification of the editors of MTO.
[2] Any redistributed form of items published in MTO must
include the following information in a form appropriate to
the medium in which the items are to appear:
	This item appeared in *Music Theory Online*
	in [VOLUME #, ISSUE #] on [DAY/MONTH/YEAR]. 
	It was authored by [FULL NAME, EMAIL ADDRESS],
	with whose written permission it is reprinted 
	here.
[3] Libraries may archive issues of MTO in electronic or paper 
form for public access so long as each issue is stored in its 
entirety, and no access fee is charged.  Exceptions to these 
requirements must be approved in writing by the editors of MTO, 
who will act in accordance with the decisions of the Society for 
Music Theory.
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
END OF MTO ITEM