Hunter S. Thompson once wrote, "If you work in either journalism or politics . . . you will be flogged for being right and flogged for being wrong." I was reminded of Thompson's words when I read a forum post on our website. "Why is Stereophile way behind the other magazines?" asked "rs350z," explaining that, among things, he objected to Stereophile's supporting its reviews with measurements. "why waste the ink on doing measurements on each product reviewed," he wrote, with a disregard for capital letters. "There is no need to. I don't care if the distortion is 0.00005 or 0.00007, nor do i care about all of the other tests you do. What i care about is the sound, quality, finish, looks."
Around the time this forum posting appeared, I was finishing up an essay for the Dutch audio magazine Linear Audio (footnote 1). Its publisher, Jan Didden, had asked me to write a guest editorial: "It dawned on me that you're probably the Last Editor Standing who has an in-depth knowledge of the technical issues involved in audio and perception," Jan had e-mailed me. "Linear Audio caters to the technically savvy and that their view of 'the audio glossies' is not always favourable. So I thought: why not ask JA to write a Guest Editorial? That would give him a stage for a balanced view of what he feels is important in reviewing and judging audio equipment, as well as what can and cannot be done with measurements."
That essay, "The Art and Science of the Audio Review," appears in Volume 8 of Linear Audio, which is will be available from www.linearaudio.net. My first paragraph reiterated something I have written before: A Damascene moment for me, as a young hi-fi enthusiast was a review of Ortofon's S15TE moving-coil cartridge in the July 1966 issue of the English magazine Hi-Fi News, written by the magazine's then editor, John Crabbe. While that 1966 review featured measurements, the measurements were not the whole of the review. Those measurements and the description of the listening went together; one could not be read without reference to the other, and vice versa. Each supported the other.
That review was the prototype for what I then felt—and, nearly 50 years later, still feel—an audio magazine's equipment review should be. More than a century ago, George Bernard Shaw wrote that you don't need to be a carpenter to judge the quality of a table; that is, a product should be judged on how well it performs its core function, not on how it is crafted. So while you don't have to be an audio engineer to judge the result of audio engineering, the quality of the craftsmanship still needs to be examined if the review is to give a complete picture of that product.
Before 1986, when I joined Stereophile, the magazine's reviews had only occasionally included measurements. J. Gordon Holt founded Stereophile on the Shavian idea that the optimal way in which to judge the performance of an audio component was to do what its purchasers would do: listen to it. I didn't disagree with that idea, but following that moment of satori with John Crabbe's Ortofon review, my goal was to start including measurements in Stereophile's equipment reviews.
That happened in the fall of 1989. I explained to the readers that, unlike the mainstream magazines, where measurements appear to define or even replace the listening experience, Stereophile's measurements would have four goals: 1) discover measured behavior that would affect a product's possible compatibility with other products; 2) uncover inadequate engineering; 3) ensure that there isn't some simple reason for a component to sound the way it does (spending a lot of money on a component with a built-in tone control in the form of a nonflat response is not my idea of a good deal); and 4) build up a measurement database that will eventually reveal correlations between what is heard and what is measured.
Achieving the first and third goals was straightforward. There are now more than 1600 reviews posted in Stereophile's free online archive—615 loudspeakers, 625 amplification products, 375 digital products—almost all with measurements. If, for example, you want to see if your speakers will need careful matching with an amplifier, you will find all you need in our unique archive.
The second goal has been more controversial. I wanted readers to be able to distinguish between questionable products that are based on a shaky knowledge of audio engineering and those that demonstrate mastery of the craft. I felt that even if a reader was technically naïve, the explanatory text I write about every product's performance will provide the necessary insight. As reader Jim Tavegia wrote on our forum, "Testing exposes misstatements and glittering prose. The truth is often not in ad copy, but in science . . . people spending serious money would like to know they [are buying] more than nice casework." But there are those outliers that continue to mystify, like the Lector S-192 DA processor, whose sound Art Dudley praised in his June 2014 review, and which I felt was poorly engineered (see "Letters," p.11).
A quarter century later, however, that fourth and final goal seems as far away as it was in 1989. All I can do is repeat the words of architect Ludwig Mies van der Rohe that head this essay. And in those 1600+ reviews, there are a lot of details.—John Atkinson
Footnote 1: Linear Audio, published in English and first published in 2010, is written by audio engineers for audio engineers. It is the closest thing to an academic journal in the field of audio product design. Published authors include the late Peter Baxandall, Richard Burwen, Bob Cordell, Stan Curtis, Joachim Gerhard, Malcolm Hawksford, Siegfried Linkwitz, Richard Marsh, Nelson Pass, Bruno Putzeys, Douglas Self, Peter van Willenswaard, and Scott Wurcer.
Footnote 1: Linear Audio, published in English and first published in 2010, is written by audio engineers for audio engineers. It is the closest thing to an academic journal in the field of audio product design. Published authors include the late Peter Baxandall, Richard Burwen, Bob Cordell, Stan Curtis, Joachim Gerhard, Malcolm Hawksford, Siegfried Linkwitz, Richard Marsh, Nelson Pass, Bruno Putzeys, Douglas Self, Peter van Willenswaard, and Scott Wurcer.
