It's been 30 years since I began work on my very first equipment report, of the Goldbug Brier moving-coil phono cartridge, for Hi-Fi News & Record Review. That review appeared in the British magazine's May 1983 issue; I have lost track of how many equipment reviews I've written since then, but my review of the Vandersteen Treo loudspeaker in this issue is at least my 500th.
I touched on the subject of reviews in my Richard C. Heyser Memorial Lecture, "Where Did the Negative Frequencies Go?," which I was invited to give on October 21, 2011, at the 131st Audio Engineering Society Convention, in New York City. I had mentioned in that lecture that my education in electronics and audio was based exclusively on tubes. Even the logic circuits I constructed at school used tubes! But there was one experience that foreshadowed my career as an audio reviewer. For one of my final exams for my bachelor's degree, I was handed a black box with two terminals, and had to spend an afternoon determining what it was. (If I recall correctly, it was a Zener diode in series with a resistor.) That experience is echoed every time I try to characterize the performance of the audio components I review. Every product—be it a loudspeaker, amplifier, or CD player—is fundamentally a black box with input and output terminals. As a reviewer, I have to answer the question "What does it do?"
The next question, however, is more complex: "How well does it do it?" Throughout 2012 I appeared on panels at audio shows with other reviewers and editors, and the question came up more than once: How do reviewers assess the quality of the products they write about? One answer was also given more than once: "I compare the sound of the product against that of live, unamplified music."
Really? I know this is a common meme, coined by Harry Pearson back in 1973 when he founded The Abso!ute Sound. But, as I have argued in the past, it doesn't hold together with two-channel reproduction, in which the ambient sound at the original event is folded into the front channels. And as I demonstrated in my presentations last October at the 2012 Rocky Mountain Audio Fest, it is based on a fallacious assumption: that an accurate representation of the original recorded event is encoded in the grooves, pits, or bits.
But for the sake of argument, let's assume that a state-of-the-art audio component could make a system's sound indistinguishable from the real thing. With anything other than assaults on the state of the audio art, using the "absolute sound" to judge an audio component's quality is akin to trying to measure something with the meter set to the wrong scale. As every product falls short of meeting that goal, many by a large degree, the needle on the quality meter hardly moves from its rest position.
That doesn't even mean that such products are bad. Far from it. With very few exceptions, audio products are designed to a price point, which limits, often drastically, the resources with which a designer can work. To stay within the build budget, the designer must trade some aspects of performance off against others. So rather than proclaim, for example, that "this speaker has no bass" or "the midrange is colored" compared with "the absolute sound of live, unamplified music," the responsible reviewer must try to get into the designer's head. Why did the designer do it this way? Why does this speaker have no bass? Why is it colored in the midrange? What is the trade-off for which the designer has felt it worthwhile to sacrifice low-frequency extension or midrange neutrality? And why was that trade-off necessary?
Before passing absolute judgment on a product, therefore, the reviewer must first see how well the product measures up against what appear to be the designer's own goals. Then and only then can the reviewer assess how well the product performs, both against a more universal set of goals, and against a different set of compromises embodied in a competing product.
Of course, there is always what my erstwhile colleague Tom Norton, now technical editor at Home Theater magazine, used to call "designer tunnel vision": a concentration so narrowly focused on maximizing one aspect of performance that shortfalls elsewhere are apparently overlooked or unheard. But the reviewer can't take for granted that the designer is unaware of the problems—they may just be inevitable. Or, more likely, the problems don't intrude so much with the particular music the designer likes to play. This is why Stereophile's reviewers list all the recordings they use to form their value judgments. The review is analogous to inviting the reader 'round to listen to your system, and the choice of what music to play is critical. Very often, a product inserted into a system affects that choice, and that metadata becomes relevant information to readers of the review.
That's why the question reviewers are continually asked—"What's the best speaker at [insert price point]?"—is so difficult to answer. The most honest and least satisfying answer is "It depends." There is no "best speaker" that will suit every listener's needs. The reviewer lays out how the product sounds to him or her, but each reader then must examine her or his own preferences. Do I need maximum low-bass extension? Do I desire midrange accuracy? Do I value maximum loudness? Do I want holographic imaging? Can I give up one of these to get maximum performance elsewhere?
Most important, both reviewer and reader need to answer this binary question: "Do you like the sound of this component, yes or no?" The Golden Rule for all listeners: To thine own ears be true.
