Here's an experiment: Record your favorite LP with a high-quality analog-to-digital converter, then play that digital recording with a DAC and digitally record the DAC's output—again and again, with an ADC/DAC loop. What if you were to listen to, say, the 16th recording and find you can't distinguish it from the original LP? Siau says that, with the DAC3 HGC (footnote 6), he can detect the difference only on pass 17. Now imagine that the DAC3 has a sound of its own that is audible in a single pass. By pass 17, that "sound of its own" would be so exaggerated that it would mask everything else.
But if the DAC3 HGC performs as its designer claims it does, it's a chameleon that precisely reproduces the sound it's been fed. Any sound attributed to it isn't really the sound of the DAC but the sound of the recording, or the room, or other components, or the reviewer's expectations.
Actually Listening
With the DAC3 HGC set to its highest output level—the default setting—it was much too "hot" in my system. It worked fine—no overload—but was limited to the first quarter or so of the volume control's travel, whether the DAC3's own knob (with the DAC3 feeding amplifiers directly, as recommended for lowest noise) or my preamp's knob (with the DAC3 set to fixed volume). Using the DAC3's –10dB attenuators kept me at or near the top half of either volume control; at –20dB, the volume maxed out a little too soon.
After that, I went through the whole menu of inputs and formats, with files sourced from a Roon Rock—Roon's Intel-based dedicted audio computer—to ensure that everything worked as specified: PCM to 24-bit/192kHz; single-rate DSD over PCM (aka DoP). All sample rates and bit depths of PCM worked fine, but when I sent the DAC3 a DSD track via DoP, all I heard was quiet static. I decided to move on and come back to this later.
Intrigued by intersample overs, I wanted to see if I could reproduce their characteristic distortion pattern with the DAC1. My first-generation DAC1 lacks a USB input, so, following instructions in the DAC3's manual, I set up the DAC3's D4 digital input as a digital output and used that to feed the S/PDIF input on my DAC1. I got music—nifty!—but when I sent the DAC1 a test signal designed to produce intersample overs, I got a clean spectrum with none of the expected distortion.
Both the DAC3's failure to play DSD and my inability to reproduce intersample overs with the DAC1 via the DAC3 suggested that the DAC chip wasn't receiving bit-perfect data. Either the server or the DAC itself was altering the data.
I'd followed all instructions and correctly set Roon's settings, including "Fixed Volume" output, so this was a mystery. I contacted Roon and Benchmark, seeking guidance. It was Danny Dulai, Roon's COO, who found the solution: "Set Max Device Volume at Playback Start" in Roon's DAC3 settings. Apparently, the volume was being fixed at less than 100%—not bit perfect. Whether this was happening inside the DAC3 or in the Linux-based drivers that control the DAC3, is still in dispute as I write this (footnote 7). But now I can produce intersample overs, and DSD files play just fine. Problem solved.
Because of my intersample-over experiments, I now had my old DAC1 in the system, and decided to give it a listen. According to Siau, there are two possible ways the DAC1 and DAC3 could sound different: those intersample overs and a possible channel imbalance, most likely at the low end of the volume knob. In fact, I found the sounds of the two DACs quite different. The DAC1 was brighter, all sheen and surface. The DAC3 was all about depths, in several respects: the sound was weightier, with a darker frequency balance; the soundstage was deeper—I heard deeper into the music. With "Gaslighting Abbie," from Steely Dan's Two Against Nature (CD, Giant 7599 24719 2)—a well-recorded song that nevertheless should display intersample overs—the voices through the DAC1 were a little splashy, but sounded much more natural through the DAC3. Through the DAC3, percussive upper-midrange/lower-treble sounds took on a 3D quality, a shape in space, that I've come to associate, ironically, with MQA—ironic because Siau is one of MQA's most vocal online critics.
Maybe I was hearing intersample overs through the DAC1 that could account for the splashiness—or maybe my old DAC1 has drifted out of spec. Either way, the difference between Benchmark's first- and third-generation DACs wasn't subtle.
After that, I backed away from close listening, still listening frequently via the DAC3 but focusing on the gear less intensively. I listened for days, mostly casually, only occasionally playing close attention, switching between my PS Audio DirectStream DAC and the Benchmark DAC3 HGC. Listening casually over time sometimes helps me get a sense of a component's sound; this time, though, nothing much stood out. Apparently, these two DACs sound much alike.
Then I did something I hadn't done in years: I compared the DACs directly, head on, playing the same music and switching between them. I used Roon to set up each DAC, creating a "zone" for each. Then I "grouped" the zones so that Roon would send the same music to both zones simultaneously.
First, I sent the output of the two DACs to different channels of my PS Audio BHK Signature preamp. Later, I sent the output of the DirectStream DAC into one of the DAC3's analog inputs, switching between this input and the USB input coming directly from my server. (This approach gives a slight edge to the DAC3, because the DAC3's analog inputs are unbalanced, and because the DirectStream's output is passing through the DAC3's preamp.) Both approaches worked, but switching between DACs was a bit faster on the DAC3—a Benchmark design goal, and an advantage for this sort of listening. Matching volume levels was a matter of playing a sustained sound—I used a white-noise track, then a pure 500Hz tone—switching back and forth rapidly, and adjusting the volume until I couldn't hear a difference.
After that, I listened for hours to a wide range of music, switching back and forth as the spirit moved me, sometimes several times in a single measure, sometimes only after whole movements.
Why do this? Because the tools now available for making this work—especially with digital sources—are really, really good, and also because this kind of testing is a good idea, whatever audiophile orthodoxy may say. I can switch back and forth hearing precisely the same recording, with no meaningful delay, during the duration of a single sustained note or its decay in an acoustically rich space. Differences in how the acoustic is characterized, or in the musical texture, should be immediately obvious.
A few musical moments stood out: The loud, blatty trombones in the second movement of Shostakovich's Symphony 5, with Mstislav Rostropovich conducting the London Symphony Orchestra, made me shiver (24/96 download, LSO Live 822231105824/HDtracks). The quiet, sustained violins in the next movement were delicate and clean—really lovely. Rostropovich despised this symphony, but he produced a stunning performance recorded in extraordinary sound.
Live at the Bad Habits Cafe is a really good album by Bad Influence—a great, longstanding (approaching 30 years) band based in Washington, DC—and minimally recorded by Mapleshade's Pierre Sprey (rip from CD, Wildchild! 03152). Wes Phillips raved about it in "Quarter Notes" in the March 1996 issue. It's great music—roots rock performed in a modern style. The track "Night Life" has immediacy, depth, and tonality: whether it's voices or sax, the textures are enlivening, and they were fully conveyed by both of these DACs in my system.
I spent an afternoon of focused listening, then part of the evening with my 19-year-old son, a serious musician, listening to music he's exploring: Led Zeppelin, Eminem, Schoenberg, J.S. Bach. We sat back to listen (he was in the good chair), switching between the two DACs, sometimes quickly, sometimes slowly.
Neither of us heard a difference—nothing. Soundstage depth? Tonality? Reverb tails? No difference. Did the DAC3 sound very slightly cleaner? Perhaps. Did it have a little more energy in the very lowest bass? Maybe. Was the DSD-based DirectStream a touch smoother? I thought it would be, but didn't hear it.
The next day, a friend and fellow audio reviewer stopped by for a listen. I checked to make sure the volume levels were matched, then handed him the remote. He did the same demo I'd done, listening mainly to percussion (he's a drummer), switching at will. His conclusion was the same as mine: No difference.
It's possible we missed some quantitative distinction, like the absolute length of reverb tails, or whatever. But I was listening to and for music, not—or not only—for sounds. I can conclude with confidence that whatever I might have missed, it was not musically important. From a perspective of emotional impact, at least, with this full-range system, these two DACs were equivalent. If you think I've got tin ears, I welcome you to do this test yourself.
Conclusions
Once, in these pages, I expressed regret that rigorous listening tests are not truly viable in many high-end audio contexts, because without them we're unmoored. Without rigorous testing, it's hard to know what progress is real and what is only apparently real. There's a danger of being misled, of repeating the same mistakes again and again, of spending way too much money on things of little value. All of that helps the more cynical part of the high-end industry thrive at the expense of the more authentic, conscientious part. That's regrettable.
My point is that, as a hobby, industry, and avocation, we may have shifted too far toward the subjectivist side. Yes, I know the history, and accept its lessons: Measurements sometimes lead us astray, and there are plenty of open questions about what makes a component sound good.
Benchmark Media Systems' DAC3 HGC is evidence that an engineering-focused company—one that seems to spend more time staring at instruments than testing its devices with their ears, and that cares more about transparency than it does about whether or not you like the sound their products make—can produce a product of astonishing fidelity and emotional expressiveness. Science works. Artists may be sexy, but never underestimate a good geek.
Footnote 6: Siau didn't say which ADC he used, but presumably it was Benchmark's ADC1, which John Atkinson reviewed in the February 2014 issue.
Footnote 7: After much discussion with Benchmark and Roon, I've tentatively concluded that it's both: The DAC3 inadvertently exposes a volume control that's then seen and exploited by the Linux drivers Roon uses.
With the DAC3 HGC set to its highest output level—the default setting—it was much too "hot" in my system. It worked fine—no overload—but was limited to the first quarter or so of the volume control's travel, whether the DAC3's own knob (with the DAC3 feeding amplifiers directly, as recommended for lowest noise) or my preamp's knob (with the DAC3 set to fixed volume). Using the DAC3's –10dB attenuators kept me at or near the top half of either volume control; at –20dB, the volume maxed out a little too soon.
After that, I went through the whole menu of inputs and formats, with files sourced from a Roon Rock—Roon's Intel-based dedicted audio computer—to ensure that everything worked as specified: PCM to 24-bit/192kHz; single-rate DSD over PCM (aka DoP). All sample rates and bit depths of PCM worked fine, but when I sent the DAC3 a DSD track via DoP, all I heard was quiet static. I decided to move on and come back to this later.
Intrigued by intersample overs, I wanted to see if I could reproduce their characteristic distortion pattern with the DAC1. My first-generation DAC1 lacks a USB input, so, following instructions in the DAC3's manual, I set up the DAC3's D4 digital input as a digital output and used that to feed the S/PDIF input on my DAC1. I got music—nifty!—but when I sent the DAC1 a test signal designed to produce intersample overs, I got a clean spectrum with none of the expected distortion.
Both the DAC3's failure to play DSD and my inability to reproduce intersample overs with the DAC1 via the DAC3 suggested that the DAC chip wasn't receiving bit-perfect data. Either the server or the DAC itself was altering the data.
After that, I listened for hours to a wide range of music, switching back and forth as the spirit moved me, sometimes several times in a single measure, sometimes only after whole movements.
Why do this? Because the tools now available for making this work—especially with digital sources—are really, really good, and also because this kind of testing is a good idea, whatever audiophile orthodoxy may say. I can switch back and forth hearing precisely the same recording, with no meaningful delay, during the duration of a single sustained note or its decay in an acoustically rich space. Differences in how the acoustic is characterized, or in the musical texture, should be immediately obvious.
A few musical moments stood out: The loud, blatty trombones in the second movement of Shostakovich's Symphony 5, with Mstislav Rostropovich conducting the London Symphony Orchestra, made me shiver (24/96 download, LSO Live 822231105824/HDtracks). The quiet, sustained violins in the next movement were delicate and clean—really lovely. Rostropovich despised this symphony, but he produced a stunning performance recorded in extraordinary sound.
Once, in these pages, I expressed regret that rigorous listening tests are not truly viable in many high-end audio contexts, because without them we're unmoored. Without rigorous testing, it's hard to know what progress is real and what is only apparently real. There's a danger of being misled, of repeating the same mistakes again and again, of spending way too much money on things of little value. All of that helps the more cynical part of the high-end industry thrive at the expense of the more authentic, conscientious part. That's regrettable.
Footnote 6: Siau didn't say which ADC he used, but presumably it was Benchmark's ADC1, which John Atkinson reviewed in the February 2014 issue.
