Sound science
Musical tastes differ, but tests show that listeners respond with the consistency of spectrum analyzers to loudspeaker performance.
Rick Nelson, Chief Editor -- Test & Measurement World, 11/1/2004
 |
| Allan Devantier and Floyd Toole prefer popcorn while experiencing the JBL Synthesis consumer home theater system. Your tastes might run to Junior Mints, but they'll bet you'll share their loudspeaker preferences. Photo by Steve Labadessa. |
NORTHRIDGE, CA. Veuve Clicquot or Bud Light? Even if price is no object, your tastes might run toward the brew instead of the bubbly.
With loudspeakers, though, choice isn't a matter of personal taste. High cost doesn't guarantee high performance, but people of varying tastes in music, home decoration, drinks, and favorite movie munchies are remarkably similar in what they want from a loudspeaker. At least, that's the view of engineers at Harman International here, and they've conducted the experiments to back up their position.
Those experiments have encompassed a range of dimensional and acoustic measurements along with blind listening tests in which listeners having various skill sets evaluate loudspeakers under test. The goal behind these experiments is to demonstrate that loudspeaker design is a science and, consequently, that scientific measurements can be predictors of listener preferences.
"Musicians are artists," said engineering VP Floyd Toole. The goal of loudspeaker makers such as Harman, he said, is to scientifically reproduce for the listener the artistic experience that the musician intended¡ªscience in the service of art, as Toole put it (Ref. 1). He joined Harman in 1991 from Canada's National Research Council (www.nrc.ca) with a mandate "to bring the scientific method to the development of our products."
|
See our Audio Test Forum Also, see: Wine and song . |
The scientific method wasn't an easy sell in an industry dominated by craftsmen who considered themselves artists on a par with the musicians themselves, carefully voicing their speakers to produce a "JBL sound" or an "East Coast sound," all the while claiming that technical measurements are not reliable indicators of sound quality. Toole disagreed, noting "loudspeakers should be as neutral and transparent as possible, and technical measurements should be able to demonstrate whether they are or not."
And in addition to overcoming the cultural resistance, scientific approaches haven't come cheap. According to Allan Devantier, manager of objective evaluation, a resistance to technical measurements has stemmed from the high costs of making them¡ªcosts that industry players were unwilling or unable to bear. After all, he said, the barrier to entry in the loudspeaker business has been relatively low¡ªyou need a garage and a table saw¡ªand many practitioners simply lacked the resources to build anechoic chambers and outfit them with the necessary instrumentation.
Toole and his team at Harman, which develops and markets brands including JBL, Revel, and Infinity, have spent considerable time, effort, and money to determine what objective measurements are the best predictors of listener preferences. Beyond that, they have had to determine what test instruments would enable them to make the objective measurements quickly and accurately¡ªin the lab as well as in production. And finally, they have had to wrestle with the final component of any loudspeaker system: the room in which a customer will place a speaker¡ªoften in a suboptimal location.
The inadequate "+/- 3 dB"Of course, loudspeakers have always come with some specifications, which Toole said often served as an ornaments on advertisements and brochures, typically boasting "+/-3 dB from 20 to 20,000 Hz" and implying that such a spec represents the ideal. "That's rubbish," said Toole. +/-3 db can be shrill, if a speaker's response exhibits a midrange rise from -3 dB to +3 dB, or honky, if it exhibits a +6-dB midrange peak in an otherwise flat response. In addition, a +/-3-dB range can include many troubling, audible resonance points. If you can't achieve a +/-3-dB spec, said Toole, "you're in deep trouble. +/-3 dB is a giveaway. +/-2 is better, and if you reach +/-1, well, you have my attention." But ultimately, he said, you don't need a single spec--"You need a graph!"
But what to graph? Reverberant sound power is one spec that's commonly plotted over a speaker's frequency range. Sean Olive, manager of subjective evaluation, said that Consumers Union uses a similar approach in developing ratings for its Consumer Reports publication.
In fact, it was some middling ratings for Harman speakers in Consumer Reports that prompted an examination of just how effective that approach is, and according to Olive, the answer is "not very. In fact, the sound-power model suggests that if you turn a speaker around so it's facing a wall you'll have the same listening experience, and we know that's not the case." Indeed, Olive has discovered a negative correlation between CU's results and listener preferences and has developed his own models (Ref. 2) that indicate what objective measurements best correspond to subjective listener preferences.
A key to the Harman team's efforts has been to devise listening tests that control "nuisance" variables. Listeners are often influenced by a speaker's styling, price tag, manufacturer's reputation, and relative position within a listening room. And according to Devantier, listening tests for new products were often conducted by jet-lagged executives whose evaluations often depended more on recent earnings figures than on the acoustic attributes of the speakers they were evaluating.
To control these nuisance variables, Olive devised a listening room, called the multichannel listening lab, which employs a pneumatically operated shuffler with a 3-s index time. Under listener control, the shuffler brings any one of four speakers under test into an optimal listening position, and the speaker presents a brief music segment. A screen shields the speakers from view, thereby controlling any biases the listener may have relative to any of the nuisance variables. The listener may switch from speaker to speaker at will until he or she is confident in assigning a rating to each. Olive discovered that listeners respond consistently and repeatably when nuisance variables are neutralized.
Further, the Harman team learned that listener preferences correspond not with a single technical measurement such as total sound power but rather to a family of curves such as those indicated in Figure 1. Those curves represent on-axis response, the response averaged over a typical listening window, the early reflected sound response, the sound power, and directivity indices related to the sound power and early reflections.
|
| Figure 1. Harman engineers employ multiple frequency curves in their speaker model: on-axis response, listening window, the early reflected curve, the sound power, and directivity indices related to the sound power and early reflections. a) A good speaker exhibits smooth response in all curves. b) A poor one suffers from many resonances. |
 |
| Figure 2. This ray diagram, one of 15 representing typical domestic listening rooms and loudspeaker placements, helps to illustrate the importance of direct sounds and early reflections to the listening experience. Courtesy Allan Devantier/AES |
To generate the Figure 1 curve family, Harman engineers employ what they've dubbed the "Spin-o-rama"¡ªa turntable positioned within an anechoic chamber. With the speaker under test mounted on the turntable, a microphone located 2 m from the speaker acquires the speaker's acoustic response to a pink-noise-like input. The Spin-o-rama makes 70 measurements, spaced 10¡ã apart horizontally and vertically.
The chamber incorporates 4-ft. wedges, giving it a flat response down to about 60 Hz. To make accurate measurements below that, Devantier and Johnny Ventura, lab manager for the Harman Consumer Group, conducted ground-plane tests on 12 speakers having a range of sizes (from a 5-in., two-way model on up to a top-of-the-line Infinity model) in the company parking lot on a Saturday. The data from the ground-plane tests, using the same instrumentation that they used with the chamber, enabled them to derive calibration curves for the chamber so they can use it to characterize the 20- to 60-Hz performance of speakers under test to within 0.5 dB.
Instrumentation evolvesAs for the instruments themselves, the engineers are in a process of evolution. A key goal is to make the measurement process as automated and fast as possible. The chamber, says Ventura, is booked 9 out of 8 hours per day, and measurement speed is critical.
They have relied on instruments such as the MLSSA (maximum-length sequence system analyzer) acoustical measurement system from DRA Laboratories (www.mlssa.com) for amplitude measurements. For harmonic-distortion and compression measurements, they've used an LMS PC plug-in card, from LinearX (www.linearx.com). Devantier described the LMS as basically a low-cost ($1000) product that was adequate for the job once Ventura disabled its internal filters and substituted external Br¨¹el & Kjaer (www.bkhome.com) tracking filters in their place.
But both approaches are running out of steam. MLSSA is a DOS product requiring an ISA slot, and DRA Laboratories has no plans to move to Windows. Ventura said he combs the world for the Br¨¹el & Kjaer tracking filters and can no longer count on finding enough to meet his needs. What's more, said Ventura, "It's a nightmare keeping all these DOS and Windows products communicating with rotators, microphones, and printers." In addition, he wants a single data format that makes it easy to share data with sister companies.
Consequently, Devantier and Ventura are working with Audio Precision (www.audioprecision.com) and Listen (www.listeninc.com) to adapt those companies' instruments for laboratory acoustic measurements. Each firm's system has presented some challenges, as the Listen system is tailored for production acoustic tests while Audio Precision has been dominant in electronic, but not acoustic, applications, according to Devantier. The bugs are being worked out, said Ventura, and the systems are now able to talk to the Spin-o-rama turntable and meet other lab-test requirements. Devantier said he would ultimately like to see a 50-50 mix of AP and Listen systems in both lab and production applications.
Trained listeners
|
Do you agree that objective measurements can predict subjective experiences? To post your comments, and to see additional viewpoints, visit our Also: Wine and song |
"The audio industry has gone to the food industry to try to learn about trained evaluators," he said, "and has come away with the idea that training isn't valid for preference tests. That may be true for food," he added, "but not for audio. And the reason is there is no absolute reference for wine, for example. It's a manmade product. With audio, though, we do have an absolute reference¡ªthe original sound source, whether it's Pavarotti, a tree falling in the forest, or pink noise. Once you accept that premise, it's relatively easy to quantify audio quality objectively and subjectively. The rules for sensory tests for Chardonnay don't apply to speakers."
Olive's listening tests have involved 268 listeners (256 untrained listeners and a panel of 12 trained listeners) making more than 5000 preference ratings in 36 sessions over an 18-month period. He concluded that the preferences of trained listeners are generally the same as those of untrained listeners.
The training itself takes the form of a PC program; all a trainee needs besides a PC are accurate speakers or headphones with good bass response. The program presents a series of sounds to which the trainee can apply various filters of unknown characteristic. The trainee must then identify the effect of each filter¡ªfor example, boosting the low bass, attenuating the high treble, or boosting the low midrange. In essence, the program trains a person to become a spectrum analyzer having at least six frequency bins.
There are some subtleties to account for in listening tests, according to Olive. Trained listeners share untrained listeners' preferences in speakers, but they tend to be harder to please, scoring all speakers lower. Context errors can also intrude. In Figure 3, for example, the assessment of certain loudspeakers varies widely depending on test conditions¡ªin this case, the three speakers grouped with the one being rated. Olive reported that ratings tend to improve for a speaker tested in the context of inferior models, and vice versa. He suggested that larger sample sizes or the inclusion of a reference speaker could minimize these context errors.
|
|
| Figure 3. Context plays a role in listening tests. Here, speakers L1, L5, L7, L8, L9, L10, L11, and L12 have acquired varying ratings depending on what other speakers they have been grouped with for each test. Large sample sizes or the inclusion of a reference in each test could minimize such context errors. Courtesy Sean Olive/AES. |
Olive and Devantier's experiments have focused on consumer loudspeakers, which Devantier acknowledged can be the easiest to work with. "They represent the 'sweet spot,'" Devantier said, "where you have a 4¦Ð environment and know you want to minimize room effects." More challenging can be the test activities of Harman's JBL Pro group, which makes speakers for large concert venues and stadiums, and the Harman Multimedia group, which makes miniature speakers that fit within laptop computers or occupy minimal space on a desktop.
 |
| Doug Button of the JBL Pro operation practices “guerrilla engineering”—finding sly, unusual methods that yield accurate measurement results despite the unwieldy dimensions of his units under test. Photo by Steve Labadessa |
Doug Button of the JBL Pro group might like to operate within a 4-pi anechoic environment, but that's not practical for the huge loudspeaker systems he tests. In addition, the test microphone must be positioned at sufficient distances to make the relative distances from the microphone to each speaker transducer nearly equal. The speakers themselves, let alone the distantly placed microphone, simply won't fit within any anechoic chamber Harman could economically build.
Consequently, Button and his team practice what he calls "guerrilla engineering." They employ pseudo-anechoic gated measurements (capturing a speaker's direct response before reflections have time to reach the microphone) over 360¡ã. They'll also haul speakers up to the factory roof to make ground-plane measurements, and they've gone so far as to commandeer an airline hangar at nearby Van Nuys Airport.
Engineers in the multimedia group faces challenges of their own. Development engineer Clayton Williamson commented that in a customer's home or office listening environment, reflecting surfaces are so close to multimedia speakers that the surfaces must be taken into account in Harman's labs. To accomplish that, Williamson and his team have placed a schematic, representative desktop in an anechoic chamber that they use to test multimedia products.
Refining the modelMeanwhile, Olive continues to refine his predictive model based on anechoic measured frequency response, describing it in a paper prepared for last month's Audio Engineering Society conference (Ref. 5). In that paper, he describes using 1/20-octave measurements that better represent human hearing resolution than do the 1/3-octave measurements traditionally used in the industry. He further notes that the model predicts loudspeakers preference ratings with a correlation of 0.86 using a sample of 70 loudspeakers evaluated in 19 listening tests, vs. -0.22 for the Consumers Union model.
|
|
| Clayton Williamson of the Harman Multimedia division employs a schematic desktop in an anechoic chamber to derive measurements that will reflect customers’ experiences in the home or office computing environment. Photo by Steve Labadessa |
The Harman engineers' combination of psychoacoustic and technical measurements has enabled them to make significant progress in delivering favorable price/performance for the company's line of loudspeakers. "We can build the best loudspeaker in the world," said Toole, "and we can build the best speaker available at a given price." It's the latter, he said, that's most interesting from an engineering perspective. Toole said that the law of diminishing returns comes in to play at $1800 per pair, and that you can do well at $700 per pair if you're willing to sacrifice some bass. "We take pride in the fact that we can produce the best-sounding inexpensive loudspeaker in the world. If you can't build a good sounding speaker for $10,000 a pair, there is something seriously wrong with you. But if you can deliver comparable sound quality at a fraction of the price, then you really know what you're doing."
| REFERENCES |
|
