Formal vs. Informal

Recently a reviewer, whom I respect, had this to say about another reviewer’s comments on our reference system, “But I have some impression that under informal circumstances, audio people can get excited and come to debatable conclusions.” This is a fair enough comment, (To be clear, he has not heard the system, but is rather commenting from afar.) however, it got me thinking about the methodologies of inquiry that are commonly accepted practice and the weight that is given to them. It’s not that I think the typical review cycle of: listen to baseline system, insert new item, listen and repeat, and in some more rare instances, measure the device, is wrong, but rather, is that any more valid than say, visiting a room at an audio show and granting it a ‘best of show’ award? 
​
It seems to me that all empirical listening tests require one to carry over a memory from a previous time. That memory may focus on something heard 5 seconds, 5 minutes or 5 hours ago. It may be narrowly focused, but each event will add to the sum total of one’s experience so that the frame of reference will be altered by some amount the next go round. As Heraclitus of Ephesus said, “You cannot step into the same river twice.” If we could set up a room where a substitution could be made instantly, without having to wait for the change, you would still be operating off of a memory.

To what extent do we grant ourselves permission to trust our own ears? And what amount of validity or lack thereof are we willing to grant to the hearing of another? Is there any absolute authority? Are scientific measurements the answer? Concerning the latter, to some extent yes, and to another no. The yes, is that, given the ability to measure and calibrate the interaction of a set of speakers with a room, one is able to maximize their performance while minimizing negative interactions. However, anyone familiar with the process of calibration knows that it always involves a compromise. One measures the interaction from multiple positions and takes an average. The readings are smoothed to make them legible, and changes are made based on both the readings and listening. When you change a cable or insert new isolation device the greatest likelihood is that the measurements will not budge. This doesn’t mean that nothing happened, it just infers that our ears may be able to hear subtleties that our equipment was not designed to measure.

Going from room to room in a show, reviewers gather a backlog of reference points. They are comparing not only the rooms at the show with each other, but they are comparing each moment of listening with the sum total of their experience. 

Let me return to the operable word in my reviewer friend’s comment, “informal”. It is this word more than any other in his email that set this train of thought in motion. Does there exist any such absolute as “formal”. My contention is there is not. Just as you can know instantly when something is amiss, you can also know when something is very very right. It may be something you have never heard before, but that becomes instantly familiar, as though you had been intimate with it all along. Whatever context this occurs in is valid.

So we have discussed the two major contexts of reviewing, the insertion of devices into a reviewer’s system and listening at trade shows. I believe there is a third, no less valid context: visiting a mature and highly evolved system. Many times I have heard show goers say, “my system at home sounds better than the (insert name here) system that costs many multiples of mine.” Are these listeners deluded or just mouthing sour grapes over what they cannot afford? I think not. There is a high level of likelihood that many of these “lesser” systems do sound better. Why?

1. The mega buck system was set up the day before and hasn’t had a chance to settle in. The one at home has       been settling in for weeks, months, years.
2. The show equipment is brand new.
3. The manufacturer had to pair up with another to be able to afford the room, and their equipment is antagonistic for one reason or another.
4. Realistically there isn’t time to endlessly move speakers, add tweaks etc.

I think that reviewers should be keen to visit manufacturers’ reference systems because, presumably, that is where they put their greatest effort. The reference system is the laboratory where forays into the next level and the next in sound reproduction are forged. It is where care has been given to the finest details and the maximum is in the process of being extracted.
​
These ears have developed over millennia. They are designed to distinguish between the crunch of a twig and that of a leaf. They can triangulate location in three dimensions with pinpoint accuracy. I say, “Trust your own ears. By all means repeat and verify. Draw your own conclusions. Keep an open mind.”