Why is A/B/X testing so popular?

Please find all of the differences between the two pictures, but you may only look at each picture for 10 seconds each. Is this a feasible task?

Note: I am not here to convince anyone to purchase anything in particular or to upgrade x, y, or z. I wholeheartedly believe that the potential buyer should test/demo out their purchase and trust their own ears, rather than reviews.

The thing I want to point out is how one should demo their potential buy. I am baffled about how popular the A/B/X method of testing is in this particular hobby. Many headphones, amps, DACs, cables, whatever else, work on subtle differences. Thatā€™s not something you can pick out in 10 seconds and certainly not blindly if you donā€™t know to look out for it. The thing is, those subtle differences can make a difference in enjoyment for someone. Maybe that slightly wider soundstage or the little bit more accurate imaging fools a listener into believing it is almost a real performance.

Before this gets to ranty, I think how Steve Guttenburg suggests to demo makes a lot of sense. Make one change in your chain. Live with it for 7 days. Get used to it, get intimate, this is the new norm. Afterwards, remove that item and go back. Is there a difference? If not, done deal, you donā€™t have to buy it, but if you notice a difference and enjoy the new item, there you go. Again, I am trying to talk about that this hobby is one based around subtleties. I donā€™t get a-ha moments on every single song. Sometimes I enjoy one build up on a particular song after an hour of listening. Same idea here.

Why would you try to pick out differences in 10 seconds when it should take much longer to get intimately familiar with it? One doesnā€™t visit a museum and spend only 10 seconds per art piece, just like we donā€™t spend only 10 seconds per song.

Ending note: I am also not writing this to pick any fights or anything like that as well. Just looking to have a discussion or maybe convince a person or two.

8 Likes

I think the general thinking here on HFGF isā€¦A for one or two days and B for another day or soā€¦giving time for mental burn in and to actually access the different sound signaturesā€¦then may be A/B to confirm :man_shrugging:

4 Likes

Itā€™s a big :man_shrugging: moment for me too.
Maybe Iā€™m the crazy one, but I have never been able to judge timbre, sound stage, texture, imaging in a short time. All of that takes at least a few hours for me and definitely a few days if I am making a larger purchase.
In fact, most things that I have been immediately impressed by ended up being very fatiguing for me in the long run, whereas my most enjoyable pieces took some time to grow on me.

3 Likes

No makes perfect sense to me :+1:

I think thatā€™s sometimes a knee jerk wow factor responseā€¦something totally different to what youā€™re normally down withā€¦@Wavetheory with his assessment of the DiPhiā€¦a great HP but not his bag.

1 Like

You know what? I love this question. Iā€™ve been wanting to comment on this for a long time, actually. I started doing it in this post and it started turning into one of my review-length works, lol.

I had to study and apply a lot of cognitive psychology in grad school. Based on that training Iā€™m pretty comfortable opining that neither extreme - all quick A/B/X testing or an all long-term-get-to-know-the-equipment approach on the other end - is sufficient for really understanding the differences between like audio components. The key difference being that our short-term memories are better equipped to handle certain tasks and our long-term memories are better equipped to handle with other tasks. And then pattern-recognition is involved, like really, really, heavily involved. Like ā€˜we donā€™t understand music at all if our brains arenā€™t as good at recognizing patterns as they areā€™ kind of involved.

Iā€™ll come back to this someday when I have more time to dedicate to it.

3 Likes

3 Likes

Here is a quick A/B/X stress test. This is often given to see if you are currently under too much stress. In the picture below, there are two animals. If you can immediately tell the difference between them, you are too stressed:

Click to take test

image

:rofl:

(yes, this is a joke)

8 Likes

I must be too stressed. I canā€™t even see the tiny rope and handle those chipmunks are using to water ski. The boat is kicking up a nice wake thoughā€¦

:man_shrugging:

3 Likes

For me itā€™s always easier to identify when something is missing, at leas in the short term comparison. Because I can mentally compare that that neuron path thatā€™s been burned as to how it used to sound. When I get something new I listen to it a lot and then like I said, take it a way.

The only thing Iā€™ve been able to spot the difference on in a short term has been when making upgrade to power supply and upgrade to source clock or path where itā€™s been immediately apparent the improvement.

The analogy of how your monitor looks when youā€™ve cleaned the screen when you didnā€™t think the screen was dirty before but after you clean it, itā€™s obvious.

1 Like

I think extremes are not useful. Only long term listening or only quick A/B/X are attempts at simplifying complex things (like what measurements attempt to do). But things are complex weather you like it or not.

Both need to be used. Long term listening for a general ā€œfeelā€ of the gear in question supplemented by A/B/X aimed at very specific specific musical passages that would highlight differences (things like width of stage, high frequency extension and decay, a bow sliding along a string, etc.).

I mean from a testing standpoint. A/B/X is what you want. Especially when making claims of fine nuance that only $15,000 speaker cables can give you. If you canā€™t pass an A/B/X test matching A/B to X then it shows thatā€¦ your $15,000 dollar cable is full of shit.

Now what A/B/X wonā€™t do is tell you which one is the one people will prefer and in that, itā€™s a pointless format.

1 Like

A lot of times, I do quick A/Bs right away. Then, I spend a few days with new gear. Then, I A/B again. Repeat until I come to some sort of conclusion.

I still havenā€™t decided if headphone burn in is an actual thing (plus I get most of my gear used). I do find, however, that ear burn in is absolutely a thing.

Iā€™ve had numerous headphones that I initially didnā€™t like, but came to appreciate what they did after I became accustomed to them.

1 Like

My impression is that A/B/X testing is exceedingly UNpopular. Hope Iā€™m wrong.

1 Like

Thereā€™s nothing to decide. Every mechanical thing that operates based on motion has break in. Because materials are stressed when they move. A stiff material (like the suspension of a speaker) will lose some of itā€™s stiffness and become more pliable (easier to move). Once the initial break in is done the material will reach its natural state and changes will be slower than drying paint and continue for the life of the product. The productā€™s life ends when said material has reached its limit of stress and fails.

1 Like

I believe that is often done at the factory, isnā€™t it?

Some do it. Most donā€™t. Especially headphones - they get cranked out and sent off. The few that do it are usually expensive (but not exclusively) speaker manufacturers. Breaking in drivers takes time, time is money. Those with higher margins can afford to do it if they choose to. Those with low margins donā€™t even consider it.

Please nobody allow me to start speaking about wine.
But youā€™ve hit the nail on the head here regarding the value of nuance vs. impact.
And it all ultimately comes down to usage, what does this item ā€œneed to doā€? Itā€™s why numbers alone is such a cold way to assess.

1 Like

This is what I was thinking about earlier. I left this thinking it doesnā€™t sound like burn in was too big of a deal. Definitely nothing to get too hung up in anyway.

Yeah I know that video and I have the utmost respect for Andrew. However, there have been tons of studies done and Iā€™ve done my own measurements using a Clio system that show speaker resonance frequency changing with burn in. The more appropriate question to ask is: does burn in (with the ensuing change in T/S params of a driver) change a speaker enough to be audible. I expect the answer to be: depends on the particular driver model. Depends completely on the surround and spider (headphone drivers only have a surround - no spider) materials and their various attributes with regards to elasticity. Because different materials can be used and have different properties. So it depends on what combinations a particular manufacturer has chosen.

Planars are a bit different because the diaphragm itself is the surround/suspension. Some planar drivers have a loose diaphragm so the up and down motion of it relative to the chassis itā€™s attached to doesnā€™t make a difference since there is no stretching involved. Other planars are tightly attached to the chassis so every time the diaphragm moves up or down, it stretches to some degree. Think of a balloon cut so itā€™s flat and then stretched over something. Now start moving the surface up and down. Why is this motion possible? Because the surface stretches. How much of an effect will this stretching have on the diaphragm (the balloon surface)? Depends on the material. How many up/down stretching motions are required for permanent deformation of the material? And again, as above, will the deformation be enough to cause an audible difference? No studies have been done to my knowledge. Neither scientific nor subjective.

On top of all that, there IS brain burn in. So if you DO hear a difference, how much of that difference is your brain vs the driver changing? All? None? Some?

Itā€™s one of those things that can be argued endlessly. But burn in itself is absolutely real and measurable in speakers. For electronics I wonā€™t even begin to surmise lol.

3 Likes

Iā€™ll go a bit further and add that the enclosure or lack of one would play a role as well. A sealed (acoustic suspension) enclosure would reduce the effects of break in because as its name implies, the internal air is part of the suspension (that stays constant). A vented enclosure or a lack of enclosure (like open back headphones) would be more susceptible to sound change with break in since the entire control of the driver motion is dependent on that moving thing - the driver suspension (surround and spider).