Lies, damn lies and FR graphs

This post is a bit of a detective story that will interest myself and maybe four or five others on the forum. If you sometimes look at frequency response graphs but only have a hazy idea how to read them, suffice it to say this is a cautionary tale on how misleading they can be. For the grisly details read on. I’ve tried to present the relevant information in an easily accessible manner.

The first many of us knew that there was such a thing as a new Audeze LCD-1 was Metal571’s October 16th YouTube review (https://www.youtube.com/watch?v=AabXYdmrqgo&t=1s). He was clearly excited verging on euphoric, and his sound description was on the favourable side for a Metal review.

The unrepentant geeks among us went right to the two FR graphs he posted in the description box for the video. He has a Mini-DSP EARS and posted the graph he made using it:

This is a compensated graph (HPN), so in theory a perfect frequency response would show as a straight horizontal line. Any up or down variations should represent imperfections in the tuning. Vertical axis is loudness, horizontal is frequency or pitch. Lows or bass is on the left, mids are in the middle, highs are on the right. The graph says the lowest bass has a sharp drop-off, there is a serious dip between 1 kHz and 3 kHz and another serious dip between 3 kHz and 10 kHz.

Translated this means the bass doesn’t have quite the flat-line extension we expect from the best planars, vocals and many instruments may sound a bit thin in the high end of the mids and the treble should be mostly laid-back, although good in the air region or top octave. However, in his spoken review Metal warns us that he had a hard time getting a good seal when measuring the LCD-1, meaning some or all of the sub-bass dip could be due to that. And beyond all that the sheer jaggedness on the right end of the graph suggests a truly harsh sound in flat contradiction to his oral report.

But Metal also attached a frequency response graph sent to him by Audeze, dated from the day before:

This is an UN-compensated or raw graph, so making sense of it involves some prior knowledge. Overall, this graph has an uncanny resemblance to a typical Head-Fi/Jude graph. The similarity is so strong one can almost imagine Jude or one of his staff having sent it to Audeze, then Audeze in turn passing it on to Metal571. But Metal only states that it comes from Audeze, so we’ll have to leave it at that.

Like a Head-Fi graph, the loudness/vertical axis is foreshortened, the FR/horizontal axis is stretched and it has something like 12th-octave smoothing applied. Metal tells us it was done using GRAS plus Kemar and the graph itself bears the Audio Precision logo. This is some of the latest, greatest and most expensive measurement gear going. The squashed vertical plus the smoothing puts the measurements in the best possible light. The bass fall-off looks nearly non-existent. The 1 to 3 kHz recess is essentially non-existent. Only the jagged treble seems to carry over from Metal’s graph – and that’s something we tend to take for granted.

The above, plus a few early listener reports, were all we had to go on for the first month. Then today I stumbled upon a graph for the LCD-1 from Oratory1990 in his reddit-linked repository, presumably fresh from the oven. Oratory also uses the latest and greatest measurement equipment and presumably has no horse in the race, so what does he have to tell us?

Audeze%20LCD-1%20raw%20FR%20-%20Oratory1990

Both are raw, but it’s hard to imagine anything more different to the eye from the Audeze raw graph above. The gentle knoll topping out at 3 kHz in the Audeze graph looks like an impossible Mt Everest here. Not sure I’ve ever seen anything quite like it elsewhere. The bass, 1-3 kHz, and treble dips Metal’s graph show are all here, but in a much more understandable layout.

Is there any relationship between the Oratory1990 and the Audeze graph?

Here I’ve taken the Oratory and Metal571 graphs, stretched them horizontally and squashed them vertically until they have the same scaling as the Audeze graph. Now what’s most remarkable is how similar they suddenly revealed to be, especially the two raw graphs on top. To my eyes the main difference between the two is that the Audeze graph lacks nearly all the 1 to 3 kHz scoop. Even the compensated EARS graph makes sense if we imagine it tilted upward from 1 to 3 kHz then back downward after that.

All of which should lead us to conclude that we need to beware of the semblance of technical authoritativeness that graphs exude. Even when no active data falsification is involved (presumably a rare to non-existent thing), even when equipment limitations and operator error are a non-issue, presentation remains everything.

9 Likes

Or ya know, just listen to it and not really worry about graphs and specs lol

But seriously good write up :+1:

1 Like

learned some stuff today. :+1::grinning:
I like a good graph.

12 Likes

I like a good graph too and the one you post is a beaut! (Not to mention very appropriate to the ones I posted.)

1 Like

No two listeners tell the same story so we need graphs. No two graphs tell the same story so we need listeners. No two pocketbooks have the same coinage so we need you.

3 Likes

Wouldn’t the proper way be to print input signal and return (or measurement) in the same graph?
So basically have a 4 channel scope do a fouriertransform on all 4 channels, 2 of wich beeing exact mirrors of input, 2 beeing from measurement mics.

Edit:
Found what I was describing:
image
https://www.rfwireless-world.com/test-and-measurement/Gain-flatness-measurement.html


Edit: Don’t believe in data you have not faked yourself applies everywhere.

Honestly, this should be pinned to the top of the forum. I went through a similar discovery when I was looking into “Unbelievable” looking flat FQs from a company I won’t name some time ago. It turned out to be because the way it was presented it was heavily skewed to look extremely “flat”… when in reality it was VERY V shaped relative to many other high-end headphones.

1 Like

Yes, graphs from manufactures are most of the time compensated to their advantage

1 Like

compensated, smoothed, and literally stretched and squashed in the right ways as well.

1 Like

TBH I just prefer raw graphs even though they can be harder to understand, but then you don’t have to worry about compensation tbh (just removes 1 thing out of the many factors of the graph)

I also believe some people underestimate, looking at a graph, then listening to the actual headphone, can effectively skew how they perceive the sound as well. Like if you see a graph you “believe” to be accurate and flat/neutral, you’ll listen to the headphone and “hear” flat/neutral.

2 Likes

Yes I agree lol, it can really skew what you think

1 Like

Boy does this echo my own observations. Many YouTube reviews seem to be the reviewer looking at an FR graph – often one he created himself – then putting what the graph shows into words.

1 Like

Yeah, not much of a review lol, more of a spec reading

2 Likes

Your technical thoughts are 'way over my head so hoping further discussion will shed some light. But that Don’t believe aphorism is one I may adopt as my personal motto from now on, lol.

1 Like

So as a noob are any graphs trustworthy and is there a good place to learn more about how graphs work.
Or should I continue to be happy in my bubble without graphs?

Being “happy” is the ultimate goal here… IMO. Graphs can be useful, you just have to be wary. But honestly, if you aren’t neurotic about audio like a lot of us are, then that is a gift I would hold on to dearly…

2 Likes

Yup.
The art of a review is to present specs and measurements in context, explain what x means for y.

The idea is to compensate for the shortcomings of the measurement rig.
When all you record is the data from the mic that measured the frequency response, you skew all your data for that mic.
In other words, the data obtained that way is only comparable to the same setup.
For the reader that means the data can only be compared (!) to the data from that website.

What I described and linked is a way to have the “skew” logged aswell as the measurement.
That way the reader can use the “skew” to compensate, then compare to data from other sources where their skew is given.

2 Likes

Yes, but not everyone will be willing to do that lol. It would be nice but I don’t think it would be very often done

1 Like

Honestly I rarely worry about specs or graphs, I just worry about real world performance and how it sounds to me lol

2 Likes