Can you trust your ears ? Can anyone ?

In particular - do data-identical audio files always sound the same ?

I've just been taking parting an interesting conversation on the LinkedIn Pro Tools User group about the best type of lossless file compression to use for moving audio around.

Some people, including me, have replied "any, they're all the same", but several more very experienced, respected and successful engineers have commented there that they never use data-compression on their audio files, because they think it can affect the sound.

Now when it comes from people you respect, this is a fascinating claim, because the there is no theoretical way that this can be true, unless there is a fault with the playback system, or the testing method.

Because lossless data-compression has to be that - lossless - or it's broken, by definition. And, if two files are data-identical, they are effectively the same file. So they must sound indistinguishable - again, by definition. If they don't, then there is a fault or design flaw with the replay system.

Thats the theory, at least. And in my experience, the theory holds up. Here's an example:

Sometime in the mid nineties, I came face to face with the 'data identical files don't always sound the same' issue myself. I had completed mastering on a really great-sounding independent production, and both I and the artists were delighted with the results. Imagine my surprise when shortly after the discs were pressed, the engineer called me up, very disappointed with the way the pressed discs sounded. To his ears, the CDR test copy sounded much better !

I took a quick listen, and was shocked to find I agreed with him.

My immediate assumption was that there was a pressing fault, and I loaded up both the test CDR and the manufactured disc in order to do a bit-for-bit comparison. To my surprise though, the disc checked out - the numbers were identical. They should have sounded identical, too !

Now at that time a debate about this subject was raging on the Mastering Webboard (RIP), with people who claimed to have heard differences like this for themselves putting forward quite plausible theories about ways that data-identical discs might sound different because of poor player design revealing variables in the manufacturing process - "pit-jitter" and so on.

I'd always been sceptical, but now I had my own example to check out!

Having been fooled by my own 'observer bias' in the past, though, I knew I had to do a few listening tests myself to make sure I really knew I could trust what I was hearing. So, we set up a test scenario - two identical CD players, playing level-matched into a Studer desk, through B&W 801 monitors.

I found the test harder to repeat listening blind, but after a gruelling 40 repeat listens, I came out being able to hear a difference between the CDR and the pressed disc with about 70% accuracy. The differences were subtle, but that's a statistically significant result. I was very excited - now, the problem was how to explain this to the plant, and figure out how to solve the problem.

I was literally about to get on the phone to my client, when I suddenly realised - there was a flaw with my testing method.

We had randomised which disc I was listening to, but not which player.

With a sigh, I steeled myself. We swapped the discs over in the players, and I did another 40 of the tedious blind tests. This time was easier though, because I knew what I was listening for, and sure enough, this time I picked one of the discs as sounding better with 80% accuracy.

The problem was, this time I picked the other disc.

The first time I picked the CDR as sounding better, but when we swapped the discs between machines, suddenly I picked the pressed CD, instead.

Which means, I wasn't hearing a difference between the discs at all - I was hearing a difference between the players. Well actually, I can't even say that for certain. It might have been a difference between the players, or perhaps I was hearing a minute level-difference between the two analogue signal paths. We had checked the output levels to +/- 0.1 dB, but even that doesn't guarantee they were exactly the same.

So, that means the CD actually sounded the same as the CDR after all, right ?

No ! It just means I didn't spot any difference - or rather, that if they did sound different, those differences were masked by the differences between the two players and/or the signal path.

Now in my opinion, that does actually mean there was no audible difference. I've tested myself in this way many times and I know I can hear a decent number of fairly subtle differences in digital audio - for example, EQ level differences of 0.3dB, the differences between different flavours of dither and noise-shaping, the difference between dithered and truncated 24-bit audio at 16 bits, absolute phase...

But if there was a difference here, it was too subtle for me.

That's not really the point, though. ("What, oh what is the point", I hear you cry ?!?)

The point is, I jumped to a conclusion, because of the opinion of a client I trust and like - and I stayed fooled by my own observer bias, right till the last moment, when I had to admit I'd proved to myself I was wrong, despite my initial instincts.

And, it's not the first time.

So, I know from my own experience how easy it is to make mistakes when we're listening to small (and sometimes, not-so-small ! ) differences. And, it helps demonstrate again just how hard it is to create a proper test. It's even been shown that problems like a small delay or a tiny click when switching between two identical sound-sources can make us perceive a difference in quality.

Now, I have no way of knowing exactly what the engineers I was talking to on LinkedIn heard, how they tested themselves, or what might have caused the differences they heard. But I think there's a good chance they may be mistaken, especially since the thing they heard - data differences between identical files - are theoretically not possible, in a properly designed digital audio system.

But maybe this is just my observer bias again, although this time from the other angle...!

Can you trust your ears ? When did you last fool yourself ?

Awesome image by Furryscaly