View Single Post
Old 24th September 2013
  #46
Quote:
Originally Posted by Rumi View Post
Hi!

Your points are very valuable, and thank you for posting them!
Some years ago I did many comparison tests, but after that test with the CD de-magnetizer, where it turned out in the end that we were comparing bit-identical files (and two participants still could hear the difference!), I stopped doing comparisons. (Interestingly enough a physics professor bought that CD de-magnetizer when I put it on ebay after those tests. I would have loved to hear why he does that, since he must know that there is nothing magnetizable in a CD.)
I remember that product! Pure snake oil. That and those tiny little discs the size of large coins you strategically place on your walls for dramatic acoustic alteration.

The two that heard a difference only thought they heard the difference. I'm sure you know this, but that is a common theme a lot of people cannot relate to. Just because a person thinks he/she can hear a difference doesn't mean there is a difference. You would be surprised how many people do not understand that concept. We're only human.

When we get to a certain point of very fine details/differences between two audio files, for example, even though a scope can easily pick this up, it doesn't mean our ears/brain can. Furthermore, this is where the brain will inevitably inject "placebo," and you *will* begin to hear things that are not there. Because of this, we [scientists] developed a way to determine whether or not we are actually hearing a real difference, or whether or not we are just guessing. And we can do this with very high confidence.

The problem with posting non-scientific tests is that they can easily favor the 'inferior' product. So, while the poster has good intentions, the opposite effect can result. A good example is comparing two identical files, but one of them has an overall RMS figure of 0.5 dB higher than the other. Not only will the subjects continually pick the hotter file, they will almost always remark on how different the file sounds using various terms like "beefier," when in fact, it's the same file.

The above is a simple concept. But, what a lot of people don't understand (including many audio professionals) is that a difference of 0.5 dB (or even less) can MASK a product's attributes that would otherwise be favorable to it. Instead, they can easily be mislead by the hotter file, making the "better" product seem inferior.

The 0.5 dB example is just one of many flaws in typical comparisons you see on boards like this. There are many other things that can easily flaw a comparison, rendering it to the opposite effect of its real intention.

I believe people poke holes in a lot of tests not to be mean, but rather out of frustration (I'm no exception). That said, some tests on here do have value, but many can actually become much more relevant and useful by doing a couple of extra things (the mic/preamp swapping, for example).

Quote:
Originally Posted by Rumi View Post
I still wonder how people can hear the power cord of their amplifiers - a few feet of lenght, after miles of cable coming from the power plant. And you have to turn the amp down, switch cables, turn the amp on again, and then say "Ah yes, now the lows are in place again, the highs are much less strident, and the mids are just so smooth - this is why I spent $500 for that cable!" Even if there actually is a difference between the "tone" of power cables, you are not able to tell it with such a test.
Again, they only think they can. But, if it makes them feel better sometimes there's something to be said for the old saying, "ignorance is bliss". If too many pieces of equipment were shown to sound identical in scientific tests, then it would squash our desires to salivate over the latest and greatest, and that's no fun!

Quote:
Originally Posted by Rumi View Post
I will do the Nebula vs. hardware test in the link you've posted. I like Nebula a lot.
Great! Please try not to look at the other answers, as that can skew the results. The test isn't perfect, but later I plan on using the files in my comparison application, which is MUCH easier, faster, and with much more relevant results.

Some files posted on gearslutz may qualify to be used with the application. They have to follow certain guidelines so that they do not contribute to any flaws in the outcome. Other files may qualify with some post-production. In the case of your files, as long as you swap the preamps, they will likely qualify with a little post-production (level matching, for example). The beta version will not be available for a while, but eventually there will be tools to do this kind of thing automatically. I'm not developing the app with the intention to make a profit, but rather to contribute to science by taking aural comparisons to another level beyond what is available, and to aid in psychoacoustic research. It's basically a highly advanced, nearly non-existent bias file comparison solution that goes way beyond other ABX tools out there. If you want to really know what you're hearing, or what you think you're hearing, this is it.

In the meantime, posting your files on here can still be very valuable, and I'm looking forward to them!