Quote:
Originally Posted by
AudioRecorder
All I know is I did a Null test with a mp3 & a wave & it is ********! There is a constant noise floor that never goes away on an mp3. It is disturbing how bad mp3s are actually are!

It's often said that a null-test is as much a test of the person creating the test as anything. There are so many variables at play that it is difficult enough for a lot of folks to get truly identical files to null. Any difference in volume/level will make an enormous impact on the difference file. Even as much as .1 dB difference in levels can really shift the results. Additionally, although it's likely not an issue in this case, many people do not realize that audio encoded by different AD devices will be unlikely to null (because of even very, very tiny timing differences between their crystal controlled clock circuits).
Additionally, anyone who understands what an Mp3 is, should not expect complete nulling under even the most rigorously and correctly applied methodology.
Mp3s, AACs, WMAs, Ogg and others are called
perceptual encoding for a reason. They throw out large amounts of data, keeping 'just enough' in order to deliver the perception of relative accuracy (at a given bitrate).
The
proper test of perceptual encoding is -- wait for it --
perceptual.
That is to say, we already know the files are far from identical -- but we are
hoping that they may sound as similar as possible.
For
that, we resort to the time-honored tradition of double blind testing perceptual preference testing.
If we're trying to gauge how a given codec affects the general popluation, we'll randomly select a pool of people, a large enough sample that we can interpolate results to the general population with a given degree of accuracy.
If, on the other hand, we are, as individuals, just seeing how the encoding affects our own perception, the job is much easier. Carefully prepare the comparison files -- and, here, too, volume is critically important, as the ear is extremely sensitive to changes in volume... as little as .3 dB difference has shown to be recognizable by those with trained hearing and even lay people seem to be able to identify files with as little as .4-.5 dB difference. So getting those levels between the A and B files as close to identical as possible is critical for proper results.
Anyone interested in playing around with double blind ABX testing can use the ABX test comparator utility -- it's quite good -- in the Foobar media player. (Sorry, darkside only.)