The No.1 Website for Pro Audio
 Search This Thread  Search This Forum  Search Reviews  Search Gear Database  Search Gear for sale  Search Gearslutz Go Advanced
Launch of Pono Studio Headphones
Old 17th April 2014
  #3181
Motown legend
 
Bob Olhsson's Avatar
 

I meant it doesn't mask truncation distortion.
Old 17th April 2014
  #3182
Motown legend
 
Bob Olhsson's Avatar
 

Some DSD sounds better than some PCM, some PCM sounds better than some DSD. My conclusion has been that it's a matter of implimentation and not format. This year the best I've heard happened to be PCM.

Has anybody else noticed that combining the same manufacturer's A to D and D to A tends to sound better than mixing manufacturers?
Old 17th April 2014
  #3183
j_j
Lives for gear
Quote:
Originally Posted by Bob Olhsson View Post
I meant it doesn't mask truncation distortion.
Very true, it eliminates it. Agreed.
Old 17th April 2014
  #3184
j_j
Lives for gear
Just for hoots, I designed a 64 tap filter, and made the minimum phase version of it.

Attached are 64 tap constant delay, 64 tap minimum phase frequency responses, and of course a plot of the two impulse responses.

64movingav is what happens when you do a moving average that long.

As you can see, moving average has its problems with frequency response.

Remember, when you do things like "averaging" what you are doing is just applying bad lowpass filters.

BAD lowpass filters.
Attached Thumbnails
Launch of Pono-64cdelay.jpg   Launch of Pono-64filts.jpg   Launch of Pono-64minph.jpg   Launch of Pono-64movav.jpg  
Old 17th April 2014
  #3185
Quote:
Originally Posted by j_j View Post
Not sure what that's directed to. It's not, but it does create a flat noise floor (which is the goal), which does mask things too far below that noise floor, because it's noise.
I'm not sure exactly what your meaning is here. Dither doesn't simply mask things below a noise floor it creates. Maybe that's not what you meant, but if it is, that doesn't seem to be the case.

Now, I'm not an expert at these things, so I have to admit that some of this goes over my head, but according to Bob Katz and others I've read, dither actually preserves the precision of the dynamic range down below the signal to noise ratio of the target bit depth. Or something like that.


From Bob:
"Random numbers such as these translate to random noise (hiss) when converted to analog. The amplitude of this noise is around 1 LSB, lying at about 96 dB below full scale. By using dither, ambience and decay in a musical recording can be heard down to about -115 dB, even with a 16-bit wordlength. Thus, although the quantization steps of a 16-bit word can only theoretically encode 96 dB of range, with dither, there is an audible dynamic range of up to 115 dB!

The maximum signal-to-noise ratio of a dithered 16-bit recording is about 96 dB. But the dynamic range is far greater, as much as 115 dB, because we can hear music below the noise. Usually, manufacturer's spec sheets don't reflect these important specifications, often mixing up dynamic range and signal-to noise ratio. Signal-to-noise ratio (of a linear PCM system) is the RMS level of the noise with no signal applied expressed in dB below maximum level (without getting into fancy details such as noise modulation).
It should be, ideally, the level of the dither noise. Dynamic range is a subjective judgment more than a measurement--you can compare the dynamic range of two systems empirically with identical listening tests. Apply a 1 kHz tone, and see low you can make it before it is undetectable. You can actually
measure the dynamic range of an A/D converter without an FFT analyzer. All you need is an accurate test tone generator and your ears, and a low-noise headphone amplifier with sufficient gain. Listen to the analog output and see when it disappears (use a real good 16 bit D/A for this test). Another important test is to attenuate music in your workstation (about 40 dB) and listen to the output of the system with headphones. Listen for ambience and reverberation; a good system will still reveal ambience, even at that low level. Also listen to the character of the noise--it's a very educating experience."
Old 17th April 2014
  #3186
Lives for gear
 
GJ999x's Avatar
Still following this quite interesting thread...

Just occured to me how ironic it is that so many plugs have a "hiss" button - some of them dont even have a button to turn hiss off! - meanwhile we discuss -96db vs. -116db noisefloors and 24 bit recording etc...
Old 17th April 2014
  #3187
Lives for gear
 
paul brown's Avatar
Quote:
Originally Posted by Sounds Great View Post

I haven't to this day been provided with proper documentation of any test that was done and the details of how the test was conducted. Just talk that "lots of people were tested and they couldn't tell the difference".
i provided you with a link to a test. it included the details of how the test was conducted. what did you not like about it?
Old 17th April 2014
  #3188
Gear Guru
 
Sounds Great's Avatar
 

Quote:
Originally Posted by paul brown View Post
i provided you with a link to a test. it included the details of how the test was conducted. what did you not like about it?
Ok, I revisited that again. Sure wish I would have been there, I find that putting a 44.1 loop into the stream kind of a strange way to do it. And wouldn't that add a delay as well? Is the loop going through the A/D and D/A simultaneously?

Regardless, there are some facts that seem to support differences.

Quote:
The “best” listener score, achieved one
single time, was 8 for 10, still short of the desired 95%
confidence level. There were two 7/10 results.
7 out of 10 and 8 out of 10 seem to show some people are able to tell even under these circumstances.

Quote:
In one brief test with two subjects we added 14 dB of
gain to the reference level quoted and tested the two
sources with no input signal, to see whether the noise level
of the CD audio channel would prove audible. Although
one of the subjects was uncertain of his ability to hear the
noise, both achieved results of 10/10 in detecting the CD
loop. (We have not yet determined the threshold of this
effect. With gain of more than 14 dB above reference,
detection of the CD chain’s higher noise floor was easy,
with no uncertainty. Tests with other subjects bore this out.)
Sure seems like any time there were differences found, they downplayed that in every case. They found a difference so they kept that test brief and didn't bother to explore further. Yes, it sure looks like the people who put together this test were biased.

And yet they are forced to admit:
Quote:
though we did find one or two productions in which there was a detectable
difference in room tone at gain settings of +20 dB or more
above the reference level. At these very high gains we
could also hear subtle low-level decoding errors in all but
the most expensive of the high-resolution players.
"Subtle" "In all but the most expensive of the high-resolution players."

BUSTED!

And if that weren't enough:

Quote:
Though our tests failed to substantiate the claimed advantages
of high-resolution encoding for two-channel audio,
one trend became obvious very quickly and held up
throughout our testing: virtually all of the SACD and
DVD-A recordings sounded better than most CDs—
sometimes much better. Had we not “degraded” the sound
to CD quality and blind-tested for audible differences, we
would have been tempted to ascribe this sonic superiority
to the recording processes used to make them.
Plausible reasons for the remarkable sound quality of
these recordings emerged in discussions with some of the
engineers currently working on such projects. This portion recordings at least, seems to lie not in the high-bit recording
but in the high-bit market.
"virtually all of the SACD and
DVD-A recordings sounded better than most CDs—"

So what they really were testing for was to prove that CD was adequate in most listening environments as far as dynamic range and frequency response range. Really? Except that isn't the question.

Thanks for this test, it actually proves what I have been saying.
Old 17th April 2014
  #3189
Motown legend
 
Bob Olhsson's Avatar
 

Quote:
Originally Posted by edwinhurwitz View Post
I'm not sure exactly what your meaning is here. Dither doesn't simply mask things below a noise floor it creates. Maybe that's not what you meant, but if it is, that doesn't seem to be the case...
That's not what he meant. He was clarifying my uber-overly broad statement about dither not masking anything. More noise does indeed mask things way down. Bob Katz and I both learned at least 90% of what we know about digital audio from JJ's posts during the decade before he got silenced by non-disclosure agreements.
Old 17th April 2014
  #3190
Lives for gear
 
bogosort's Avatar
Quote:
Originally Posted by Sounds Great View Post
Regardless, there are some facts that seem to support differences.

7 out of 10 and 8 out of 10 seem to show some people are able to tell even under these circumstances.
You're misinterpreting the statistics. Take a fair coin and flip it 10 times. You'd expect to see 5 heads and 5 tails (though there's a good chance it won't work out that way). What is the probability of getting 8 heads out of the 10? About 5%. Well, repeat the process 100 times and you're almost guaranteed to get 8 out of 10 heads at least once, purely by random luck.

Only in the aggregate can you derive a statistically meaningful conclusion.

Quote:
Sure seems like any time there were differences found, they downplayed that in every case. They found a difference so they kept that test brief and didn't bother to explore further. Yes, it sure looks like the people who put together this test were biased.
They kept that test brief because they were listening to the noise floor (no signal) turned way up. Talk about fatiguing! In any case, it's not surprising that the 16-bit noise floor could be heard first -- this is exactly what one would expect. How does this demonstrate bias?

Quote:
"Subtle" "In all but the most expensive of the high-resolution players."

BUSTED!
What, exactly, has been "busted"? The authors claim to hear subtle decoding errors at very high gains in the cheaper high res players. Big deal?

Quote:
"virtually all of the SACD and
DVD-A recordings sounded better than most CDs—"
Their wording is not entirely clear, but I take it that they're referring to different masters. They specifically state that the superiority is not due to high resolution recording, and then suggest it is due to the "high-bit market". A reasonable interpretation is that SACDs and DVD-As, which are specifically marketed to audiophile types, have been more carefully remastered.

Whatever their meaning, it is clear that their comments are less about the format and more about the intent of those involved. How you can take this as some kind of technical win for DSD is beyond me.

Quote:
Thanks for this test, it actually proves what I have been saying.
You're so biased that you actually believe this!
Old 17th April 2014
  #3191
Quote:
Originally Posted by Bob Olhsson View Post
That's not what he meant. He was clarifying my uber-overly broad statement about dither not masking anything. More noise does indeed mask things way down. Bob Katz and I both learned at least 90% of what we know about digital audio from JJ's posts during the decade before he got silenced by non-disclosure agreements.
Good to know! I'm sure you guys have forgotten more than I'll ever know about it. My purpose in posting was not so much to question someone else's expertise as it was to try to see if someone could respond and clarify for my sake. As I said, I'm far from an expert and I do have much gratitude for the experts who participate.

Thanks for everything, JJ (and Bob)!
Old 17th April 2014
  #3192
Gear Guru
 
UnderTow's Avatar
Quote:
Originally Posted by Sounds Great View Post
7 out of 10 and 8 out of 10 seem to show some people are able to tell even under these circumstances.
If you run the test often enough someone will score 10/10 just by pure luck even without listening. That is why you have to look at all the results to get any meaningful understanding out of them.

Quote:
Sure seems like any time there were differences found, they downplayed that in every case.
The reference level used was 85 dB SPL for a -17 dB FS RMS source. 14 dB above reference would mean around 100 dB SPL reference level for this signal and much more for program material!

They didn't downplay anything. They understand what they are doing so they understand what the results do and do not mean. They are just trying to convey that understanding to the readers. Some get it. Some don't.

Quote:
They found a difference so they kept that test brief and didn't bother to explore further. Yes, it sure looks like the people who put together this test were biased.
They just turned the playback system (with no music playing) up until they heard the noise of the system. Surprise surprise a 16 bit format has a higher noise floor than a 24 bit format. I think everyone in this thread already knew that.

Quote:
And yet they are forced to admit:


"Subtle" "In all but the most expensive of the high-resolution players."

BUSTED!
We are now even higher compared to reference levels. And they are talking about listening purely to the room tones (meaning tails of room reverbs etc). You wouldn't want to accidentally skip to a piece of music when doing this kind of test...

On the other hand it seems that most of the so called "hi-rez" players had encoding/decoding errors. Those devices were busted. Welcome to the audiophile world.

Quote:
And if that weren't enough:

"virtually all of the SACD and
DVD-A recordings sounded better than most CDs—"
Yeah different masters sound different. You knew that already, right?

Quote:
Thanks for this test, it actually proves what I have been saying.
No it absolutely does not. All this proves is that you are unwilling to accept any evidence that might indicate you are wrong.

But hey, 24 bit is not a problem. If people want to pay extra for it that's fine by me. We need that in our studio converters anyway. 48 Khz is also fine. I need to for all my film and TV work. As far as I am concerned 44.1 Khz can be ditched completely once physical CDs are truly obsolete. It is the higher sample rates (and DSD) that make no sense for delivery formats.

Alistair
Old 17th April 2014
  #3193
Gear Guru
 

Quote:
Originally Posted by Sounds Great View Post
And wouldn't that add a delay as well?
if it did, wouldn't that be a possible clue for "cheating"? Shouldn't the scores be higher if cheating was an option?

Quote:

Quote:
The “best” listener score, achieved one
single time,
was 8 for 10, still short of the desired 95%
confidence level. There were two 7/10 results.
Quote:
7 out of 10 and 8 out of 10 seem to show some people are able to tell even under these circumstances.
Nearly 100 people took the test! Over the course of a year, ONE guy got it almost right! Two more did a little 'better than chance'. If the test was about coin flipping, I am sure you would have some very similar outliers in a group of that size.

Quote:
The test results for the detectability of the 16/44.1 loop on SACD/DVD-A playback were the same as chance:
49.82%. There were 554 trials and 276 correct answers
.


as to the "circumstances"- what is your quibble there? The test was performed on top equipment in professional mastering studios and University level purpose-designed listening spaces. Levels were matched to within .01dB. Are you saying the converters at the mastering studio are vastly inferior to what's inside the Pono?

the Beef in this thread is about delivery format. People do not crank up the noise floors in their delivery formats to levels that would blow their speakers should they suddenly skip to somewhere where there is music! If they did they would hear a little more noise floor in the 16 bit.

wow, stop the presses!
Old 17th April 2014
  #3194
Quote:
Originally Posted by Sounds Great View Post
Ok, I revisited that again. Sure wish I would have been there, I find that putting a 44.1 loop into the stream kind of a strange way to do it. And wouldn't that add a delay as well? Is the loop going through the A/D and D/A simultaneously?

Regardless, there are some facts that seem to support differences.



7 out of 10 and 8 out of 10 seem to show some people are able to tell even under these circumstances.



Sure seems like any time there were differences found, they downplayed that in every case. They found a difference so they kept that test brief and didn't bother to explore further. Yes, it sure looks like the people who put together this test were biased.

And yet they are forced to admit:


"Subtle" "In all but the most expensive of the high-resolution players."

BUSTED!

And if that weren't enough:



"virtually all of the SACD and
DVD-A recordings sounded better than most CDs—"

So what they really were testing for was to prove that CD was adequate in most listening environments as far as dynamic range and frequency response range. Really? Except that isn't the question.

Thanks for this test, it actually proves what I have been saying.
Your misunderstanding of the science of statistics is subverting your conclusions. j_j posted some probability tables a hundred or so posts back, I think, that show confidence levels from a range of results (up to 20 trials, as I recall) and that should give you a better feel for how the numbers work out.

Statistics can take more than a second to wrap one's head around, without question.


If you don't get inserting the 16/44.1 ADC/DAC step in to 'downgrade' the SACD sound to 'CD quality,' I honestly don't know what to tell you.

(And I'm perplexed why you would think the conversion delay of ~2 ms would matter in this case. In the tests no one listens to both signal simultaneously. What was your concern there?)
Old 17th April 2014
  #3195
Quote:
Originally Posted by Bob Olhsson View Post
That's not what he meant. He was clarifying my uber-overly broad statement about dither not masking anything. More noise does indeed mask things way down. Bob Katz and I both learned at least 90% of what we know about digital audio from JJ's posts during the decade before he got silenced by non-disclosure agreements.
Guys like Bob and j_j know a lot and sometimes may assume that we're more on top of what is being discussed than some or maybe even many of us necessarily are.

I know I totally misinterpreted something Bob said some score posts ago and then, pondering later posts by him, realized I'd probably misunderstood. Sure enough, I went back and there was a twist of semantic logic I'd failed to negotiate. He said it right; I misunderstood -- I completely misinterpreted what he had said and derived a pretty much opposite meaning.
Old 17th April 2014
  #3196
j_j
Lives for gear
Quote:
Originally Posted by edwinhurwitz View Post
I'm not sure exactly what your meaning is here. Dither doesn't simply mask things below a noise floor it creates.
I mean what I said.

The purpose of dither is not to mask anything. What dither does is eliminate signal-correlated distortions due to quantization.

HOWEVER the noise created by dither, which must exist, DOES mask signal.

If you look back in this thread, I also explain how you hear "through" noise in order to hear tones below the noise floor.

Perhaps some research is in order here?
Old 17th April 2014
  #3197
j_j
Lives for gear
Quote:
Originally Posted by UnderTow View Post
If you run the test often enough someone will score 10/10 just by pure luck even without listening. That is why you have to look at all the results to get any meaningful understanding out of them.
Yeah, that was covered about 20 pages or so ago.

The chance of getting 10/10 right by random chance is 1/1024, exactly. So if you run 1000 tests you'll probably see at least one.

Again, the chance of scoring 0 through 10 by random chance in 10 trials follows. This is already posted in this thread (and UT has noticed it, but apparently some others are ignoring it) once, but apparently it requires repeating.

0 right 0.000977 chance in 10 trials
1 right 0.009766 chance in 10 trials
2 right 0.043945 chance in 10 trials
3 right 0.117188 chance in 10 trials
4 right 0.205078 chance in 10 trials
5 right 0.246094 chance in 10 trials
6 right 0.205078 chance in 10 trials
7 right 0.117188 chance in 10 trials
8 right 0.043945 chance in 10 trials
9 right 0.009766 chance in 10 trials
10 right 0.000977 chance in 10 trials


What does this mean? It means that if you run 20 tests, each involving 10 trials, you expect at one person to get 8/10 or better, and 2 or 3 to get 7 or better.

Yeah, let's add a column to that printout, come to think of it.

0 right 0.000977 chance in 10 trials 0.999023 chance of doing better than 0 right
1 right 0.009766 chance in 10 trials 0.989258 chance of doing better than 1 right
2 right 0.043945 chance in 10 trials 0.945312 chance of doing better than 2 right
3 right 0.117188 chance in 10 trials 0.828125 chance of doing better than 3 right
4 right 0.205078 chance in 10 trials 0.623047 chance of doing better than 4 right
5 right 0.246094 chance in 10 trials 0.376953 chance of doing better than 5 right
6 right 0.205078 chance in 10 trials 0.171875 chance of doing better than 6 right
7 right 0.117188 chance in 10 trials 0.054688 chance of doing better than 7 right
8 right 0.043945 chance in 10 trials 0.010742 chance of doing better than 8 right
9 right 0.009766 chance in 10 trials 0.000977 chance of doing better than 9 right
10 right 0.000977 chance in 10 trials 0.000000 chance of doing better than 10 right

So there you have it. Now there is a column that tells you the chance you'll do better than 'n' right by RANDOM CHANCE. Note that you have a really good chance of doing better than 6 right, in fact that's about 1 in 6.

So no, a few 7's and 8's in a test of 100 subjects and 10 trials means nothing. In fact, in 100 subjects, one would expect a good chance of seeing one 9/10 and it wouldn't be unthinkable to see 10/10 once.
Old 17th April 2014
  #3198
@j_j
How would a test analysis (and test design) account for cognitive bias in terms of testees who believe no difference exists or testees who believe they can hear a difference (but who cannot)? Wouldn't random guesses skew the significance of the results?
Would a different experiment design using a chi analysis and testee groupings yield a more precise result? e.g. 1/expert testees who can regularly discern differences; 2/ testees who believe they can discern a difference; 3/ testees who believe they will not discern a difference; and, 4/ testees unsure as to whether they will discern a difference; (if in theory a difference does exist and is discernable)..
Old 17th April 2014
  #3199
Lives for gear
 
3rd Degree's Avatar
 

Quote:
Originally Posted by j_j View Post
So there you have it. Now there is a column that tells you the chance you'll do better than 'n' right by RANDOM CHANCE. Note that you have a really good chance of doing better than 6 right, in fact that's about 1 in 6.

So no, a few 7's and 8's in a test of 100 subjects and 10 trials means nothing. In fact, in 100 subjects, one would expect a good chance of seeing one 9/10 and it wouldn't be unthinkable to see 10/10 once.
I don't want to down play probability but I mentioned when I was doing my tests that the more confident I was, if I got 8/10 or 7/10, I found that I would get 7 or 8 right in a row. I would either get the first and last wrong, or the last two wrong when I got an 8. I think that should be accounted for as when I was truly guessing, my results were far more random. I also picked things out far more quickly.

I don't want to get in another debate about this, I am just stating what I learned from doing my own tests and if I were going to draw conclusions, I would likely not count the first and last. Ironically, when comparing things that are identical outside of format, basically the same file just at a lower or higher resolution, I found listening fatigue was a very high factor, even when it was obvious so a lengthier test may actually be hurtful as well. Again, not trying to debate, just sharing my own experience here. On one side, 7/10 can be easily guessed randomly, but sometimes, 7/10 is extremely hard to get when you have a confident educated "guess".
Old 17th April 2014
  #3200
j_j
Lives for gear
Quote:
Originally Posted by 3rd Degree View Post
I don't want to down play probability but I mentioned when I was doing my tests that the more confident I was, if I got 8/10 or 7/10, I found that I would get 7 or 8 right in a row. I would either get the first and last wrong, or the last two wrong when I got an 8. I think that should be accounted for as when I was truly guessing, my results were far more random. I also picked things out far more quickly.
Well, if you did 128 total trials, chances are you got 7 in a row at least once by random chance.

But what you should do is go back and repeat the trials where you got 7/10 or 8/10 a bunch more times.

If you CONTINUE to get 8/10 repeatedly, then you've got something.

8/10 by itself will happen randomly 1 of 20 times, give or take,

BUT

16/20 is a bit more rare.

Table follows.

0 right 0.000001 chance in 10 trials 0.999999 chance of doing better than 0 right
1 right 0.000019 chance in 10 trials 0.999980 chance of doing better than 1 right
2 right 0.000181 chance in 10 trials 0.999799 chance of doing better than 2 right
3 right 0.001087 chance in 10 trials 0.998712 chance of doing better than 3 right
4 right 0.004621 chance in 10 trials 0.994091 chance of doing better than 4 right
5 right 0.014786 chance in 10 trials 0.979305 chance of doing better than 5 right
6 right 0.036964 chance in 10 trials 0.942341 chance of doing better than 6 right
7 right 0.073929 chance in 10 trials 0.868412 chance of doing better than 7 right
8 right 0.120134 chance in 10 trials 0.748278 chance of doing better than 8 right
9 right 0.160179 chance in 10 trials 0.588099 chance of doing better than 9 right
10 right 0.176197 chance in 10 trials 0.411901 chance of doing better than 10 right
11 right 0.160179 chance in 10 trials 0.251722 chance of doing better than 11 right
12 right 0.120134 chance in 10 trials 0.131588 chance of doing better than 12 right
13 right 0.073929 chance in 10 trials 0.057659 chance of doing better than 13 right
14 right 0.036964 chance in 10 trials 0.020695 chance of doing better than 14 right
15 right 0.014786 chance in 10 trials 0.005909 chance of doing better than 15 right
16 right 0.004621 chance in 10 trials 0.001288 chance of doing better than 16 right
17 right 0.001087 chance in 10 trials 0.000201 chance of doing better than 17 right
18 right 0.000181 chance in 10 trials 0.000020 chance of doing better than 18 right
19 right 0.000019 chance in 10 trials 0.000001 chance of doing better than 19 right
20 right 0.000001 chance in 10 trials 0.000000 chance of doing better than 20 right

Note "better than 15 of 20" is something like 1 of 160. And so on.

Doing the same set of 10 trials over and over will determine if you were answering randomly or not.
Old 17th April 2014
  #3201
j_j
Lives for gear
Quote:
Originally Posted by Arthur Stone View Post
@j_j
How would a test analysis (and test design) account for cognitive bias in terms of testees who believe no difference exists or testees who believe they can hear a difference (but who cannot)? Wouldn't random guesses skew the significance of the results?
Would a different experiment design using a chi analysis and testee groupings yield a more precise result? e.g. 1/expert testees who can regularly discern differences; 2/ testees who believe they can discern a difference; 3/ testees who believe they will not discern a difference; and, 4/ testees unsure as to whether they will discern a difference; (if in theory a difference does exist and is discernable)..

That's why you include control stimulii. A negative control is comparing a to a, and that better come out random. A positive control is something that you should be able to hear, and should not come out random.

But the key is to find people who get scores above media, and retest them with the same sets of stimulii (but new randomization) over and over, and see if that continues to be true.

There are lots of other kinds of analysis one can do, but one has to be very careful not to cherrypick data.

If I get to pick the right 10 sets of results out of 100 tests, I can prove anything I want. That doesn't actually PROVE anything, of course, it proves that if one chooses the lucky (or unlucky) one "proves" whatever one set out to show, but the statistics are invalid.

If somebody does 8/10 on a given set of 10 stimulii, run them again on the same stimulii. See what happens the next time. Of course, give them rest, etc. 10 trials at once is MORE than enough trials between rest periods.
Old 17th April 2014
  #3202
j_j
Lives for gear
Ok, people, I think it's time for those who care to load octave (plus the signal processing and audio packages) on their own computer. It's freeware from sourceforge and the octave project. It's free, and is available for at least windblows and apple, if I recall, and probably for linux, too.

Here is the script to calculate binary distribution. Note: it will start to get "off" a bit if you go over 100 trials, I did this the quick and dirty way. But at 100 trials you had better have a clear answer, too!

nt is the "number of trials". I did not make the file name fancy and elegant, have at if you wish.

If you want to know what the outcome for a less random result is, you can, for instance, use
p=[.3 .7];
and that will show you the distribution that comes about if you have .7 chance of getting the right answer.

Note, this is only for a binary (i.e. choice between two items) distribution.

clc;
clear all;
close all;
nt=20;

p=[.5 .5];

y=1;

for ii=1:nt
y=conv(y,p);
end

fid=fopen('bin10.txt','w');

for ii=1:(nt+1)
tot=1-sum(y(1:ii));
fprintf(fid,' %d right %f chance, %f chance of doing better than %d right\n',ii-1,y(ii), tot, ii-1);
end
fclose(fid)
Old 17th April 2014
  #3203
Thanks for your response j_j: my thinking with the groupings is that if a discernable difference does exist then the expert group will show significance whereas the other groups are more likely to be random, with perhaps a little more significance for the agnostics. I'm not trying to prove anything but rather develop a test which will account for innate listening abilities and bias from beliefs. I much appreciate your comments. Thank you.

EDIT: maybe I can rephrase that to say the question I'm addressing is not whether a difference (between A and B) can be discerned at a general population level but whether individuals/groups within a population exhibit a range of hearing abilities.
Old 17th April 2014
  #3204
Gear Guru
 
Sounds Great's Avatar
 

Quote:
Originally Posted by theblue1 View Post

(And I'm perplexed why you would think the conversion delay of ~2 ms would matter in this case. In the tests no one listens to both signal simultaneously. What was your concern there?)
I suppose. But if I was setting up the test I would have both signals running simultaneously and switch between them with no shift in time or gap between the two sources, continuous listening with a seamless switch between them.
Old 17th April 2014
  #3205
Airwindows
 
chrisj's Avatar
I hope that, since we can establish statistically that virtually NO positive result is 'proof' of hearing sonic differences (merely probability that can be put down to good luck),we can eventually move on to listening to stuff and judging it good or bad more informally. I still hope to hear what Pono does with sound, though I've missed the kickstarter and will have to pay full price one day to get one.
Old 17th April 2014
  #3206
Lives for gear
 
3rd Degree's Avatar
 

Quote:
Originally Posted by chrisj View Post
I hope that, since we can establish statistically that virtually NO positive result is 'proof' of hearing sonic differences (merely probability that can be put down to good luck),we can eventually move on to listening to stuff and judging it good or bad more informally. I still hope to hear what Pono does with sound, though I've missed the kickstarter and will have to pay full price one day to get one.
Going further, I think it's fair to say that perception vs. proof is not all that important when you are talking about a product, both from the seller's perspective, and the buyer's perspective. If the buyer feels good about the purchase, it really doesn't matter how it compares. I would imagine very few who really want this are directly comparing it to other products on the market to make a decision.
Old 17th April 2014
  #3207
Quote:
Originally Posted by Sounds Great View Post
I suppose. But if I was setting up the test I would have both signals running simultaneously and switch between them with no shift in time or gap between the two sources, continuous listening with a seamless switch between them.
OK... clearly you didn't understand how the test is set up.

There is ONLY ONE SOURCE.

This completely gets around the entire issue of whether or not a given test SACD or DVD-A source is the same as a 16/44.1 CD-A version of what we might hope is the same mix/master.

It does put an 'extra burden' on the 16/44.1 part of the test -- an extra round of conversion -- the SACD or DVD-A DAC followed by the ADC/DAC of the 16/44.1 insert.

Now -- if the test results had shown with statistical significance a higher than expected (by random result) of those who could differentiate, it might have been appropriate to find a rest method that didn't so-burden the lowly CD-audio 'contestant.' But since the statistical analysis showed that even that higher burden didn't make the CD-audio discernible from the straight DSD and DVD-A audio, there's not much reason to reach for a method that would be make it 'easier' for the CD-A.

Get it?

Old 17th April 2014
  #3208
j_j
Lives for gear
Quote:
Originally Posted by Arthur Stone View Post
Thanks for your response j_j: my thinking with the groupings is that if a discernable difference does exist then the expert group will show significance whereas the other groups are more likely to be random, with perhaps a little more significance for the agnostics. I'm not trying to prove anything but rather develop a test which will account for innate listening abilities and bias from beliefs. I much appreciate your comments. Thank you.

EDIT: maybe I can rephrase that to say the question I'm addressing is not whether a difference (between A and B) can be discerned at a general population level but whether individuals/groups within a population exhibit a range of hearing abilities.
***ALWAYS*** USE TRAINED, EXPERT LISTENERS.

Yes, I am shouting.

Why? Because when you sell it to the population, they'll eventually be trained. Eventually. But the results may just turn out to be disinterest or "does not buy" rather than explicit "this problem exists".
Old 17th April 2014
  #3209
j_j
Lives for gear
Quote:
Originally Posted by 3rd Degree View Post
Going further, I think it's fair to say that perception vs. proof is not all that important when you are talking about a product, both from the seller's perspective, and the buyer's perspective. If the buyer feels good about the purchase, it really doesn't matter how it compares. I would imagine very few who really want this are directly comparing it to other products on the market to make a decision.
Seriously, if you can re-run the stimuli that you did well on, see if you do well the next time.

If you can get 8/10 on each of 5 sets, you've pretty much shown that you can indeed hear something. Even 8/10 on 2 sets is pretty conclusive.

But just add up all the successes and all of the trials. That's how to calculate the statistics. Otherwise you could hide some marginal results.
Old 17th April 2014
  #3210
Motown legend
 
Bob Olhsson's Avatar
 

Quote:
Originally Posted by j_j View Post
***ALWAYS*** USE TRAINED, EXPERT LISTENERS.

Yes, I am shouting.

Why? Because when you sell it to the population, they'll eventually be trained. Eventually. But the results may just turn out to be disinterest or "does not buy" rather than explicit "this problem exists".
This is possibly the most important post on the subject of audio quality ever!
Topic:
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
Thread Tools
Search this Thread
Search this Thread:

Advanced Search
Forum Jump
Forum Jump