The No.1 Website for Pro Audio
 Search This Thread  Search This Forum  Search Reviews  Search Gear Database  Search Gear for sale  Search Gearslutz Go Advanced
Science Versus Gearslutz
Old 23rd December 2008
  #1
Gear Head
 

Science Versus Gearslutz

I never really considered myself a true audiophile since I don't own any fancy $1000 per foot cables or color my CDs with a green felt tip marker. However I was accused of being one recently because I appreciate highend gear (mytek converters and DAV preamps) and ignore the science specifically blind testing. I was accused of having bias influence the way I perceive things.

I shrugged this off but as I poked around the internet I realized that you can't embrace both double blind testing and be a gearslut. So I hereby totally renounce double blind testing? Why? See the link to the peer reviewed study below:

Richard Clark Amplifier Challenge FAQ

Through testing of thousands of people (including some well known ears) the conclusion it came to is that all modern amps (and by incorporation the audio compenents like op-amps, caps, powersupplies within them) sound indistinguishable from each other.

Concluding quote:

"When compared evenly, the sonic differences between amplifiers operated below clipping are below the audible threshold of human hearing."

I definitely can hear differences between amps, CD players, mic amps, converters, etc. I don't believe in double blind testing anymore.
Old 23rd December 2008
  #2
Lives for gear
 
steveschizoid's Avatar


(sorry Tony!)
Old 23rd December 2008
  #3
Gear Head
 

While it's true that the double blind studies provide evidence which shows that all modern audio components designed to be transparent from amps to CD players, to DACs, op-amps etc. are indistinguishable when driven within their linear range there are other reasons to buy gear besides sonic quality. Some other reasons include features, customer support, reliability, build quality, and status/name recognition.

Schizoid - cool avatar. What is it?
Old 23rd December 2008
  #4
Lives for gear
 

Quote:
Originally Posted by Gary Schmitt View Post
Richard Clark Amplifier Challenge FAQ

Through testing of thousands of people (including some well known ears) the conclusion it came to is that all modern amps (and by incorporation the audio compenents like op-amps, caps, powersupplies within them) sound indistinguishable from each other.

Concluding quote:

"When compared evenly, the sonic differences between amplifiers operated below clipping are below the audible threshold of human hearing."

I definitely can hear differences between amps, CD players, mic amps, converters, etc. I don't believe in double blind testing anymore.
Again, I would venture that making music is not a game of musical jeopardy.

The ability to consistently correctly identify mics, pres, converters, guitars, drums, etc just by listening to a track is a neat parlor trick, but of little practical use.

Making music is about learning how your gear performs over time and applying it as best suits the strengths it demonstrates to you over that time.

It is not a circus sideshow 'hot seat' guessing game.

I'm sure you could similarly 'prove' a number of major guitar amps are indistinguishable and thus scientifically identical. eg.
YouTube - Soldano Avenger 100 vs. Cornford MK50h

But any guitar player would laugh, because such a finding would have no basis in the practical reality of music making.
Old 23rd December 2008
  #5
Moderator
 
TonyBelmont's Avatar
 

Quote:
Originally Posted by steveschizoid View Post


(sorry Tony!)






heh
Old 23rd December 2008
  #6
Lives for gear
 
big country's Avatar
 

so your saying , if your listing to music its not like thinking

Oh . thats 335

oh thats tele


pre amps and converters aren't the same?
Old 23rd December 2008
  #7
Lives for gear
 

Quote:
Originally Posted by big country View Post
so your saying , if your listing to music its not like thinking

oh thats tele
Ya, but what year is the tele? What pickups, what strings, Mexican or USA? Can you hear and identify all that blind?

If you can't does it mean all teles and tele set-ups sound the same?
Old 23rd December 2008
  #8
Interesting read, but you're right: double-blind tests are often meaningless, as they don't - by design - have much to do with real world usage.

Can I tell the difference between a Neve 1084 and an API 512b? I sure can, but I do so by hearing the way they react to being pushed by higher levels, much the same way I use them. How they sound when runnin' cold is of little use OR interest to me.

Those kinds of tests are a waste of time. Audiovisceral nailed it quite eloquently:

Quote:
Making music is about learning how your gear performs over time and applying it as best suits the strengths it demonstrates to you over that time.
Old 23rd December 2008
  #9
Gear Maniac
 
Gearhero's Avatar
 

Quote:
Originally Posted by audiovisceral View Post

I'm sure you could similarly 'prove' a number of major guitar amps are indistinguishable and thus scientifically identical. eg.
YouTube - Soldano Avenger 100 vs. Cornford MK50h

But any guitar player would laugh, because such a finding would have no basis in the practical reality of music making.
We are talking about linear amps here but I'll bite...if a thousand different guitarists tweaked and played through both amps and no one could tell the difference then I would say that both those amps aren't going to sound very different when tracked.

To ignore double blind testing is foolish. I'll be the first to admit that I wasted thousands on different 'flavors' of clean pres and converters. I wish someone had enlightened me earlier.

Oh and by definition Gearslutz are audiophiles!!!!
Old 23rd December 2008
  #10
Lives for gear
 
Chaellus's Avatar
Quote:
Originally Posted by Gearhero View Post
We are talking about linear amps here but I'll bite...if a thousand different guitarists tweaked and played through both amps and no one could tell the difference then I would say that both those amps aren't going to sound very different when tracked.

To ignore double blind testing is foolish. I'll be the first to admit that I wasted thousands on different 'flavors' of clean pres and converters. I wish someone had enlightened me earlier.

they will sound diffrent...just because you cant hear the diffrence doesnt mean there isnt a diffrence
Old 23rd December 2008
  #11
Lives for gear
 

Double blind testing is a form of statistics. You know ... lies, damned lies, and then statistics ...

The purpose of most statistics is to prove a point, usually to sell an idea to somebody. As such, they can be - and routinely are - twisted to serve the agenda of the promoter.

In a business situation - when somebody asks me to prepare some statistics or a graph or something, I ask them "what do you want it to say?" - even before I have the data. Because I know that I can present the data practically any way without falsifying the data to prove whatever point is required. Or at least, present it in the best possible light, if the data is contrary to the desired result.

When I audition an amp or piece of audio kit, I'm very conscious of noise floor artifacts and headroom issues, and other things that make me reject, tolerate or love that item.

But if I had to sell a piece of kit that I would personally reject - I would know how to present it in the best possible light. A noisy hall full of people would be a very good start. Perhaps a classic vinyl track that has a bed of tape hiss to further disguise the defects in the system. More devious people might have hidden eq's. But the simplest thing is to always ensure that the chosen winner of the test is half a dB louder than the others. Works like a charm.

There are a bunch of psychological and hypnotic tricks that can twist the punters perception too. The whole branch of science is called Industrial Psychology - and it works.

Never trust a double blind test for those reason. But it sure ships a lot of product.
Old 23rd December 2008
  #12
Lives for gear
 

Quote:
Originally Posted by Gearhero View Post
We are talking about linear amps here but I'll bite...if a thousand different guitarists tweaked and played through both amps and no one could tell the difference then I would say that both those amps aren't going to sound very different when tracked.
You cannot tweak in the type of double blind testing we are talking about.

Thus the conclusion of such a test would most likely be Cornford = Soldano, and almost every guitarist I know would laugh.

YouTube - Soldano Avenger 100 vs. Cornford MK50h

Apparently, Soldanos are virtually indistinguishable from Mesas too. Whodathunkit?

YouTube - Soldano Hot Rod 50 Avenger VS Mesa Single Rectifier

Better let all the guitar players out there know they're being scammed, and Cornford, Mesa, and Soldano are actually pretty much the same thing. I bet they'll be sooo pissed.
Old 23rd December 2008
  #13
Gear Addict
 
Renegade Prod's Avatar
 

Where's our genius Ethan Winer at? .... I need some 'enlightenment' with his Mackie and foam
Old 23rd December 2008
  #14
Lives for gear
 
Chaellus's Avatar
[quote=audiovisceral;3755774]You cannot tweak in the double blind testing


Old 23rd December 2008
  #15
Lives for gear
 
Chaellus's Avatar
i guess i miss qouted agian....ahh
Old 23rd December 2008
  #16
FAT
Gear Maniac
 

Quote:
Originally Posted by Renegade Prod View Post
Where's our genius Ethan Winer at? .... I need some 'enlightenment' with his Mackie and foam
From what I've seen, Ethan Winer is eroding any ounce of credibility he once had by producing R-13 soft porn to satisfy his burgeoning senility.

Old 23rd December 2008
  #17
Gear Addict
 
Renegade Prod's Avatar
 

Maybe he could show me how he turns a 57 through a TLA into a 414 through an SSL channel.

He can shove an elegant Q8 on it and then give me an mp3 to 'prove' his insight ... **** the ****
Old 23rd December 2008
  #18
FAT
Gear Maniac
 

Quote:
Originally Posted by Renegade Prod View Post
Maybe he could show me how he turns a 57 through a TLA into a 414 through an SSL channel.

He can shove an elegant Q8 on it and then give me an mp3 to 'prove' his insight ... **** the ****
he he
Old 23rd December 2008
  #19
Lives for gear
 
RCM - Ronan's Avatar
I actually have a degree in psychology with an emphasis on quantitative analysis, so feel free to take that into account in regard to my biases, but...

The value of double blind testing is that our expectation of the results will almost always alter how we perceive subjective experience. The end goal of double blind testing is not to be a parlor trick and show that you can identify that something is a Neve 1073, the value is to be able to listen to comparison between, say, an API pre amp and a Millennia pre amp and be able to evaluate them with out biases. If people know which is which it is a pretty safe bet that most people would describe the API as warmer, fatter, more aggressive.... and the Millennia as clean, transparent, natural, etc. But if you set up the test as double blind you would probably find people having rather mixed opinions as to where they applied those same descriptors and which might be better for certain applications.

Setting up double blind tests across a range of conditions is one of the best ways I know to evaluate gear.
Old 23rd December 2008
  #20
Gear Nut
 

Quote:
Originally Posted by audiovisceral View Post
Ya, but what year is the tele? What pickups, what strings, Mexican or USA? Can you hear and identify all that blind?

If you can't does it mean all teles and tele set-ups sound the same?
if you can't hear a difference, your ears are
if you own both and still can't hear a difference fuuck


+1 to rcm
Old 23rd December 2008
  #21
Lives for gear
 
Joram's Avatar
 

Who is Richard Clark?

I always do listening tests before I buy some sort of audio equipment. It's fun and it can save money. I never had the impression that there is no difference in tonal quality, puls response, dynamic ability etc.
I also did listening test with cable using white and pink noise: and yes, difference is audible. If people are asked to chose from a list of words to define the sound of a piece of equipment or the difference between them, it's remarkable how often they chose the same terms.

When I worked for Polygram is the Netherlands, one of the techical engineers told me that is was sometimes very difficult to measure the the differences the recording engineers heard. His idea was (and is) that the human ear and test equipment are sometimes incompetable.

Of course it's important to be sure that audiolevels during the test are even. The human ear is quite sensitive to difference in level.
Old 23rd December 2008
  #22
Lives for gear
 
Joram's Avatar
 

Something I forgot:
One a award-winning engineer/producer told me: it's not that I have better ears than someone else. It's psychological: my mind is just less tolerant to difference in sound.
Old 23rd December 2008
  #23
Lives for gear
 
Empire Prod's Avatar
 

So I guess all the hundreds of people that tried the Alesis RA500 power amp to drive their NS10's because it specs out as a perfect match for them, only to find out that it made them sound like complete crap, are all apparently delusional. Damn!!! I am one of them. Lock me up!!!
Old 23rd December 2008
  #24
Lives for gear
 

Quote:
Originally Posted by Fluner View Post
if you can't hear a difference, your ears are
if you own both and still can't hear a difference fuuck

+1 to rcm
Wow you can name which guitar strings players use just by listening? And you can identify by ear the country and year their guitars were made in? I'm guessing you can do pickups and guitar cable too, huh?

You must be Superman. Or if not, I guess none of those things matter either.
Old 23rd December 2008
  #25
Lives for gear
 

Finding one guys testresults (Clark) and writing of proper testing becasue of that is just silly.

Clark does not speak for the rest of the world or for science as such.

If one wants to argue about Clark's tests, the only proper thing to do is falsify his theory.

One also must remember that any test is only valid for that specific test situation (and could be true as such) but it takes only one positive to falsify a thousand negatives... in an absolute sense.

A test is never better than the people seting the test up. I know people that have performed plenty of tests on power amps (and other gear) and most amps (and other gear) color the sound audibly.

Other than that I agree with rcm in his post.


BTW, anyone know where allencollins is?


/Peter
Old 23rd December 2008
  #26
Lives for gear
 
ssaudio's Avatar
 

Quote:
Originally Posted by rcm View Post
Setting up double blind tests across a range of conditions is one of the best ways I know to evaluate gear.
Agreed 100%, well put.
Old 23rd December 2008
  #27
Lives for gear
 

Quote:
Originally Posted by Audiop View Post
BTW, anyone know where allencollins is?
He got banned from high end for annoying Jules and seems to have left.

Re: science, I am actually a scientist by education, but I do not consider music a generally scientific pursuit.

How do you scientifically prove one melody is better than another, or one compressor fits a track more appropriately than another? In theory, there might be ways to analyze, but why bother?

It is almost all dictated by mood and personal taste and may vary even from day to day.

As for hearing tests, I maintain they are trivia unless the person was actually able to sit down and use the unit as they would in studio and with enough time to get familiarized (eg. with a blank faceplate to maintain blind user status).

Identifying random unfamiliar units in unfamiliar musical contexts with no hands on tweaking allowed has little relevance to how music is made in real life.

I therefore fail to see the point.
Old 23rd December 2008
  #28
Lives for gear
 

Quote:
Originally Posted by audiovisceral View Post

Re: science, I am a scientist by education, but I do not consider music a scientific pursuit.
Me neither, but audio engineering should be IMO.


/Peter
Old 23rd December 2008
  #29
Lives for gear
 
midnightsun's Avatar
 

Science

A double blind test is simply a scientific tool. Using a double blind test on a theory no more guarantees science that using a tape recorder guarantees music. First and foremost a specific point must be focused and it should be fairly obvious that the cut and dry results of a test should determine if the point is true or not.

By the way, double blind means that both the tester and the test taker are blind to the specifics of the results. Most audio test were single blind, not double blind.

Many forms of analysis get statistical power from starting with the hypothesis that there is no difference between two sets of data-- "null hypothesis." Disproving the null hypothesis then allows one to say with a specific degree of confidence that there is actually a difference.

Given the study where people supposedly couldn't tell the difference between various amplifiers, all that is being determined is that the null hypothesis was not disproved given the design of the study.


An interesting study I think would be to put various preamps in boxes to blind the engineer as to the exact pre he was using. Would anyone argue that all preamps would perform identically in the hands of an engineer. I think that it would be very easy to disprove the null hypothesis that there is no difference between the boxes under these circumstances.
Old 23rd December 2008
  #30
Lives for gear
 

Quote:
Originally Posted by midnightsun View Post
An interesting study I think would be to put various preamps in boxes to blind the engineer as to the exact pre he was using. Would anyone argue that all preamps would perform identically in the hands of an engineer. I think that it would be very easy to disprove the null hypothesis that there is no difference between the boxes under these circumstances.
Another interesting test would be to take three amps and split them into their 6 distinct L-R channels. Lock the 3 left channels that would result to subjects' left ear and the 3 right channels to the right, but provide 1-2-3 dials to control which amp each side listens to at any time independently. Jumble the numerical amp references for each side, and let the listener to try and match up the proper left-right channels for each amp by cycling through on their own.

I have little doubt that with diverse amp selection, most listeners with good hearing would succeed in matching th L-R pairs. If so, this would completely invalidate the 'all amps are equal' conclusion of the first study.
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
Thread Tools
Search this Thread
Search this Thread:

Advanced Search
Similar Threads
Thread
Thread Starter / Forum
Replies
andre tchmil / So Much Gear, So Little Time
13
DanielaPetri / The Good News Channel
11
hduncan / So Much Gear, So Little Time
1
feyshay / So Much Gear, So Little Time
1

Forum Jump
Forum Jump