The No.1 Website for Pro Audio
 Search This Thread  Search This Forum  Search Reviews  Search Gear Database  Search Gear for sale  Search Gearslutz Go Advanced
a/b testing deceiving? Channel Strips
Old 5th September 2007
  #1
Gear Addict
 
daillman's Avatar
a/b testing deceiving?

in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it and then giving a good listen,all of this one channel against the other is not really giving a true picture of what the pre is all about.....ok you can move or attack me now....heh
Old 5th September 2007
  #2
Lives for gear
 
Bob Ross's Avatar
 

Quote:
Originally Posted by daillman View Post
in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it and then giving a good listen,all of this one channel against the other is not really giving a true picture of what the pre is all about

Your point is valid

...if what you're "testing for" is the ability to sound good when stacking multiple tracks.

However, if you're "testing for" the ability to sound good on individual tracks -- say, a solo performer, or a minimally-miked live-to-2track ensemble performance -- then A/B'ing individual channels is as good a way to form a subjective opinion as any.
Old 5th September 2007
  #3
Lives for gear
 
ISedlacek's Avatar
Quote:
Originally Posted by daillman View Post
in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it and then giving a good listen,all of this one channel against the other is not really giving a true picture of what the pre is all about.....ok you can move or attack me now....heh
What about if you want to record a solo instrument ? Not everybody is necessarily doing dense "mixes".

And also - if the difference is such that you are not sure if it is there at all and you must look for it with a microscope, it is probably not worth considering (so many devices are designed in quite a similar way today).

For me , only the AB difference which is obvious and easily detectable has some sense as regards a choice of a device

In the present overproduction of professional audio devices, it is not surprising that most of them behave and sound in quite similar way, without being a special, unique original each. Just some few rare gems are destined to make a real difference
Old 5th September 2007
  #4
Lives for gear
 
gainstages's Avatar
i agree with both sides of this argument so far, but the original point is extremely valid.

I've been involved in several tests with a preamp, compressor, even converters, where you compared 2 of them with seemingly no significant differance at all to most people when comparing a single track. however, as you begin to stack multiple tracks and build the project, you begin to realize more and more that one set of tracks sounds better than the other, with the end result being a mix that has great spectral balance, warmth, better imaging, or other desireable characteristics, whereas the other mix is clearly lacking in comparison.
Old 5th September 2007
  #5
Lives for gear
 
ssaudio's Avatar
 

maybe I'm weird, but I've never needed the results of or been remotely interested in the idea of these tests - why give oneself the grief, time's too short as it is...
Old 5th September 2007
  #6
Lives for gear
 
Audio Hombre's Avatar
 

Quote:
Originally Posted by gainstages View Post
ihowever, as you begin to stack multiple tracks and build the project, you begin to realize more and more that one set of tracks sounds better than the other, with the end result being a mix that has great spectral balance, warmth, better imaging, or other desireable characteristics, whereas the other mix is clearly lacking in comparison.
this is where i hear RADAR always shining.
Old 5th September 2007
  #7
Lives for gear
 
ISedlacek's Avatar
Quote:
Originally Posted by ssaudio View Post
maybe I'm weird, but I've never needed the results of or been remotely interested in the idea of these tests - why give oneself the grief, time's too short as it is...
It depends ... for someone using behringer and digi 001 within his busy short life spam, a sudden AB test comparing them to say Lavry and Telefunken V76 can be quite revealing ...
Old 5th September 2007
  #8
Lives for gear
 
ssaudio's Avatar
 

if they need to hear such a test there is obviously something far wrong.

I realise that the abundance of cheap gear that's available can cloud judgement etc, but anyone that is truly interested doesn't need to hear tests like that to know there is better out there
Old 5th September 2007
  #9
Dan
Lives for gear
 
Dan's Avatar
 

Yeah, anyone can know what's better by reading here.

Or you should just know how much better is better.

A/B tests are a starting point. I almost always feel differently about gear after living with it a while. Also, they are extremely aplication specific. There's so much gear that doesn't work for a particular sound source, and is magic on others.
Old 6th September 2007
  #10
Lives for gear
 
ISedlacek's Avatar
Quote:
Originally Posted by ssaudio View Post
if they need to hear such a test there is obviously something far wrong.

I realise that the abundance of cheap gear that's available can cloud judgement etc, but anyone that is truly interested doesn't need to hear tests like that to know there is better out there

Again it depends. If you have no chance to try some gear personally (and you want) and you are able to hear at least the results of somebody else´s tests (however limited they may be), it is 100 times better than nothing, isn´t it ? To give at least some approximate idea
Old 6th September 2007
  #11
Lives for gear
As far as I'm concerned, the Preamps in Paradise DVD from Lynn Fuston addresses the issues of stacking and solo'd comparisons beautifully. That's an amazing effort he put out.

If it doesn't open your ears to some different sonics from different gear, then either you're deaf or much, much more experienced than I!
Old 6th September 2007
  #12
Lives for gear
 
ISedlacek's Avatar
Quote:
Originally Posted by Confusionator View Post
As far as I'm concerned, the Preamps in Paradise DVD from Lynn Fuston addresses the issues of stacking and solo'd comparisons beautifully. That's an amazing effort he put out.

If it doesn't open your ears to some different sonics from different gear, then either you're deaf or much, much more experienced than I!
Yes, who else has an opportunity to collect all these things together and carefully compare ? Such a source has a definite value for those who want to have some practical idea how approximately all those devices sound and behave ... And although it has some limitations, it is MUCH more useful than to describe something by words, within some endless discussions. Even one AB sample is more useful than 10 pages of discussion...
Old 6th September 2007
  #13
Gear Nut
 

IMO a good shootout with audio samples is more insightful than 1000 text posts. Adjectives are a weak substitute for sound. Solo or mixed, I learn from them all. I've spent hours googling gearslutz history looking for "keyword" + "sample" or "demo" or "example" or ".wav" or ".mp3" or ... (you get the point) I'm just a junkie for this stuff.
Old 6th September 2007
  #14
Lives for gear
 

an aside.... while we're on about a/bing, we all know the golden rule is that the comparisons must be: a/ instantaneous and : b/ the same level through the same path etc etc. But it took me way too many years to realise this:

The most useful way to a/b anything is NOT to simply play a few seconds of one and then to play the other. tutt I think most of us do this and become seduced by which is "brighter" or more "excited" or "detailed" or "warmer" or "punchier"........

I say listen to a whole song, or listen to a whole album, then decide how it makes you feel before declaring that a is superior to b.

For the sake of this argument, play 3 seconds of Led Zep 1 and flip to a volume adjusted clip of 3 secs of Green Day. Do this for 100 passers by who don't know what they're listening to. Of course GD wins, but play the 2 entire albums back to back and the preference may be quite different. I try to be mindful of this when a/bing my mixes against references.

The ol' a/b is not always your friend, methinks. OK, as you were....
Old 6th September 2007
  #15
Gear Guru
 
Ethan Winer's Avatar
 

Quote:
Originally Posted by daillman View Post
in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it
Actually, the only true way to test is to measure the device at high resolution using regular audio test equipment. All that matters are frequency response, noise, and distortion.

This is not to say listening tests are never useful, but measuring trumps listening assuming you have access to proper test gear.

--Ethan
Old 6th September 2007
  #16
Gear Maniac
 
Stu Gutz's Avatar
 

Quote:
Originally Posted by Ethan Winer View Post
Actually, the only true way to test is to measure the device at high resolution using regular audio test equipment. All that matters are frequency response, noise, and distortion.

This is not to say listening tests are never useful, but measuring trumps listening assuming you have access to proper test gear.

--Ethan
tutt I've heard that all my life but the ear is the best test gear and always will be IMO.
Old 6th September 2007
  #17
Dan
Lives for gear
 
Dan's Avatar
 

Quote:
Originally Posted by Ethan Winer View Post
Actually, the only true way to test is to measure the device at high resolution using regular audio test equipment. All that matters are frequency response, noise, and distortion.

This is not to say listening tests are never useful, but measuring trumps listening assuming you have access to proper test gear.

--Ethan
Are you being serious?

Specs only help know if a piece of gear meets the technical requirements for a job. That's just about a given these days.
Old 6th September 2007
  #18
Lives for gear
 
allencollins's Avatar
 

Quote:
Originally Posted by daillman View Post
in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it and then giving a good listen,all of this one channel against the other is not really giving a true picture of what the pre is all about.....ok you can move or attack me now....heh
I agree. In fact I think this method also holds true with a/d converters.
To test an A/D I like to play a guitar solo 4 times see how it stacks up
randy rhodes style. The weaker converters always sound even more
sterile multiplied by 4. Not too many hold Up.

I tested a UA2192 this weekend and even that sounded a little digital when double and quadruple tracking.

One of the reasons I like the ISA 110's is they stack up great. You don't
get as much 'haze' as a 1066/73
Old 6th September 2007
  #19
Lives for gear
 
vernier's Avatar
Listening to records reveals the console etc. ...its what made me go all-tube.
Old 7th September 2007
  #20
Lives for gear
 
ssaudio's Avatar
 

Quote:
Originally Posted by ISedlacek View Post
Ait is 100 times better than nothing, isn´t it ?
It only 'depends' if you want something that's similar to something else, in which case, why would you want it anyway? Just buy the cheapest, or as Ethan correctly points out, the 'best' one.
If it's different you want, then you need to be researching what different means and I don't think a/b listening is the way to do that research
Old 7th September 2007
  #21
Quote:
Originally Posted by daillman View Post
in my experience the only true way to test for a pre would be after stacking multiple tracks recorded with it and then giving a good listen,all of this one channel against the other is not really giving a true picture of what the pre is all about.....ok you can move or attack me now....heh
That only the begining. You need to not only listen to how it sits in a final mix, but use different mics and mic positions and try it on multiple instruments. It takes a lot of use to have a truly informed opinion on a piece of gear.
Old 7th September 2007
  #22
Lives for gear
 
ISedlacek's Avatar
Quote:
Originally Posted by ssaudio View Post
It only 'depends' if you want something that's similar to something else, in which case, why would you want it anyway? Just buy the cheapest, or as Ethan correctly points out, the 'best' one.
If it's different you want, then you need to be researching what different means and I don't think a/b listening is the way to do that research
This is just one small part of the story. Many things receive lot of comments or hype or are considered almost legendary, or bad or are pushed by advertisments or commercial interestes or are praised to advocate individual purchases, but a big part of the group has often never tried them and have no idea how they REALLY sound. Hearing some decent samples and if possible also in comparison with something they are familiar with, can be quite educative (sometimes also breaking down some myths and pre-conceptions)
Old 7th September 2007
  #23
Gear Guru
 
Ethan Winer's Avatar
 

Quote:
Originally Posted by Dan View Post
Are you being serious?
I thought I was! heh

Quote:
Specs only help know if a piece of gear meets the technical requirements for a job.
Okay, then what else is there? Please be as specific as possible. Not words like "sounds better" or "beefy" or "transparent" etc, but a specific technical explanation using proper terminology.

--Ethan
Old 7th September 2007
  #24
Lives for gear
 
ssaudio's Avatar
 

Quote:
Originally Posted by ISedlacek View Post
Many things receive lot of comments or hype or are considered almost legendary, or bad or are pushed by advertisments or commercial interestes or are praised to advocate individual purchases, but a big part of the group has often never tried them and have no idea how they REALLY sound. Hearing some decent samples and if possible also in comparison with something they are familiar with, can be quite educative (sometimes also breaking down some myths and pre-conceptions)
If someone cannot see through all the nonsense and relies on a test(s) by someone they don't know, have no idea of their ability or motive and will then have their judgement further clouded by the rest of the opinions on a public forum well, what chance do they really have?

If you want to compete with the 'best', do as them
If you cannot afford it, just buy what meets your needs, because the differences are seldom ever worth getting excited about

Better that one learns to engineer to the best of their ability - gear doesn't tend to be a problem after that
Old 10th September 2007
  #25
Dan
Lives for gear
 
Dan's Avatar
 

I don't think that it's necesarily "sounding better" that is point of A-B testing. It's which one is better for this application, and my sense of aesthetic. Specs rarely play a role, however, I have seen some fx boxes that were so noisy that they got ruled out of being used. The problem I think the OP is pointing out is the perspective shift that is somewhat inherent with extended A-B testing. This has little to do with specs.

For example, the earthworks QTC-50 has incredible specs, but isn't the best mic for every application. The TLM-103 has one of the lowest noise floors around, but that's not why it gets used. Ribbons would have stayed out of popular use if people used gear based on specs.

There's a book called Blink that's a great read about training your mind to make decisions quickly, and the value of it. I think it applies nicely to the conversation.

thumbsup
Old 11th September 2007
  #26
Lives for gear
 
ssaudio's Avatar
 

Quote:
Originally Posted by Dan View Post
my sense of aesthetic
Old 11th September 2007
  #27
Dan
Lives for gear
 
Dan's Avatar
 

Quote:
Originally Posted by ssaudio View Post
Something you don't understand? Do you not have one?
Old 11th September 2007
  #28
Lives for gear
 
JohnRoberts's Avatar
 

If you can hear a deviation from linear response that is real and not imagination, you can measure that deviation. If you can measure it, you can manage it. If it 's good you can name it and reproduce it (preferably with a bypass switch). If it isn't good, you can make it go away.

JR
Old 11th September 2007
  #29
Lives for gear
 
ssaudio's Avatar
 

Quote:
Originally Posted by Dan View Post
Something you don't understand? Do you not have one?
No, yes.
Old 11th September 2007
  #30
Lives for gear
 
nosebleedaudio's Avatar
 

Most engineers I've been around over the years could care less about the specs, they want to HEAR it. If fact I can never remember one of them asking about them.
And many can hear those little things that might not show up in a distortion ect test.
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
Thread Tools
Search this Thread
Search this Thread:

Advanced Search
Similar Threads
Thread
Thread Starter / Forum
Replies
peabody / So much gear, so little time
7
nuendoness / Remote Possibilities in Acoustic Music and Location Recording
0
Kenny Gioia / Gearslutz Secondhand Gear Classifieds
0

Forum Jump
Forum Jump