TEST: 5 DAWs 5 MBP Optical O/P.

[Larry Mal]

Was there a lot of vitriol? I would have thought it was a relatively low-vitriol post of mine. Maybe on another, I lose track, I have been addressing this "test" on this forum and the MOTUNation one. I had hoped to keep it cool since while I might be in disagreement, Radiogal is a beautiful woman, and I was hoping not to be an asshole too much.

Oh well. Not the first time that has happened.

[valis]

I said "a lot"? In seriousness no, but by comparison to some of the subdued posts you were being more direct. Certainly more than I was in terms of calling out the subjective bias that was only implied in my posts and that of others. I don't disagree either, there's a reason that double blind tests are the norm in testing for medicines, of course there's also a counter argument about the strange effectiveness of placebos (placebo effect) and the overall holographic/holotropic nature of existence in general but that takes us into that silly rabbit hole where the debate goes on ad nasuem.

I still suggest though that maybe there WERE differences, that the files *didn't* null, and that project files given up to the community for inspection could help 'refine' such tests...suggest an improved testing methodology...and then we could see what the results are. But again as I suggested such geekery probably wouldn't garner 2k+ thread views, multiple pages of back & forth banter and certainly would make it harder to simply read whatever you want in this thread.

[danworks]

Originally Posted by Radiogal

Plz Stop accusing and blaming me and force me to stand up for this test results.
I´m only the messenger, really!!!

It's funny how some members feel personally criticized for a simple test like this one, just because the "owned" DAW isn't on the top list. I've been successfully using Genelec speakers for years and been very happy n proud of it. Two weeks ago I've borrowed a pair of Dynaudio and ATC, and I've found that the Genelec aren't that good. Not a big deal.

[Amun Ra]

Originally Posted by danworks

It's funny how some members feel personally criticized for a simple test like this one, just because the "owned" DAW isn't on the top list. I've been successfully using Genelec speakers for years and been very happy n proud of it. Two weeks ago I've borrowed a pair of Dynaudio and ATC, and I've found that the Genelec aren't that good. Not a big deal.

The main critique in this thread is not directed at the DAW that won the test, but at the test method itself. The listening test is so poorly deviced that it does not tell us anything at all about what was being tested. The fact that the OP seem to think otherwise is her problem - it is not fair to blame honest critique of rubbish "listening tests" that don't work.

We are supposed to be audio engineers. This means, to me anyway, that we should have some minimal knowledge of how to conduct studies regarding the perception of sound.

That's all.

[Dom & Roland]

I'm not undermining the way the test radiogirl was a part of was done, but something more scientific (if it's even possible) needs to be done to put this to bed if only for my state of mind whenever i start work these days.

I think a bunch of us should band together and do this properly (as in the way all the non believers want it done) once and for all with Reaper , Studio 1 and Live included too.

It would certainly make me relax about whatever daw i choose to carry on with (i have finally had enough of logic after 15 years).

Only people that have engineered or mastered more than 50 released records including vinyl should apply to be listeners. I'm positive i could get some of my colleagues interested including a couple of top (shall remain nameless for the time being) mastering engineers from metropolis and people i beta test plugins for.

Anyone here in london with these credentials interested ?

[valis]

My suggestion Dom, before even undertaking listening tests get the project files together and get everything normalized/ratified by passing it in front of a few forums like GS/LUG (maybe sospubs, thegrid etc). That was the differences between DAW settings and file handling can be accounted for by people who use the apps on a daily basis. Also there should imo be some basic dsp qualifier tests run *before* anything else (sine/square/noise sweeps summed together and then checked for null/audible stepping) to eliminate basic mix engine bugs which crop up from time to time in software builds version to version, and won't be obvious on complex material. And while I like the thought of establishing 'experienced' producers based on the # of published tracks, you'll automatically eliminate some people with dsp experience (namely plugin/software devs) whose input could be invaluable on the test setup between the different softwares...

Anyway I think aside from the listening tests that you're after, what I'm mentioning here could be invaluable in identifying the actual settings that need to be normalized between applications to begin with, as that's a HUGE source of the constant back&forth over which app is better/worse to begin with. Ie, we could finally say "did you check X Y & Z" when people bring up specific mix differences in threads like these. And then we could point at the 'definitive' listening tests after the fact too of course...

Just my 'amateur' 2 cents (based on a lack of published vinyls eh?)

[Dom & Roland]

I was not trying to alienate people with the release comment, i just think a lot of new school engineers who have only ever used software are sometimes blind to what they should be listening for in terms of depth and transients and dare i say it "mojo", the minute differences an experienced ear can hear.

[danworks]

Originally Posted by Amun Ra

The main critique in this thread is not directed at the DAW that won the test, but at the test method itself. The listening test is so poorly deviced that it does not tell us anything at all about what was being tested. The fact that the OP seem to think otherwise is her problem - it is not fair to blame honest critique of rubbish "listening tests" that don't work.

We are supposed to be audio engineers. This means, to me anyway, that we should have some minimal knowledge of how to conduct studies regarding the perception of sound.

That's all.

Exactly, audio engineers, musicians and producers. Every choice we made during a recording, mixing, mastering (loudspeakers, preamp, console, mics) is very personal and there's nothing scientific in it.

Supposing that the test has been conduct in the most honest way and that this "Swedish Sound Engineer Society" is a group of serious folks, if they got a sort of consensus about blind listening test whatever is the DAW who won the test, I don't see how a "scientific test" would change the listeners mind.

Only my humble opinion …

[valis]

Originally Posted by Dom & Roland

I was not trying to alienate people with the release comment, i just think a lot of new school engineers who have only ever used software are sometimes blind to what they should be listening for in terms of depth and transients and dare i say it "mojo", the minute differences an experienced ear can hear.

Well I did mention that there's evidence even in scientific tests that things aren't always as objective as they may seem (mojo/maya/trippy call it what you want). Anyway my point wasn't really one about alienation, but rather there's more to be served here than just helping Dom et. al pick their next sequencer (no offense). If there are bugs that still plague PT 10's 32bit flt, or DP etc then it would help serve to identify that...or more correctly eliminate the constant suggestion that such bugs exist (in Logic, Live etc etc) and support the idea that the differences that exist between the sequencers show up *in use* and not by dint of the summing with everything 'normal'...ie with no faders, mixing features or plugins in use. Use whatever ears you want but I think taking advantage of this opportunity to standardize a 'audio reproduction benchmark' in the same way that TAFKAT has attempted to establish some objective cpu based benchmarks in his DAWBench DSP / VI Universal - Cross Platform DAW Benchmarks thread would help a great number of people just get on with work.

Or to put it another way, if there are reasons you're dissatisfied with Logic (or Apple etc) then having a solidly established test that is sure to compensate for pan laws etc before you guys start your listening test would be helpful no? Which isn't to say that I doubt your ability with the apps you've chosen to use as your track record is well established, but noone knows them all equally well. I know at least in my case I've used Live, Logic, Cubase & my Scope cards long enough to know them pretty damn well (having even helped make a few dsp tools & plugins over the years). But ask me to setup identical projects using a mixture of mono, stereo interleaved and stereo broadcast wave files and *insure* that pan laws are identical across the board etc and I'd certainly appreciate input from others.

[bongo_x]

I use DP a lot. I read the post and thought "interesting". That’s it. A bunch of people listened and gave an opinion. I don’t think anyone claimed to prove anything. Apparently some of you are waiting for the Audio Gods to give the One True Opinion.

I wouldn’t even consider changing what DAW I used for certain tasks if it were proven beyond all doubt that one sounded better than another. I’m more interested in workflow. I’m making music, not test signals.

What if it were a listening test with a 4k SSL and an 80 series Neve? I bet you’d have people preferring one over another. Would everyone be upset about that too?

I use DP and Pro Tools a lot, and sometimes Live, Logic, Mixbuss, Ardour, and others depending on what I’m doing, or the phase of the moon or length of my beard. I’ve always thought that DP sounded good, but I’ve never A/B’d it to anything else because I don’t care. It sounds good to me, that’s all I need to know.

And, your DAW sucks.

[samicide]

Originally Posted by bongo_x

A bunch of people listened and gave an opinion. I don’t think anyone claimed to prove anything.

so what do you think she meant by the following quote:

Originally Posted by Radiogal

So now you know

and

Originally Posted by bongo_x

What if it were a listening test with a 4k SSL and an 80 series Neve? I bet you’d have people preferring one over another. Would everyone be upset about that too?

no, that comparison is irrelevant and not even close to the same thing at all, but it does show your lack of understanding of this topic. I don't think anybody here is actually upset.

[Resonance5]

Originally Posted by djanthonyw

It's already been confirmed numerous times that when all settings are the same including pan laws that every sequencer nulls.

that said i agree there is not much to discuss about

[edit] having understood that analog mixing is the only real pro one, who cares about any hipotetical difference in quality when you use the DAW only to route channels to the interface's output channels!

[Radiogal]

Originally Posted by samicide

so what do you think she meant by the following quote:




and




no, that comparison is irrelevant and not even close to the same thing at all, but it does show your lack of understanding of this topic. I don't think anybody here is actually upset.

By saying: "So now you know" I ment...
So now you all know about the test and the what we found out... Gee, I´m dutch and living in Sweden, learned my english in school long time ago..
I´m by all means no native american.. so plz don´t dive into the microscope saying "what does she mean by that."
I meant what I wrote.... so now you know! End of report..!

[mholloway]

I played a song in every DAW to a group of pre-school children of mixed ethnicities and more of them started crying during DAW 1 and 5 (sometimes 4) and more started pooping during DAW 2, and DAW 3* got the most laughing by far in fact the children were dancing and shouting the most during this one! SO VERY INTERESTING, ISN'T IT??!!


*DAW 3 was microsoft Songsmith

[DAW PLUS]

While you might not agree with the methods of the Swedish team, the attempted humor is misplaced imo...

[rksguit]

Originally Posted by engelen62

Another point is that from the picture the listening is far from ideal for anyone.

Originally Posted by Radiogal

I heard the engineers say/yell "nr 1 is the best" all the time. I agreed.


MBP 1= Digital Performer
MBP 2= ProTools 9
MBP 3= Logic 9
MBP 4= ProTools 10
MBP 5= Cubase (latest version)

Something's is missing ere Edit;sorry,just super bored by the tired,outdated,Pro Music/Mac only approach,no REAL offence mean't,just frustrated boredom.

[reddirt]

Don't take it personally Radiogal, this is the nature of the beast.
Right or wrong,many here have had their good intentions torn down by those who could/should disagree in a well mannered way but who choose to be a bit pushy behind their computer screen.

Cheers, Ross

[bandorr]

Originally Posted by reddirt

Don't take it personally Radiogal, this is the nature of the beast.
Right or wrong,many here have had their good intentions torn down by those who could/should disagree in a well mannered way but who choose to be a bit pushy behind their computer screen.

Cheers, Ross

This is a thought that speaks volumes to me. My personal belief is, that we are all colleagues regardless of our level of understanding, or knowledge, and, above all, we are all HUMAN beings. We can make mistakes (and we will) we can disagree, but (maybe)we have to do in a way that, don't leave a bad taste in our mouths .

I tried to bring people to have discussions with that principles in their minds. Sometimes(most of the time) I ended up accomplishing exactly the opposite.

That being said, I would like use your phrase in my signature, but I understand, if you don't want your name associated with the "other" phrases from my sig. But if it is ok with you, just let me know.

[Radiogal]

I found this today here on GS...

The Ultimate Converter DA/AD Loopback Shootout Thread!

[Papanate]

How were the original tracks recorded? Because I see a flaw if they were recorded in DP or Protools.

[Sylvester217]

Originally Posted by Radiogal

This test took 2,5 hours and the material was tested on 30+ members of the Swedisch Sound Engineer Society. All professionals.

I can't help but thinking "where is the hidden (web-)cam" that fools us all...
What conclusion might one draw when reading this ?
...like "switching to DP will make my mixes sound better...?

How come a bunch of so called professionals have 2,5 hours time for such
nonsense?
Once more I'm so glad that i've learned my skills only from experience and mistakes.
I have -and always will- stay away from any kind of societies or academic circles for sure.

I know this is gearsluts ( and I do love my API,Urei,ELI stuff etc., too ) but this is getting rediculous.
Well, maybe I'm more of a "slut" than "gear", but I'm happy (...and busy !)

long live rock'nroll

[Muser]

I think on a test like this, they should set every DAW to be -3db.
make sure they all are reading the same clock source
then use some mechanism to make sure the levels are exact enough that they null. if they won't null one would want to know why.

have some way to check for dither then run the the test.

if they start picking different DAWs and a repeatable pattern arrises, then you have a little bit of information to wonder about, if they nulled.

if they didn't null then why they wouldn't null, combined with the patterns of choices, might hold some information.

The main thing is the levels need to be exact so the amps and speakers performance can be elliminated.

[bugscoe]

Interesting read. Long time DP user up until the last year in which I used Logic exclusively. Going to add DP back into the mix.

[DAW PLUS]

Originally Posted by bugscoe

Interesting read. Long time DP user up until the last year in which I used Logic exclusively. Going to add DP back into the mix.

I am missing the sarcasm, correct?

[duckoff]

I dont really think the battle here is between DP users and non DP users. Its between the people who believe different DAWs sum audio better than others, and the people who know otherwise. We aren't saying that DP can't be the best, only that no one can be better at summing audio than another.

Bingo - they're all the same, this is just an interesting demonstration of group psychology & expectation bias.

You wold have got similar results playing the same file from the same computer each time.

You need to do properly implemented ABX testing to get any meaningful results.

Even the most accomplished & experienced professional is susceptible to these things.

[bugscoe]

Originally Posted by DAW PLUS

I am missing the sarcasm, correct?

I'm thinking of switching to Pro Tools because it's the industry standard that all the hits are made on. I think it will increase my odds of making a hit, a hit that at least sounds standard anyways.

However, since all DAW's sound the same I may just use Garageband to save some money. I could probably buy a lot of Skrillex dub step bass patches with that extra money.

Correct?