The Ultimate Converter DA/AD Loopback Shootout Thread!

[adrianww]

I see results in the list for the Focusrite Saffire LE. I assume that's the old desktop unit?

I can try the tests out on an up-to-date Liquid Saffire 56 if you want. Might take me a few days to get around to it (since I'll need to find the time to re-cable at the back of the rack - maybe one day next week) but I'll give it a go if anyone's interested.

I could possibly follow that with tests on an old Alesis iO/14 and a Yamaha n12 as well, although I'll have to get those out of the cupboard and re-install them on my PC first, so that will take a bit longer.

[nms]

Hey Adrian. Sure the Saffire when you have time would be good. The others I wouldn't worry about.

As for the Motu 896mk3 It would be good to add one in but I wouldn't expect better results than the 828mk3.

[adrianww]

OK, I'll try the LS56 out when I get the chance - like I said, might be sometime in the next week or so.

I'll probably try the n12 as well sometime after that. I'm already thinking of having it on standby as a backup/alternate interface and it'll be interesting to see how it does.

As for the old Alesis box, yeah well, you're right that there's not much point tinkering with that nowadays. It's only still here 'cos I'm an old softie and it was my first audio interface. Although it does make a good paperweight too. Or doorstop for fairly light doors. :-)

I'll post results up as and when I've got 'em.

Meanwhile, I'm also thinking about the test methodology itself, but that's a whole different topic and I'm not sure exactly what it is that I'm thinking yet. (It's just one of those niggling feeling things at the moment where something is bugging me but I can't pin it down.)

[nms]

alright.. as promised, going deeper! Here are some revealing snapshots of a few key areas that are exposing performance differences here. Exact same audio after 10 passes and matched levels. Of course these are things you can see/listen to yourself if you downloaded the result files for any of the converters.

Funny how the 828mk2 is looking the best in both areas while the 828 mk3 looks the worst as evident by the sawtooth mangling!

[EveningSky]

Hello NMS,
Thank you so much for doing this study.
I am a home recording hobbyist and find the issue of converter quality so confusing.
Several months ago, I took out my older Motu 828MKII and was amazed at how good it sounded. I did not used external preamps / I turned down the MOTU preamps to their lowest position. Now I understand why it seemed to sound so good. Because it was good!!!

I wish to also point out that I am a consumer am frustrated. I would like to find a ADA Converter for about 500-1000$, w a usb or FW interface. What I find are converters w build in preamps and other embellishments which I really neither want, nor need. Every thing electronics gets better and cheaper, yet the companies with a reputation for quality conversion, make more expensive products, and the discussion and frustration in the prosumer market rages on.
Thank you again for doing this study.
Yours,
ES

[adrianww]

I've now had a chance to try this test on my Focusrite LS56, but I don't have any worthwhile results. I did all the calibration as suggested, but it looks like there's still some gain in the loop somewhere, since each successive pass is coming out louder than the one before - not by much, but it is noticeable on playback and when looking at the waveform display in Cubase 6. Something just doesn't seem quite right and any results from my current test runs are almost certainly bogus. Or at least bogus enough to make a mockery of a converter transparency test.

Leave it with me and I'll see if I can get to the bottom of it and get 10 decent passes. Don't know when I'll have it ready though - I'll post it up when (if?!) I manage to get it right.

[nms]

If that's the case simply turn down the input or output gain just enough that it doesn't come back louder. Use the smallest increment you can to get it as close to unity as possible.

[adrianww]

Originally Posted by nms

If that's the case simply turn down the input or output gain just enough that it doesn't come back louder. Use the smallest increment you can to get it as close to unity as possible.

Yes, that's what I'm trying - it's just going to take me a little while to get it spot on. I'll post the results up when I've got something that looks properly matched for levels. Might be another few days before I get the chance to run through it again though.

[nms]

Hmm.. Shouldn't take any time at all if you're using the included test tone.

[adrianww]

Yeah, that's what I thought. Set things up with the test tone playing in a loop, routed out and back in, output levels at 0, input level set to give me -1dBFS on the input channel when the tone was playing. All good, went on to play back and record the test file without changing any settings and each recording pass was slightly louder than the one before. By pass 4 or 5, I was starting to get clip warnings on output and input in Saffire MixControl. Stop, scratch head, start over.

I've probably just overlooked some trivial mistake in the way I set things up and will have a Homer Simpson moment when I get the chance to try again and realise what I did wrong.

[shreddersinc]

This is an interesting test, I thought someone must of made a mistake with the 828mk3...how could it perform worse than the mk2. I own the 828mk3 ran the saw tooth test and it mangled it just as in your picture.

I guess I could of done worse than buying the mk3, but I still feel that the difference in conversion is not going to be noticable even on a massive project consisting of 80 or so tracks (in a double blind A/B test?)I know I'm speculating.

Of course, I wish it had better conversion and I might be a little sore that the new and improved version mk3 is outperformed by it's ancient ancestor mk2.
I sure can't afford to run out and buy a new interface

Why didn't you guys think of this crazy sheet years ago!
How can motu's newer products be worse than their older ones?

[nms]

You could always do as I did. I sold my mk3 and picked up a mk2 off eBay for $250 less! Companies don't count on people doing these sort of tests. In Motu's case they released the Mk3 to make more cash off an existing design with a few slight changes but seem to have botched the conversion a bit.

[jinksdingo]

Starting to make me wonder if the 2408 MkII would outperform the MkIII?

[6strings]

We need to get results for the Ensemble and UFX in here!

-pretty please!

[nms]

Originally Posted by jinksdingo

Starting to make me wonder if the 2408 MkII would outperform the MkIII?

Definitely not. The 2408 mk3 and the 828 mk2 share the same converters and the closer release date, thus the results.

And yes, definitely need to get the ensemble and UFX added! The UFX will be middle of the road though I think.

[Greg B]

Originally Posted by nms

If that's the case simply turn down the input or output gain just enough that it doesn't come back louder. Use the smallest increment you can to get it as close to unity as possible.

First of all, I respect the effort put into this test. But I think it can be misleading...

So we're accepting data from different studios with different onboard preamps, using different cables, and calling it a "converter" test? I've had measurable differences in cables that are greater than some converter differences. A proper A/B test can only have one variable. If you really want to test only the converters than you must test each AD and DA separately with all other factors being equal. This test currently has more variables than just the converters, so we don't know the real cause of the resulting data.

And to me the reason to buy any audio gear is for it's sound. You can either hear the difference blind or you can't. If it sounds better to you, use it. If you can't hear a difference, buy the cheaper one.

[nms]

I recommend reading the thread. Already addressed.
High end cables aren't going to make any of these units show up in a different ranking position in the list.. only bad cables and patch bays.

[Greg B]

Originally Posted by nms

I recommend reading the thread. Already addressed.
High end cables aren't going to make any of these units show up in a different ranking position in the list.

I read the thread, and I'm backing up what Warren said originally. I agree with him. I'm not saying high end cables will make all the difference, but different types, lengths, etc. do make a difference, and so do preamps, and certain aspects/settings in different DAWs (especially after many passes). You are telling people to turn their preamps up and down until the volumes kinda match, and you're also matching the signals with RMS averages. That right there tells me there are significant additional variables that are bigger than the converters.

And the other thing is, you seem to have concluded that the Motu is the most transparent, which happens to be what you own?

Like Warren had said, I'd need a totally unbiased blind test where only one variable is tested at a time. Otherwise I'd trust my ears and even consensus opinions from random strangers over the resulting data. A test that is ALMOST scientific can be the most misleading of all.

[Greg B]

Originally Posted by nms

Definitely not. The 2408 mk3 and the 828 mk2 share the same converters and the closer release date, thus the results.

And yes, definitely need to get the ensemble and UFX added! The UFX will be middle of the road though I think.

Last I knew, many interfaces share the same AKM converters that MOTU uses, including Echo interfaces. Don't know if that's changed recently.

I've done tests with my Lavry Blues that have shown how consistently and accurately their converters reproduce audio. Some cable differences were greater over multiple loops than the converters. I havn't tested the Burl for scientific accuracy but it sounds great, and I've been able to pick it out blind in every test so far. If the Burl is lying, then it's a white lie.

[6strings]

Originally Posted by Greg B

And the other thing is, you seem to have concluded that the Motu is the most transparent, which happens to be what you own?

You need to re-read the thread. Slowly.

He bought the MOTU after learning it was the most transparent of the lot.

And the differences you mention WERE addressed earlier in the thread.

Summary; Any differences in cable brand or length etc. do not introduce enough variance to throw off the results. The units will always rank in the same order of performance. But re-read thoroughly. And also check the other converter loop thread as well.

[jinksdingo]

Originally Posted by nms

Definitely not. The 2408 mk3 and the 828 mk2 share the same converters and the closer release date, thus the results.

And yes, definitely need to get the ensemble and UFX added! The UFX will be middle of the road though I think.

Thank nms, good to know.

I was in the market for a new interface pushing above its weight cost wise.

It was bewildering: the choices until I was directed to this and the other converter thread.

I gotta tell you, even if there are some discrepancies in these test results; they are better than what was previously available for a consumer to evaluate by some means and compare a short list.

Apart from opinion we had only a feature set to differentiate a choice.

Converters for monitoring are one area where you don;t want to use your ears
to determine whether you like it or not; not accurate enough.

These test results fill a void demystifying what is accurate.
Good enough for me

[nms]

Haha.. Greg I'm not sure where you drew some of your conclusions from my man! Calibration setup is not done using rms but rather your most basic continuous 1k sine wave and daw meter reading. It's also stated to use direct line ins unless not possible, as this is not a preamp test here. Most units aren't preamp only and those are the ones we care about most anyhow. I certainly have no interest in onboard pre's myself. I can't imagine how you could suspect any bias on my behalf based on what I've posted in this thread. I've gotten rid of 2 interfaces through the course of this thread due to the results. Who wants to waste time trying to creatively fool themselves into believing a piece of gear they own is better than it is? That won't do your productions any favors!

"The Motus" as proven here do not all perform alike. It looks to be only the 828mk2 and 2408mk3, which tested identically (3 units by two different people and a fourth coming shortly) and share the same converters upon investigating. Interesting that Black Lion Audio refused to do mods for the 828mk3.

Originally Posted by Greg B

I havn't tested the Burl for scientific accuracy but it sounds great, and I've been able to pick it out blind in every test so far. If the Burl is lying, then it's a white lie.

I'd gladly take a Burl for my studio.. almost did a few months ago but put the money towards a purchase that made a much bigger difference. Few will contest them as great sounding units. I'm not surprised you could pick it out in a blind test.. you should as it's known for being a sweetly colored unit. We're not taking on the pointlessly subjective discussion of what sounds good here though

[Greg B]

Originally Posted by 6strings

He bought the MOTU after learning it was the most transparent of the lot.

I read he sold his MOTU 828 mk3 and bought the mk2. If he said he didn't buy the mk3 until after the test, then I guess I did miss that. Sorry.

Originally Posted by 6strings

And the differences you mention WERE addressed earlier in the thread.

For me, addressing the multiple-variables issue means doing all tests again with only one variable.

Still, I do love this sort of thing, and I'm interested in seeing if I can repeat the results in a similar test. I was planning to compare my Lavry Blues and Burls. I've compared them with my ears, but not with data. The AD and DA on the Burl are designed with different goals in mind, so testing them separately would be the most helpful.

Originally Posted by nms

Who wants to waste time trying to creatively fool themselves into believing a piece of gear they own is better than it is?

Unfortunately I've found that most engineers do this (but I believe that you are not one of them). Most want to justify their purchases and they hear what they want to hear. Double-blind tests seem to be the only way to avoid the bias.

[nms]

Yep when I started this thread I had a Mytek stereo96adc and my Steinberg MR816x. Sold the Steinberg, grabbed an 828mk3, listed it for sale within 5 mins of seeing that saw tooth, and now just picked up a factory refurb 828mk2. Haven't parted with my Mytek yet. I'm going to have to do some really focused recording tests between the Mytek and Motu before considering parting with it.

Anyone will agree that the best testing is done by one person using the same exact methodology on all units with the same cables, power outlet, etc. We just don't have the luxury of such an option. Of all my testing thus far I haven't found slight level variations or high end vs adequate cables to upend the rankings though.

[6strings]

Originally Posted by Greg B

Double-blind tests seem to be the only way to avoid the bias.

Keep in mind, though, that this particular test method is for ranking units simply in regards to the transparency of the converters. Not necessarily which will "sound the best" for a particular application (or set of ears).

[lobsterinn]

I had a thought about this testing, which I think is an important grain of salt to take when considering the results.

If you are testing for sum difference in the pre- and post-conversion files, it seems really important to consider what parts of the spectrum are most / least accurate.

For example, if one converter is perfectly solid from 100hz-15k, but iffy in the extreme highs and lows, it could do poorly on the test but still sound much cleaner than a converter which nails the extremes but screws up the midrange. I know subjective comparison is not part of this test, but people really should use their ears to make purchasing decisions for this stuff.

Like frequency response charts for microphones, this analysis serves a purpose but does not tell the whole story.

[lobsterinn]

@nms:

I really like the idea of objective converter testing - not trying to rain on anyone's parade.

In fact, it would be wonderful if converter manufacturers could be pressured into providing 3rd-party documentation of their design's performance. There is so much smoke and mirrors in converter marketing.

[nms]

While that scenario could potentially cause a unit to test worse, it wouldn't play any part in the top half of the rankings where the chart topping units can only be there by having a great result. This is especially true of the top ranked units. If those are the ones scoring highest of all here then questioning those in that way is to assume that none of the units provide truly clean conversion.

Originally Posted by lobsterinn

Like frequency response charts for microphones, this analysis serves a purpose but does not tell the whole story.

Haha.. terrible terrible terrible comparison.

You have to remember here, this test provides both the Diffmaker correlation depth and the RMS level of the difference file (one of those being frequency dependent yet they they both place the units in the same place with exception of the 16x), plus you can download the audio and listen or look at the waveforms for all of the targeted areas. Some of you are forgetting just how much there is to examine there past the chart results. You have your choice of 1, 4, or 10 pass recordings to check out containing all the tones and content listed in the beginning of this thread. I'm sure a lot of people just care about the chart stats and won't bother but for the rest you can look much further into it. I'll be adding downloadable audio for a few converters this week plus we have a few more units to add to the list. I'll also be adding my factory refurb 828mk2.

[David Carlyon]

Any joy testing the 896HD? Does it have the same converters as the 828Mk3?
Thanks

[nms]

Yes definitely. If you have one then run the test and pm me a link to download the raw 1, 4, & 10 passes.

RME Multiface 1 added. Bonus points to me for guessing where it would sit after viewing the waveform. BAM!

Another Prism Orpheus added. For those who question the validity of this test due to testing not being all done under the same roof, same cables, check out how those units come within exactly .2db of each other on every figure

The Orpheus is showing a fair amount of distortion to the sawtooth waveform I noticed.