Overall I'm surprised that they are all very listenable. In fact, if I did a proper A/B/X test with levels perfectly matched, I would probably only be able to pick out the few worst of them.
My preferred way of evaluating something like this, to magnify the difference even more, is to phase null the test subject to the original. I spent quite a while lining them all up, down to the sample, and adjusting the levels. I think the alignment is perfect but the levels could still be a touch off  it's harder to lock the levels in when the subject is just oddly different than the original. I rank them in this order for phasenullingaccuracy, worst to best (best of course would be to hear no sound at all = no difference): •Saffire LE •Apogee AD/DA 16x (kind of a tie really between Saffire and Apogee for biggest difference) •RME FF400 (corrected) •Prism Orpheus •192 IO (or could be reversed with Prism) •Metric Halo  clearly the best! Listening to only the DIFFERENCE signal, the sound is: Saffire: bassy plus pinched high end and extra top Apogee: no bass, mids and low highs pronounced RME: sounds like the middle is scooped out (removed). hear the mix fully, but highs and lows predominate. Difference is a little quieter than the first two. Orpheus: Difference a bit quieter again. Sounds a bit like the Apogee but with more upper mids, kind of peaky maybe around 6k. 192: like the RME, but much less lows. Metric Halo: Amazing! Difference is significantly quieter. Fairly dark sounding. So then the question is, for example, does the difference in bass on the Saffire mean it's adding or subtracting bass? Could be either. I tried doing some EQing to the subject track, but I abandoned this as it was clearly not going to be straightforward. If I had to guess maybe on the Apogee, to get their 'warm, analog sound' it seems to make sense to accentuate the mids and lose some extreme top, with the bass being 'true'. I do wonder if it makes any difference whether you have your DAW in 16 bit or 24 bit mode. As I stated, I increased the word length of the original to 24bit and ran my PT session 24bit for my 10x looping through the 192, as well as my listening tests. It is hard to believe the report that using some difference analyzer the Apogee was the best.. not sure what would account for my totally opposite findings. Though I would be curious how the 192 ranks in that. Should I post the 'difference' files for you guys to hear? Or my PT session with the audio files retrimmed with my volumes and alignment? Fun test. I'd like to hear more different gear tested! 
Can you upload the 192 sample on some free file sharing service and post me the link so I can add it to the list? My email account can't handle very large files...
It would also be interesting to hear your difference test results kfhkh How loud were the differences? Are we talking about 0.1dB or 1dB or what? If the results are really quiet, then the difference is nearly inaudible to the ear. 
Quote:
Quote:
That, or you could do the same for everybody else's samples and create time/level idealized comparisons that let us listen to just the digital artifacts of everybody's coverters. But it'd probably be quicker for you to click "undo" twice than to rewrite everybody else's results. Science isn't about doing the coolest thing you can think of every single possible second. It's about coming up with a simple test and applying it lovingly to all the relevant possibilities. I think Mozart once said something similar about music. 
I'll find a way to get the stuff posted tomorrow. No, the differences were pretty loud, like 6db quieter perhaps than the original for the 'worst' ones. That's just a guess, I didn't measure it. But they're not 30db down or anything like that.
re Eli: I believe others also tweaked the level of their files, in fact the Metric Halo poster said he did it after every pass. I don't see how lining up the file and running a little gain change hurts the file, but I have no problem posting the 'raw' file also. 
cool thread!

1 Attachment(s)
Here is the Echo Layla 24. Some notes on what I did:
1) I ran the loopbacks and kept the file as 24 bit, as it seemed others have done so as well. 2) At the end I added 5 db of level on the fader to bring it back up. I came up with this because the original peaked at 0.1 db, my result peaked at 5.1 db. 
Would be nice if someone could do this with the Lynx Aurora as it's a pretty popular model. I will also do this for the Audiofire12 which is a pretty popular "budget friendly" one.

Digi192IO file
1 Attachment(s)
Ah, maybe I can attach a file too.

3 Attachment(s)
Seems like that file attaching worked, so here are my DIFF files. They are at the absolute level they naturally ended up (original file minus looped file via phase inversion). I did not boost or lower any of the DIFFs, so the quieter ones are less different.

3 Attachment(s)
and the rest so far...

1 Attachment(s)
Layla24 Diff.

I added the Roland MMP2 and Echo layla 24 to my previous list.
Rolland MMP2 with frequency matching 15,6msec, 0,929dB (L), 0,929dB (R)..Corr Depth: 4,7 dB (L), 4,7 dB (R) Focusrite Safire 10,25msec, 0,982dB (L), 1,291dB (R)..Corr Depth: 6,2 dB (L), 6,3 dB (R) Echo layla 24 44,82msec, 0,929dB (L), 0,840dB (R)..Corr Depth: 6,8 dB (L), 6,8 dB (R) Fireface 400 356usec, 1,467dB (L), 1,062dB (R)..Corr Depth: 7,7 dB (L), 7,6 dB (R) Orpheus 13,04msec, 0,171dB (L), 0,058dB (R)..Corr Depth: 12,8 dB (L), 12,6 dB (R) Metric Halo 70,81msec, 0,229dB (L), 0,157dB (R)..Corr Depth: 18,1 dB (L), 17,9 dB (R) Apogee 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R) 
Layla and 192 added.

best converter thread so far boing

for me it's
1: Apogee (just sounds so good) 2:Digidesign 192 (surprisingly close to Apogee) 3:Prism orpheus (for the price it's a shame) 
I added the Digidesign 192 I/O to my previous list.
Rolland MMP2 with frequency matching 15,6msec, 0,929dB (L), 0,929dB (R)..Corr Depth: 4,7 dB (L), 4,7 dB (R) Focusrite Safire 10,25msec, 0,982dB (L), 1,291dB (R)..Corr Depth: 6,2 dB (L), 6,3 dB (R) Echo layla 24 44,82msec, 0,929dB (L), 0,840dB (R)..Corr Depth: 6,8 dB (L), 6,8 dB (R) Fireface 400 356usec, 1,467dB (L), 1,062dB (R)..Corr Depth: 7,7 dB (L), 7,6 dB (R) Orpheus 13,04msec, 0,171dB (L), 0,058dB (R)..Corr Depth: 12,8 dB (L), 12,6 dB (R) Digidesign 192 I/O + Wavelab SRC 48 > 44 1,701usec, 0,021dB (L), 0,159dB (R)..Corr Depth: 17,9 dB (L), 17,7 dB (R) Metric Halo 70,81msec, 0,229dB (L), 0,157dB (R)..Corr Depth: 18,1 dB (L), 17,9 dB (R) Apogee 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R)[/QUOTE] 
oops.. now 44.1
1 Attachment(s)
Hello... My apologies for posting my 192 10x loop file at 48khz. I looked into it and I did run my PT session at 44.1 24bit like I said, so I guess I must have accidentally had the sample rate for my interleaved export at 48khz. Please find attached the corrected 44.1 version... Kraku, please replace with this version.
Didier  would appreciate it if you could redo your diff analysis with this file. Also, is there any chance you could recheck your result for the Apogee? It just doesn't match up with my listening diffs. Surely could be some mistake on my part, or maybe the fact that the low end is quite accurate means your db readings are that much better though it doesn't sound that much better? I just looked at Audio Diffmaker. It seems to create the audio difference files as well as the analysis of the difference. Maybe you should post those instead of mine since the software probably does a better job of matching levels and timings than I did. 
Done.

Thanks! That was fast. Can I talk you into changing your avatar? That freaks me out! mezed

2 Attachment(s)
I tested the 192 I/O loop at 44 kHz and got exactly the same result like the 48 kHz file converted to 44 kHz in Wavelab. I know that it's very surprising... confoosed
I retested the Apogee file and got also the same result like the first test one, which is not surprising! cooge Digidesign 192 I/O 1,701usec, 0,021dB (L), 0,159dB (R)..Corr Depth: 17,9 dB (L), 17,7 dB (R) Apogee AD&DA 16X 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R) Difference files here attached. Here is the true surprise: the Apogee one is about 4 dB louder that the Digi one... I guess that the explanation is that the figure named 'correlated null depth' delivered by Audio DiffMaker is not the level of the difference of the timealigned gainbalanced tracks. How it differs from it, I don't know...confoosed 
2 Attachment(s)
I added the RMS level measured in Wavelab of the timealigned gainbalanced difference delivered by Audio DiffMaker
Rolland MMP2 with frequency matching 15,6msec, 0,929dB (L), 0,929dB (R)..Corr Depth: 4,7 dB (L), 4,7 dB (R) Diff. RMS 23,3 dB (L) 23,8 dB (R) Echo layla 24 44,82msec, 0,929dB (L), 0,840dB (R)..Corr Depth: 6,8 dB (L), 6,8 dB (R) Diff. RMS 25,5 dB (L) 24,9 dB (R) Focusrite Safire 10,25msec, 0,982dB (L), 1,291dB (R)..Corr Depth: 6,2 dB (L), 6,3 dB (R) Diff. RMS 25,8 dB (L) 25,1 dB (R) Fireface 400 356usec, 1,467dB (L), 1,062dB (R)..Corr Depth: 7,7 dB (L), 7,6 dB (R) Diff. RMS 26,5 dB (L) 25,9 dB (R) Apogee AD&DA 16X 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R) Diff. RMS 31,8 dB (L) 30,8 dB (R) Orpheus 13,04msec, 0,171dB (L), 0,058dB (R)..Corr Depth: 12,8 dB (L), 12,6 dB (R) Diff. RMS 32,3 dB (L) 31,7 dB (R) Digidesign 192 I/O 1,701usec, 0,021dB (L), 0,159dB (R)..Corr Depth: 17,9 dB (L), 17,7 dB (R) Diff. RMS 35,5 dB (L) 34,8 dB (R) Metric Halo 70,81msec, 0,229dB (L), 0,157dB (R)..Corr Depth: 18,1 dB (L), 17,9 dB (R) Diff. RMS 35,7 dB (L) 35,0 dB (R) 
Quote:

1 Attachment(s)
Here is the Audiofire12, pretty good bang for the buck I think and the correlation depth is better than RME and almost as good as Orpheus
Parameters: 14.39msec, 0.743dB (L), 0.774dB (R)..Corr Depth: 11.8 dB (L), 11.3 dB (R) Notes: I did this on a new laptop so I cut off part of the beginning silence since I was getting a lot of latency. I also added 7.6db of gain to compensate for the iterative loopback volume losses. Attachment 234318 
AudioFire 12 added.

Echo AudioFire 12
parameters: 14,39msec, 0,743dB (L), 0,774dB (R)..Corr Depth: 11,8 dB (L), 11,3 dB (R) Diff. RMS 23,2 dB (L) 22,5 dB (R) Rolland MMP2 with frequency matching 15,6msec, 0,929dB (L), 0,929dB (R)..Corr Depth: 4,7 dB (L), 4,7 dB (R) Diff. RMS 23,3 dB (L) 23,8 dB (R) Echo layla 24 44,82msec, 0,929dB (L), 0,840dB (R)..Corr Depth: 6,8 dB (L), 6,8 dB (R) Diff. RMS 25,5 dB (L) 24,9 dB (R) Focusrite Safire 10,25msec, 0,982dB (L), 1,291dB (R)..Corr Depth: 6,2 dB (L), 6,3 dB (R) Diff. RMS 25,8 dB (L) 25,1 dB (R) Fireface 400 356usec, 1,467dB (L), 1,062dB (R)..Corr Depth: 7,7 dB (L), 7,6 dB (R) Diff. RMS 26,5 dB (L) 25,9 dB (R) Apogee AD&DA 16X 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R) Diff. RMS 31,8 dB (L) 30,8 dB (R) Orpheus 13,04msec, 0,171dB (L), 0,058dB (R)..Corr Depth: 12,8 dB (L), 12,6 dB (R) Diff. RMS 32,3 dB (L) 31,7 dB (R) Digidesign 192 I/O 1,701usec, 0,021dB (L), 0,159dB (R)..Corr Depth: 17,9 dB (L), 17,7 dB (R) Diff. RMS 35,5 dB (L) 34,8 dB (R) Metric Halo 70,81msec, 0,229dB (L), 0,157dB (R)..Corr Depth: 18,1 dB (L), 17,9 dB (R) Diff. RMS 35,7 dB (L) 35,0 dB (R) 
1 Attachment(s)
Evening,
10 x Universal Audio's 2192, normalized... 
Quote:
Listening back to my audiofire one it sounds like there is a clip in the beginning so might have to redo it, think you still get the idea though. 
Echo AudioFire 12
parameters: 14,39msec, 0,743dB (L), 0,774dB (R)..Corr Depth: 11,8 dB (L), 11,3 dB (R) Diff. RMS 23,2 dB (L) 22,5 dB (R) Rolland MMP2 with frequency matching 15,6msec, 0,929dB (L), 0,929dB (R)..Corr Depth: 4,7 dB (L), 4,7 dB (R) Diff. RMS 23,3 dB (L) 23,8 dB (R) Echo layla 24 44,82msec, 0,929dB (L), 0,840dB (R)..Corr Depth: 6,8 dB (L), 6,8 dB (R) Diff. RMS 25,5 dB (L) 24,9 dB (R) Focusrite Safire 10,25msec, 0,982dB (L), 1,291dB (R)..Corr Depth: 6,2 dB (L), 6,3 dB (R) Diff. RMS 25,8 dB (L) 25,1 dB (R) Fireface 400 356usec, 1,467dB (L), 1,062dB (R)..Corr Depth: 7,7 dB (L), 7,6 dB (R) Diff. RMS 26,5 dB (L) 25,9 dB (R) Apogee AD&DA 16X 12,92msec, 0,066dB (L), 0,210dB (R)..Corr Depth: 21,4 dB (L), 21,9 dB (R) Diff. RMS 31,8 dB (L) 30,8 dB (R) Orpheus 13,04msec, 0,171dB (L), 0,058dB (R)..Corr Depth: 12,8 dB (L), 12,6 dB (R) Diff. RMS 32,3 dB (L) 31,7 dB (R) Universal Audio 2192 parameters: 13,13msec, 0,183dB (L), 0,316dB (R)..Corr Depth: 17,7 dB (L), 17,4 dB (R) Diff. RMS 35,2 dB (L) 34,6 dB (R) Digidesign 192 I/O 1,701usec, 0,021dB (L), 0,159dB (R)..Corr Depth: 17,9 dB (L), 17,7 dB (R) Diff. RMS 35,5 dB (L) 34,8 dB (R) Metric Halo 70,81msec, 0,229dB (L), 0,157dB (R)..Corr Depth: 18,1 dB (L), 17,9 dB (R) Diff. RMS 35,7 dB (L) 35,0 dB (R) 
Could someone explain what the diff rms is saying exactly? Is that or the corr depth a better measure of accuracy?

I do not know yet what is exactly the 'correlared null depth', which are the L and R figures provided by Audio DiffMaker. I have to read the AES paper explaining the principle of AudioDiff Maker. (I have so many things waiting for reading...)
The Diif. RMS figures are my own measurement performed in Wavelab of the RMS level of the difference between the original and the 10X loop time aligned and gain balanced by AudioDiff Maker. The hierachies determined by both figure kinds are not the same ones but rather coherent. 