The No.1 Website for Pro Audio
 Search This Thread  Search This Forum  Search Reviews  Search Gear Database  Search Gear for sale  Search Gearslutz Go Advanced
Launch of Pono Studio Headphones
Old 5th March 2015
  #5311
Lives for gear
 

What is your analysis?

Come to think of it, why would you feed the thing a tone all the way down at -96dBFS and then crank gain in the analog stage? And consumer headphone out to XLR in? Isn't that a recipe for introducing analog noise?
Old 5th March 2015
  #5312
Airwindows
 
chrisj's Avatar
But you can see why: to drop the noisefloor of the measurement ADC to nearly -150db, greater than you can get out of a straight 24 bit capture. Same as the full scale output of the devices weren't coming into the ADC at 0db, but way lower. I wanted to see the noisefloor of the devices without interference from the capture ADC, and the distortion of the devices under load without adding distortion from the ADC.

The iPhone only has a headphone jack, and that's what I wanted to measure: it's possible the Pono's other jack measures even better but I think they're the same circuit. You can set them up to be dual balanced outputs, but I'm not testing that, I'm testing what normal(ish) people would actually do and what they'd get. As it stands, the noise floor is a whopping twelve db below the best the iPhone, itself a strong performer, can do, and that's at the 192K rate that potentially creates worse noise issues than lower-rate sampling.

This is the scenario Monty (yeah, I know, him again) was warning us against: that the higher rates would be actually worse, more noise and introduction of frequencies that stress out downstream gear. From my testing I'm gonna say nope, nor does it sound anything like there's a problem. Thing sounds awesome: the behavior under test shows markedly better performance than a known-good normal device, apart from increased even and odd harmonics at full crank (only below 10K, reduced above 10K)

Also, clearly not downsampling to 44.1/16, but that was always a very, very silly suggestion that has been conclusively debunked.
Old 5th March 2015
  #5313
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
But you can see why: to drop the noisefloor of the measurement ADC to nearly -150db, greater than you can get out of a straight 24 bit capture.
The noise floor is not nearly -150 dBFS, it's much higher for all of your signals. Span is showing that there is approx. a minimum of -150 dBFS in each bin, so to find the actual noise floor, all these values must be integrated after removing the input signal and distortion products. The relative visual difference between the levels at which the noise floor of each trace intersects the amplitude axis is fairly accurate, but the absolute noise floor of each is not.

Last edited by -tc-; 5th March 2015 at 11:34 PM..
Old 5th March 2015
  #5314
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by -tc- View Post
The noise floor is not nearly -150 dBFS, it's much higher for all of your signals. Span is showing that there is approx. a minimum of -150 dBFS in each bin, so to find the actual noise floor, all these values must be integrated after removing the input signal and distortion products. The relative visual difference between the levels at which the noise floor of each trace intersects the amplitude axis is fairly accurate, but the absolute noise floor of each is not.
*edit* to drop the noise floor to 'nearly eighteen db below what even the Pono player can do, much less the iPhone, so that we can compare them without muddying things up with the noise floor of the recording device'.

Old 6th March 2015
  #5315
Gear Guru
 
UnderTow's Avatar
How about full scale white noise from both devices at 44.1 KHz. (Recorded at 96 KHz). And also compare the Pono and iPhone both at 48 KHz rather than one at 48 and one at 192 KHz.

Alistair

Last edited by UnderTow; 6th March 2015 at 01:13 AM..
Old 6th March 2015
  #5316
Gear Guru
 
UnderTow's Avatar
How about full scale white noise from both devices at 44.1 KHz. (Recorded at 96 KHz). And also compare the Pono and iPhone both at 48 KHz rather than one at 48 and one at 192 KHz.

As for the roll off, that might be due to half band filters (computationally cheap).

Alistair
Old 6th March 2015
  #5317
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
Thing sounds awesome: the behavior under test shows markedly better performance than a known-good normal device, apart from increased even and odd harmonics at full crank (only below 10K, reduced above 10K)
I think I misread a post of yours above actually...

I have to wonder about the above though. Nobody listens to content at -96dBFS and as a matter of fact most listen to content that MUCH louder than that. So a real world test based on technology would be a signal that would live at some sort of "optimal" or "nominal" operating level. 0dBFS is probably closer to what we'd expect to see in modern music compared to -96dBFS. So as far as I see it the latter test, despite being unnecessarily hot in my opinion, is telling us more.

Now, what it appears to be telling us is what you wrote above, but I fail to see how that is good. More harmonics below 10kHz is in my opinion worse than above 10kHz, and have fewer above won't mitigate that.

I'd be more interested in seeing signal around -10dBFS or so.
Old 6th March 2015
  #5318
Gear Guru
 
UnderTow's Avatar
Quote:
Originally Posted by mattiasnyc View Post
Now, what it appears to be telling us is what you wrote above, but I fail to see how that is good. More harmonics below 10kHz is in my opinion worse than above 10kHz, and have fewer above won't mitigate that.
Agreed. It looks like the Pono distorts more (which is exactly what I said a long time ago based on what people were describing).

But! I don't really think this testing means much to be honest. There are way too many issues with the methodology.

Alistair

Last edited by UnderTow; 6th March 2015 at 12:53 AM..
Old 6th March 2015
  #5319
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by UnderTow View Post
How about full scale white noise from both devices at 44.1 KHz. (Recorded at 96 KHz). And also compare the Pono and iPhone both at 48 KHz rather than one at 48 and one at 192 KHz.
I would have compared the iPhone at 192K, or 96K, if it could do it. I tried 96K first. It won't do anything of the sort so it was 48K or nothing. That's supposed to be the same.

Pono is designed to do 192K. Heck, even my own recordings are just 96K! But it's designed to work at 192K, plus there's been all kinds of FUD about how that is going to end up worse, be more noisy, create problems.

Nope, apparently. It handily and conclusively outperforms the very respectable iPhone while running at 192K.

Since that is the main selling point (whether or not it SHOULD be), not to test it as that would be dishonest. I didn't downsample the test files for the iPhone, I generated new files directly at each desired sample rate so they'd be exactly the same.

What do you think you're going to see from full scale white noise from each device at 44.1K? It's do-able. Do you think the stop-band is going to behave differently? It hadn't occurred to me there would be any difference in the behavior. I've not heard any reports of Pono treating a sample rate that low, as different. It's the 96K and the 192K that are special cases, that's why I measured them, and the results are interesting and not nearly as odd as I thought they were going to be.
Old 6th March 2015
  #5320
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by UnderTow View Post
Agreed. It looks like the Pono distorts more (which is exactly what I said a long time ago based on what people were describing).

But! I don't really think this testing means much to be honest. There are way too many issues with the methodology.

Alistair
The great thing about us being on Gearslutz as opposed to some high end listener site is, we all KNOW what the harmonic partials mean to the sound. iPhone's increased odd harmonics that extend higher in frequency mean a 'solid state' type of distortion, an edgy sound. They're present even at -96db, which is kind of disturbing and not true of the Pono.

Pono's increased even harmonics and the distribution in lower partials is characteristic of what we as synth/gear/measurement nerds describe as more tubelike, more soft clipping versus a hard discontinuity. The spectral content of these things gives us the perceived effect, the tone color.

I'm not sure when the harmonics kick in for the Pono. I suspect it's a soft clip, where the iPhone seems to be generating overtones at all levels (Pono goes totally harmonic-less below some threshold, I'm not sure what). I could test that out.

As for methodology, I have the physical device and I'm playing test tones through it and capturing them to an ADC over a very short shielded wire with extremely low capacitance and inductance. I don't know what more you could want, for testing the thing. Again: I have one, I'm showing what it does on test tones and things. Surely this beats sitting around saying 'I heard it downsamples to 16/44.1 because it's Android'?
Old 6th March 2015
  #5321
Lives for gear
 
stratology's Avatar
 

Quote:
Originally Posted by chrisj View Post
As for methodology, I have the physical device and I'm playing test tones through it and capturing them to an ADC over a very short shielded wire with extremely low capacitance and inductance. I don't know what more you could want, for testing the thing.
Simple: not having another ADC in the chain.
Old 6th March 2015
  #5322
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
The great thing about us being on Gearslutz as opposed to some high end listener site is, we all KNOW what the harmonic partials mean to the sound. iPhone's increased odd harmonics that extend higher in frequency mean a 'solid state' type of distortion, an edgy sound. They're present even at -96db, which is kind of disturbing and not true of the Pono.

Pono's increased even harmonics and the distribution in lower partials is characteristic of what we as synth/gear/measurement nerds describe as more tubelike, more soft clipping versus a hard discontinuity. The spectral content of these things gives us the perceived effect, the tone color.
I'm not following you now. In which images do we see that one device favors odd harmonics and the other even?

Also, again, what amount of music that is currently available for the Pono has a significant amount of content audible at such low levels without being obscured by higher level content? I'm basically pondering whether or not the average mix/master of what is likely to sell the most for this medium will live closer to 0dBFS or -96dBFS. If the low level stuff is obscured not only by high level stuff but also its generated harmonics, it seems it's a possibly meaningless distinction.
Old 6th March 2015
  #5323
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by stratology View Post
Simple: not having another ADC in the chain.
How do you make measurements without capturing the analog signal?

Quote:
Originally Posted by mattiasnyc View Post
I'm not following you now. In which images do we see that one device favors odd harmonics and the other even?

Also, again, what amount of music that is currently available for the Pono has a significant amount of content audible at such low levels without being obscured by higher level content? I'm basically pondering whether or not the average mix/master of what is likely to sell the most for this medium will live closer to 0dBFS or -96dBFS. If the low level stuff is obscured not only by high level stuff but also its generated harmonics, it seems it's a possibly meaningless distinction.
But I've only measured two levels (at least, so far). All we know is that at FS and -96db the iPhone generates largely odd harmonics and is still doing it at whisper levels. The Pono player has around 12 db better noise floor, and at full scale it generates a different spectrum of harmonics, but at -96 it generates NONE though it still can pick up interference when it's bad. I don't know what the transition point is, but if there is obscuring by harmonics it would appear the iPhone is doing it all the time and the Pono player is totally free of added harmonics below a threshold we've not yet determined.

If it's truly a softclip, there'll be a point where the two are generating the same amount of harmonics (but distributed differently, with the iPhone giving higher harmonics that are generally considered more audible: it's very hard to hear presence of second harmonic against a sine, compared with say seventh. I'm not sure what that point is but I can find out. They're both doing very well, really: not even the loudest hyperlimited music is as much stress on the device as a single sine at full scale. That not only blasts the output to continuous full effort, it's completely open to measure artifacts and harmonics, which is why we do it.

Last edited by chrisj; 6th March 2015 at 01:47 AM..
Old 6th March 2015
  #5324
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
The Pono player has around 12 db better noise floor, and at full scale it generates a different spectrum of harmonics,
Which image shows that?
Old 6th March 2015
  #5325
Lives for gear
 
stratology's Avatar
 

Quote:
Originally Posted by chrisj View Post
How do you make measurements without capturing the analog signal?
IIRC, Monty's test set up did not convert analog signals back to digital before taking measurements.

Here are test results from an iPhone 6 Plus, tests performed with dedicated hardware.
Old 6th March 2015
  #5326
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by stratology View Post
IIRC, Monty's test set up did not convert analog signals back to digital before taking measurements.

Here are test results from an iPhone 6 Plus, tests performed with dedicated hardware.
Are you seriously trying to say you need a ten thousand dollar spectrum analyzer capable of measuring into the gigahertz to deliver meaningful relative comparison of two consumer audio devices.

I think you'll find those things also digitize in order to analyze, though the code is probably a lot trickier to get analysis bins out of sampling rates in the gigahertz.

As for the 'how are the harmonic spectra different', that is in post https://www.gearslutz.com/board/10870398-post5304.html and I could say 'if you can't figure out the one with Pono slightly offset, I'll post the source captures'… except that in that post, I already did post both examples the composite image was made from.

The overlay images make it easier to see distinctions such as the markedly lower noise floor of Pono, or the way its noise level rises to nearly the level of iPhone as it nears 50K. But if you know what to look for, the harmonic contents of each are present in their own spectrograms and they're plainly different. It's a cumulative average taken over nearly a minute of sample time and it is accurate waaaaaay beyond anything humans can hear, even if they could hear significantly below a 24-bit threshold, which they can't.

And it shows why Pono sounds a lot better than typical smartphone audio, or even something as impressive as an iPhone, and it shows that some of the quirky choices like averaging haven't produced any untoward effects… especially if you're going with 192K. MAYBE a rolloff over 30K is like floating in water versus levitating. I don't know. To me it sounds absolutely fine, terrific. I think the politeness I hear is nothing more than total lack of transition band ripple or filter ring: I expect to sense some energy up there, and it's gone.

I can get used to that, because brickwall filter ring is unnatural. Worth every penny IMO.
Old 6th March 2015
  #5327
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
As for the 'how are the harmonic spectra different', that is in post https://www.gearslutz.com/board/10870398-post5304.html and I could say 'if you can't figure out the one with Pono slightly offset, I'll post the source captures'… except that in that post, I already did post both examples the composite image was made from.

The overlay images make it easier to see distinctions such as the markedly lower noise floor of Pono, or the way its noise level rises to nearly the level of iPhone as it nears 50K. But if you know what to look for, the harmonic contents of each are present in their own spectrograms and they're plainly different.
On both I see peaks at 1k intervals up to 10k. Going from 1k, the difference relative to each prior frequency is

2k: lower
3k: lower
4k: lower
5k: higher
6k: lower
7k: higher
8k: higher
9k: lower
10k: higher

That's the same for both devices. How is one emphasizing odds and the other evens? I mean, if those images really show the difference then please educate me on how to read them, but I honestly don't understand.
Old 6th March 2015
  #5328
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
Are you seriously trying to say you need a ten thousand dollar spectrum analyzer capable of measuring into the gigahertz to deliver meaningful relative comparison of two consumer audio devices.

I think you'll find those things also digitize in order to analyze, though the code is probably a lot trickier to get analysis bins out of sampling rates in the gigahertz.

As for the 'how are the harmonic spectra different', that is in post https://www.gearslutz.com/board/10870398-post5304.html and I could say 'if you can't figure out the one with Pono slightly offset, I'll post the source captures'… except that in that post, I already did post both examples the composite image was made from.

The overlay images make it easier to see distinctions such as the markedly lower noise floor of Pono, or the way its noise level rises to nearly the level of iPhone as it nears 50K. But if you know what to look for, the harmonic contents of each are present in their own spectrograms and they're plainly different. It's a cumulative average taken over nearly a minute of sample time and it is accurate waaaaaay beyond anything humans can hear, even if they could hear significantly below a 24-bit threshold, which they can't.

And it shows why Pono sounds a lot better than typical smartphone audio, or even something as impressive as an iPhone, and it shows that some of the quirky choices like averaging haven't produced any untoward effects… especially if you're going with 192K. MAYBE a rolloff over 30K is like floating in water versus levitating. I don't know. To me it sounds absolutely fine, terrific. I think the politeness I hear is nothing more than total lack of transition band ripple or filter ring: I expect to sense some energy up there, and it's gone.

I can get used to that, because brickwall filter ring is unnatural. Worth every penny IMO.
Thanks for taking time to do this and posting your findings!
Old 6th March 2015
  #5329
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by mattiasnyc View Post
On both I see peaks at 1k intervals up to 10k.
2K iPhone: -88 db
2K Pono: -70 db

3K iPhone: -104 db over noise floor of -120
3K Pono: -82 db over noise floor of -126

and so on. Just like if you refer to the -96 db 'low level detail' one, you get

3K iPhone: -96 db over noise floor of -120
3K Pono: no harmonic, noise floor at -132

Seems like there's a difference and unless you crank up the gains on the Focusrite it masks the true noise floor of the Pono. When you listen over headphones loud you can hear the difference, though.

Anyway, that's how you read amplitudes of harmonics relative to each other.
Old 6th March 2015
  #5330
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
2K iPhone: -88 db
2K Pono: -70 db

3K iPhone: -104 db over noise floor of -120
3K Pono: -82 db over noise floor of -126

and so on. Just like if you refer to the -96 db 'low level detail' one, you get

3K iPhone: -96 db over noise floor of -120
3K Pono: no harmonic, noise floor at -132

Seems like there's a difference
I was asking about whether or not there was a difference in which harmonics are being emphasized by each device. As you can see above the relationship is the same - going from 2k to 3k BOTH devices show a lower harmonic at 3k.

So again: Where do you see odd vs even harmonics being emphasized differently in the two devices?
Old 6th March 2015
  #5331
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by mattiasnyc View Post
I was asking about whether or not there was a difference in which harmonics are being emphasized by each device. As you can see above the relationship is the same - going from 2k to 3k BOTH devices show a lower harmonic at 3k.

So again: Where do you see odd vs even harmonics being emphasized differently in the two devices?
Check 4 and 5, and 6 and 7.
Old 6th March 2015
  #5332
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
Check 4 and 5, and 6 and 7.
I did.I showed you how the two favor the same harmonics.
Old 7th March 2015
  #5333
Old 7th March 2015
  #5334
Airwindows
 
chrisj's Avatar
Jules, would you like any additional measurements? Such as they are?
Old 7th March 2015
  #5335
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
Jules, would you like any additional measurements? Such as they are?
Since I have your attention, here's what I think it looks like when one set of (in this case odd) harmonics is emphasized:



Not seeing that in your screenshots.

What am I missing?
Old 7th March 2015
  #5336
Lives for gear
 
doom64's Avatar
Thanks for your tests, chrisj! Interesting stuff.

I'd love to see a loopback test with your converter. Compare that to RMAA ( Latest News. Audio Rightmark ) files loaded into the Pono and iPhone.

Also, were tests done with the iPhone in airplane mode with WiFi off?

I wish someone could lend you a Mytek converter.
Old 7th March 2015
  #5337
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by mattiasnyc View Post
Since I have your attention, here's what I think it looks like when one set of (in this case odd) harmonics is emphasized:
Not seeing that in your screenshots.
What am I missing?
I can't just manufacture results like that. It seems to me that Pono is emphasizing even harmonics (low orders like fourth and sixth) slightly more than the iPhone (a 4s, by the way: fairly but not very recent, they've been good players for a long time). All I can show is what the things did, over how I have SPAN set. I'm more interested in relative differences and didn't realize the full extent of the desire for stuff set up like $10,000 dedicated measurement equipment does it. Things like noise floor are accurately represented relative to the other player as they're measured, sampled and charted the same way.


Quote:
Originally Posted by doom64 View Post
Thanks for your tests, chrisj! Interesting stuff.
I'd love to see a loopback test with your converter. Compare that to RMAA ( Latest News. Audio Rightmark ) files loaded into the Pono and iPhone.
Also, were tests done with the iPhone in airplane mode with WiFi off?
I wish someone could lend you a Mytek converter.
I'm assuming my converter is garbage. I'm not happy with the sound of it compared to what I had: I had an old Apogee RosettaAD, and went to the Scarlett both to get a USB connection (my iMac can no longer receive optical digital as the laptop did) and to get 96K sampling. I didn't lose that much once I use the 96K effectively, but I'm scrambling to work around the inadequacies of the converter, hence the messing with input levels and digitally adjusting for scale.

The iPhone tests were NOT done in airplane mode with wifi/bluetooth off. A silly oversight, yeah, but also representative of how the device normally works. It really is doing very well. I don't think the extreme treble artifacts are that (though the iPhone WAS nearby while the Pono was being tested. Maybe it was the iPhone causing that, and not the nearby computer screen? I know those artifacts are still bugging me)
Old 7th March 2015
  #5338
Lives for gear
 

Quote:
Originally Posted by chrisj View Post
I can't just manufacture results like that. It seems to me that Pono is emphasizing even harmonics (low orders like fourth and sixth) slightly more than the iPhone (a 4s, by the way: fairly but not very recent, they've been good players for a long time). All I can show is what the things did, over how I have SPAN set. I'm more interested in relative differences and didn't realize the full extent of the desire for stuff set up like $10,000 dedicated measurement equipment does it.
But I'm telling you: Neither device seems to emphasize either odds or evens. If you look at ONLY the Pono you will see that the signal does NOT emphasize odds or evens. If it did, it would look more like the graph I showed you - i.e. every other harmonic would be louder. The same is true for the iPhone.

Just because the levels of ALL harmonics within a range are lower or higher in one device relative to another doesn't mean that it's "odds" or "evens" - you could get the same effect by just adjusting the volume....
Old 9th March 2015
  #5339
Lives for gear
 
bogosort's Avatar
Quote:
Originally Posted by chrisj View Post
This is a plot of generated white noise at 96K and at 192K, played by the Pono. We can see that at 96K, there is averaging in effect and it is starting to roll off by 30K and clearly attentuated by the time it hits Nyquist, but not a total cancellation either.
That is not the frequency response of a moving average filter. The roll-off you see is consistent with a standard LPF: the midrange gain is -42 dB, which makes the cutoff at -45 dB, roughly 43 kHz on your plot.

I've re-attached your image and added the frequency response of a 2-tap moving average filter at 96 kHz. Clearly these are not the same type of filters.
Attached Thumbnails
Launch of Pono-pono_white_noise_96k.jpg   Launch of Pono-ma_2taps.png  
Old 9th March 2015
  #5340
Lives for gear
 
bogosort's Avatar
Quote:
Originally Posted by chrisj View Post
The great thing about us being on Gearslutz as opposed to some high end listener site is, we all KNOW what the harmonic partials mean to the sound. iPhone's increased odd harmonics that extend higher in frequency mean a 'solid state' type of distortion, an edgy sound. They're present even at -96db, which is kind of disturbing and not true of the Pono.

Pono's increased even harmonics and the distribution in lower partials is characteristic of what we as synth/gear/measurement nerds describe as more tubelike, more soft clipping versus a hard discontinuity. The spectral content of these things gives us the perceived effect, the tone color.
Ah, the old myth of "tube-like" harmonic distortion. There is no correlation between amplifier technology (vacuum tube vs solid state) and harmonic distortion characteristics; the circuit design, not the amplifying device, is what determines which harmonics, if any, are generated. In any case, harmonic distortion is only meaningful as a measure of linearity -- we cannot predict how a circuit will sound based on its harmonic distortion. Real world (i.e., non-sinusoidal) signals generate non-harmonic IMD when presented with a nonlinearity; the harmonics are a tiny fraction of the overall sound.

Back to your analysis: the spectral components in the full-scale plot of both devices are characteristic of symmetric distortion: even and odd distortion products. Both devices showed remarkably similar distortion (too similar?), though the magnitude of the distortion was clearly worse in the Pono. I don't understand why you think the Pono did well here.
Topic:
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
Thread Tools
Search this Thread
Search this Thread:

Advanced Search
Forum Jump
Forum Jump