The No.1 Website for Pro Audio
 Search This Thread  Search This Forum  Search Reviews  Search Gear Database  Search Gear for sale  Search Gearslutz Go Advanced
Launch of Pono Studio Headphones
Old 17th April 2014
  #3211
Quote:
Originally Posted by Bob Olhsson View Post
This is possibly the most important post on the subject of audio quality ever!
It's a very good point when we're testing codecs and gear.

When we're testing the population for average ranges of perception, it's obviously completely appropriate (critical, actually) to use random population samples.

But if we are looking for defects in a codec or repro system, we are not looking for an average range of human perceptual abilities, we are looking for any problems that can be reliably determined by our trained listeners. We want the certainty that double blind testing brings -- but we're not in this particular case concerned about the average listener's capabilities, rather our goal is to improve the system as much as possible within the design constraints of the system being addressed.

Two very different endeavors.
Old 17th April 2014
  #3212
j_j
Lives for gear
Quote:
Originally Posted by theblue1 View Post
When we're testing the population for average ranges of perception, it's obviously completely appropriate (critical, actually) to use random population samples.
You're forgetting that after exposure to a new technology for months the average listener will start to learn, and move toward an informed, if not expert, listener.

A wonderful example of this is the arrival of MP3, with people talking about how it sounds GREAT at 64kb/s for a stereo pair.

And then they learned the artifacts, and it sounded like ****. And so we went to 96, and 128, and now 320.

Ditto for AAC, AC3, etc.

Interestingly, it's also very true that the trained listener tolerance when down a lot. What passed MPEG-1 with a '4' ITU score was barely on the bottom of the scale by the end of MPEG-2.
Old 17th April 2014
  #3213
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by j_j View Post
***ALWAYS*** USE TRAINED, EXPERT LISTENERS.

Yes, I am shouting.

Why? Because when you sell it to the population, they'll eventually be trained. Eventually. But the results may just turn out to be disinterest or "does not buy" rather than explicit "this problem exists".
Quote:
Originally Posted by Bob Olhsson View Post
This is possibly the most important post on the subject of audio quality ever!
YES. Re-read. Learn.

The consequence of delivering mediocre audio to mediocre listeners is mediocre sales, and doldrums for all of us. People will respond to things before they rationalize their response, and the rationalization you get from mediocre sound is "this isn't worth my time or money as it isn't very good".

We WANT the reactions shown in the Pono marketing. Ideally, they'll be justified, but we must have them nevertheless if our industry is going to be exciting and rewarding.
Old 17th April 2014
  #3214
Lives for gear
 
3rd Degree's Avatar
 

Quote:
Originally Posted by j_j View Post
Seriously, if you can re-run the stimuli that you did well on, see if you do well the next time.

If you can get 8/10 on each of 5 sets, you've pretty much shown that you can indeed hear something. Even 8/10 on 2 sets is pretty conclusive.

But just add up all the successes and all of the trials. That's how to calculate the statistics. Otherwise you could hide some marginal results.
I had meant to respond to your comment but I think many shorter tests are a better way for an individual to test. I will try again with more tests when I have time and see how I do.
Old 17th April 2014
  #3215
Quote:
Originally Posted by j_j View Post
You're forgetting that after exposure to a new technology for months the average listener will start to learn, and move toward an informed, if not expert, listener.

A wonderful example of this is the arrival of MP3, with people talking about how it sounds GREAT at 64kb/s for a stereo pair.

And then they learned the artifacts, and it sounded like ****. And so we went to 96, and 128, and now 320.

Ditto for AAC, AC3, etc.

Interestingly, it's also very true that the trained listener tolerance when down a lot. What passed MPEG-1 with a '4' ITU score was barely on the bottom of the scale by the end of MPEG-2.
Oh, I totally agree.

I was trying to make a distinction between that sort system performance testing and the very different sort of testing that perceptual scientists do to find general baselines of human perception. (One thing, of course, that doesn't change is the necessity to have a way of giving us confidence that forms of cognitive bias are not skewing results.)

In fact, I had at one point put in a paragraph in that post on the 'learning' of lossy codecs by regular listeners and the upward trajectory of 'minimum' quality levels discernible by those listeners, along the lines of what you write above. Maybe I should have left it in.


[And, of course, if it seems like I'm 'explaining' things you, JJ, already clearly know, it's just because I'm trying to frame things in such a way that others may understand, since I've been in their shoes, too, and am something of a convert to the notion that understanding digital 'well enough' usually doesn't work out. 'Common sense' only applies when one understands the fundamental operations and logic of a given system, I think now. I went into my understanding of digital long ago with the idea that common sense could carry me through my lack of understanding of the underlying math -- but the more I've learned, the more I realize that 'black boxing' that which is not quite understood is a risky proposition at best.]
Old 17th April 2014
  #3216
Motown legend
 
Bob Olhsson's Avatar
 

Edison's acoustical phonograph passed lots of public blind tests which were an integral part of his sales pitch!
Old 17th April 2014
  #3217
Airwindows
 
chrisj's Avatar
There might also be a correlation between the 'naive listener instinctive reaction' aspect jj's talking about, and the notorious Jam Idiot thing Malcolm Gladwell popularized in 'Blink'. (original experiment by Jonathan Schooler and Timothy Wilson)

It suggests that when people just experience sensory input they aren't bad at matching expert opinion, but as soon as they've got to work at rationalizing the basis for their decision they absolutely fall apart. Yet, the process of ABX testing requires just such rationalizing (or at least makes it tantalizingly available). It's tempting to endlessly refer back to the source files, fiddling with the information to try and find 'tells'.

Might be better to get a general sense of A versus B, and then wing it through a bunch of trials, 'guessing' rather than rationalizing and trying to prove to yourself that X is one or the other! This is very much what I did (at high volumes) with the ABX files in this thread, trying to get 'runs' of confident answers and not refamiliarizing myself too often with the sources. The one that I totally failed, was all about relistening to A versus B (not that I would have got it in any event, it was the harpsichord one).

If Edison's original phonograph passed many blind tests against live performance in its day, I would say the blind tests of that era were not very sensitive to real differences
Old 17th April 2014
  #3218
Lives for gear
 
Ephi82's Avatar
 

Hey lets talk dollars and cents for a moment:

I think everyone agrees PONO os going to have an uphill fight to penetrate the portable listening device market.

If you are pessimistic, you probably say no more than 2% over a few years.

Optimistic, 7% ?

Get this:

Apple sold 1.3 Billion IPODS in five years leading up to 2012.

At a 7% penetration, PONO would have sold 96 million players over the same time equaling over $ 36 BILLION in revenues for just players.

At 2% penetration, the would have sold 26 million players and over $10 BILLION in revenues

For investors in PONO, if its managed well, that would not suck!

Just sayin.......
Old 17th April 2014
  #3219
j_j
Lives for gear
Quote:
Originally Posted by 3rd Degree View Post
I had meant to respond to your comment but I think many shorter tests are a better way for an individual to test. I will try again with more tests when I have time and see how I do.
Listener fatigue is very real.

Short tests is the way to go, indeed. But not so short you forget what you're listening for between tests.
Old 17th April 2014
  #3220
j_j
Lives for gear
Quote:
Originally Posted by chrisj View Post
If Edison's original phonograph passed many blind tests against live performance in its day, I would say the blind tests of that era were not very sensitive to real differences
Everything depends on the instructions and the expectations of the listener.

For an untrained person with low expectations, being able to understand what Caruso was singing would be "the same".
Old 17th April 2014
  #3221
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by Ephi82 View Post
Apple sold 1.3 Billion IPODS in five years leading up to 2012.

At a 7% penetration, PONO would have sold 96 million players over the same time equaling over $ 36 BILLION in revenues for just players.

At 2% penetration, the would have sold 26 million players and over $10 BILLION in revenues

For investors in PONO, if its managed well, that would not suck!

Just sayin.......
Never get sucked into thinking, 'if I can only get 1% of all the people in China to buy my thing, why, I'll be mega-rich!'. Not unless you're owned by Bain Capital or something, in which case you have bigger problems.

Pono isn't being funded by venture capital, it's funded (more than six times over) by Kickstarter. The initial launch is paid for, and the contributors are only billed when the kickstarter successfully concludes. Now they've got to build them, which doesn't seem that challenging to do. They're certainly not pushing the limits of miniaturization technology!
Old 17th April 2014
  #3222
Gear Head
 

Quote:
Originally Posted by animal eater View Post
I don't have the time or energy to read this book of a thread. Who out there on this site has purchased this and what are your thoughts?
I don't think anyone has actually received one yet... I had the pleasure though to listen to a demo at another forum members house given by Pono. We listened to various music that were FLAC recordings and then also the music that was on the Pono. I thought the FLAC recordings sounded great but the recordings we heard on the Pono were outstanding... Now to be fair we did not have the same recordings for both players. I do have to say though that the recordings we heard on the Pono sounded better than any recordings I have ever heard.
Old 17th April 2014
  #3223
j_j
Lives for gear
Quote:
Originally Posted by ellisr63 View Post
I don't think anyone has actually received one yet... I had the pleasure though to listen to a demo at another forum members house given by Pono. We listened to various music that were FLAC recordings and then also the music that was on the Pono. I thought the FLAC recordings sounded great but the recordings we heard on the Pono were outstanding... Now to be fair we did not have the same recordings for both players. I do have to say though that the recordings we heard on the Pono sounded better than any recordings I have ever heard.
So, should we make a player that does processing that mimics LP and tape distortions (without the added random noise, pops, and clicks), and then adds those to the digital output?

What do you think? :p
Old 17th April 2014
  #3224
Quote:
Originally Posted by ellisr63 View Post
I don't think anyone has actually received one yet... I had the pleasure though to listen to a demo at another forum members house given by Pono. We listened to various music that were FLAC recordings and then also the music that was on the Pono. I thought the FLAC recordings sounded great but the recordings we heard on the Pono were outstanding... Now to be fair we did not have the same recordings for both players. I do have to say though that the recordings we heard on the Pono sounded better than any recordings I have ever heard.
Your post is most welcome; AFAIK it's the 1st first-person account in this thread. There have been some criticisms of the listeners responses in the Pono video (which I think were unwarranted); could I ask: Did you consider the audio quality to be so outstanding as to outweigh any possible expectation bias? Could you describe (in an explicit sense) the particular qualities that made Pono outstanding e.g. sense of depth, realism of soundstage, etc.? Thank you!
Old 17th April 2014
  #3225
Gear Guru
 

Quote:
Originally Posted by chrisj View Post
Pono isn't being funded by venture capital, it's funded (more than six times over) by Kickstarter.
this is not correct
the Kickstarter money will be used to attract traditional investors according to this article in The Guardian

Quote:
Young first raised $500,000 in 2012 to design prototypes and build infrastructure, before using Kickstarter, where members of the public each fund a small amount of money, to raise awareness and further funding. He'll now use venture capital to help bring the project to market, sharing equity in the company with traditional investors.

"I can't tell you how scary this is," Young said in 2012 regarding his move into the world of financing. "All I have to do now is navigate the waters of venture capitalism, those treacherous shorelines of commerce, in the HMS Pono."
Apparently a mere $6 million is not enough to bring a product like this to market. The kickstarter is more like "proof of public interest" in the product. This hard evidence of firm support can be used to attract venture capitalists to put in the real money.

Lord only knows how much money that actually is.
Old 17th April 2014
  #3226
Lives for gear
 
paul brown's Avatar
Quote:
Originally Posted by ellisr63 View Post
Now to be fair we did not have the same recordings for both players.
this vital piece of information, what are the files being compared in the demonstrations, is not mentioned. even yourself, as a firsthand listener, were unaware of this information. why is this not mentioned? without meaningful measurements of the player or meaningful data from the listening tests of people like yourself and others on video, how can we make an informed decision? it ends up as a great deal of talk about something of which we know very little. excellent marketing! framing effect, illusory correlation, halo effect, authority bias, etc.

i'm sure it is a 'great' portable player designed to be manufactured at a price point to make a profit. not a revolution.
Old 17th April 2014
  #3227
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by joeq View Post
Apparently a mere $6 million is not enough to bring a product like this to market. The kickstarter is more like "proof of public interest" in the product. This hard evidence of firm support can be used to attract venture capitalists to put in the real money.

Lord only knows how much money that actually is.
Nope, the kickstarter is the initial run. You're talking about a further, mass-market stage, and yes you're right that it won't become another iPod on six million dollars. I don't think it's going to become another iPod at all, but I do think it'll find a niche: sort of the iPod of iPods, as it were. iPod itself is not all of the 'cheap mp3 player' market, but it's the luxury brand. Pono will further segment that market, and is well on its way to doing just that.

Maybe Apple will buy it. Then it'd have all the financing you could want
Old 18th April 2014
  #3228
Lives for gear
 
Ephi82's Avatar
 

Quote:
Originally Posted by chrisj View Post
Never get sucked into thinking, 'if I can only get 1% of all the people in China to buy my thing, why, I'll be mega-rich!'. Not unless you're owned by Bain Capital or something, in which case you have bigger problems.

Pono isn't being funded by venture capital, it's funded (more than six times over) by Kickstarter. The initial launch is paid for, and the contributors are only billed when the kickstarter successfully concludes. Now they've got to build them, which doesn't seem that challenging to do. They're certainly not pushing the limits of miniaturization technology!
Chris, my point is that many folks have said that PONO will have extremely limited penetration and say it won't survive. (Ie Most people don't care about better sound)

The math I provided shows that at only 2% adoption, PONO can be very successful from a business perspective and get a solid foothold. 2 people out of one hundred is what it will take.

This is a fresh perspective that's not endlessly arguing Nyquist, filters yada, yada, yada, probably not welcomed here!

And by the way, this is a world wide market, not just China, and business people and investors the take prospects of penetrating existing markets very seriously.

Based what I read about PONOs leadership, there will be no shortage of heavy hitters coming to the table.
Old 18th April 2014
  #3229
Problem is it's music only, and music only gadgets are in steep decline.
Some people may care about audio quality, but the vast majority of people now expect an all in one entertainment, information and communication device.
Over the last quarter of 2013 (Black Friday, Cyber Monday and Christmas etc) Apple sold 51 million iPhones and 26 million iPads, but only 6 million iPods.
Pono may be an iPod killer, but the iPod was already dead.
Who is going to carry two devices around (Pono and smartphone) when the one smart phone or tablet offers pretty damn good audio, plus everything else….
The age of the iPod is over | The Verge
Old 18th April 2014
  #3230
Gear Guru
 

Quote:
Originally Posted by chrisj View Post
Nope, the kickstarter is the initial run.
apparently you did not read the article I linked to, or even the part I quoted.

Quote:
... to raise awareness and further funding. He'll now use venture capital to help bring the project to market, sharing equity in the company with traditional investors.
the $6 million is to raise awareness and further funding. To "bring it to market" the article is saying he will NEED that further funding. Of course they want their kickstarter participants to "feel" like they alone are "making it happen", but that's part of the fantasy, now isn't it?
Old 18th April 2014
  #3231
Airwindows
 
chrisj's Avatar
Quote:
Originally Posted by joeq View Post
apparently you did not read the article I linked to, or even the part I quoted.
the $6 million is to raise awareness and further funding. To "bring it to market" the article is saying he will NEED that further funding. Of course they want their kickstarter participants to "feel" like they alone are "making it happen", but that's part of the fantasy, now isn't it?
Kickstarter doesn't run false projects that claim to bring initial production runs, and then put it off until bigger investors arrive. You're mistaken. What's been promised in the Kickstarter can be delivered (it's not really that huge a production run and well within their capabilities).

Scaling it up to a bigger scale is another question entirely, and an interesting one, but Pono is already brought 'to market' in that the whole first production run and a host of special-issue signed first editions are spoken for, and the Kickstarter people paid for that, six times as much as the Pono team needed to put out those first units.

They have The Eagles, for Pete's sake. Don't even start with the 'how shall they ever attract investors'? They have the Eagles.
Old 18th April 2014
  #3232
A Kickstarter project isn't regulated by strict law.
Against my better judgement I donated to a similar project (on a different but well known crowd funding site) and the makers broke several promises when it came down to it. They got their initial funding, but forgot their seed funders once they saw other ways they could make money and propel their product forward.
Old 18th April 2014
  #3233
Gear Maniac
 

Quote:
Edison's acoustical phonograph passed lots of public blind tests
Well it is still the best way to capture audio. No analogues involved at all.

Attach a lever to a diagram and directly mark the results to a physical medium. Reverse that process and you create movement in a diaphragm identical to the received signal.

High school physics. Move a magnet in a coil and you generate a current. A microphone. Apply a current to a magnet in a coil you generate movement. A Loudspeaker.

Analogue audio isn't so called because it is a continuous signal. It's because the voltage generated is analogous to the atmospheric pressure at a specific time.

ADC converters measure voltage and convert it to a number. DACs take that number and change it back into a voltage. A sort of double analogue.

So it's a wonderful irony that this exact conversation took place a century ago.

When ye olde subjectivists insisted only direct etching to wax conveyed the true essence of the sound and ye olde objectivists insisted the analogue sound was more accurate.

I hope they are both laughing in their graves.
Old 18th April 2014
  #3234
Gear Guru
 

Quote:
Originally Posted by RonaldDumsfeld View Post
...Reverse that process and you create movement in a diaphragm identical to the received signal.... When ye olde subjectivists insisted only direct etching to wax conveyed the true essence of the sound
Well they have a point, it's direct, direct from Enrico Caruso's actual vocal cords through the air to a needle!

Unfortunately that needle is located at the bottom of a very large cone. So it sounds like someone is holding his nose while he sings. But damn the artifacts, conceptually, it's "more pure" than a microphone!


I have read about Edison's test involving the phonograph vs a real trombone. A phonograph being totally passive, I would think any modern listener would have no problem, just based on volume alone. Was it because those people were totally new to even the concept of sound reproduction?

I used to know a guy who did stage hypnotism. He said his act of getting people to do weird stuff was based almost entirely on the fact that nobody wants to be "the guy who spoils the fun". I wonder if Edison's audience and the people in the Pono video have something in common along those lines.
Old 18th April 2014
  #3235
Lives for gear
 
Sacalait's Avatar
I didn't read this entire thread but my two sense... is 1)vinyl seems to making a comeback with the kids who didn't know what the freak vinyl ever was!! and 2) great songs (like ALL the songs on Neil Young's Harvest) are awesome on a cassette, vinyl, CD, pono, iPod, blah blah blah.
Old 18th April 2014
  #3236
Quote:
Originally Posted by joeq View Post
Well they have a point, it's direct, direct from Enrico Caruso's actual vocal cords through the air to a needle!

Unfortunately that needle is located at the bottom of a very large cone. So it sounds like someone is holding his nose while he sings. But damn the artifacts, conceptually, it's "more pure" than a microphone!


I have read about Edison's test involving the phonograph vs a real trombone. A phonograph being totally passive, I would think any modern listener would have no problem, just based on volume alone. Was it because those people were totally new to even the concept of sound reproduction?

I used to know a guy who did stage hypnotism. He said his act of getting people to do weird stuff was based almost entirely on the fact that nobody wants to be "the guy who spoils the fun". I wonder if Edison's audience and the people in the Pono video have something in common along those lines.
Seems like I recall they had the phonograph and the live sounds in a separate room and the subjects heard it through a doorway. Or maybe that was in a dream. But it's in my head that way. Speaking about potential cognitive distortion.

Edison was a right piece of work.

I wouldn't count on Edison to play fair in any game.

For a while he was demonstrating the 'danger' of alternating current by electrocuting dogs and old horses on stage for audiences. Even an elephant named Topsy.

I'm not a vegan or anything, but something in me goes a little nuts when I consider people killing animals for spectacle. And when they're dogs and horses and elephants, animals that have an aeons old relationship with man (and are extraordinarily intelligent in the case of elephants), I get pretty torqued. I'm gonna go listen to some Renaissance harp music and calm down.
Old 18th April 2014
  #3237
Gear Guru
 

Quote:
Originally Posted by theblue1 View Post
Seems like I recall they had the phonograph and the live sounds in a separate room and the subjects heard it through a doorway. Or maybe that was in a dream. But it's in my head that way. Speaking about potential cognitive distortion.

Edison was a right piece of work.

I wouldn't count on Edison to play fair in any game.
I found THIS article. Apparently the tests were not even blind. You were just supposed to sit there and say "wow, they sound exactly the same!
Quote:

ON A FRIDAY AFTERNOON IN APRIL OF 1916, a crowd of 2,500 music lovers and curiosity seekers gathered at Carnegie Hall in New York. As they took their seats, they saw a fine mahogany phonograph alone on the stage. A white-gloved man emerged from behind the curtains. He placed a record on the turntable, wound up the mechanism, and silently disappeared. The concert hall filled with the strains of the aria “Vissi d’Arte,” from Puccini’s opera Tosca . A second voice, identical to the first, joined in as Madame Marie Rappold of the Metropolitan Opera of New York walked onto the stage, singing along with her recorded voice. Occasionally she would rest and allow the record to carry on alone, and at other times the phonograph extended the same courtesy to her. The event was a “Tone Test,” a recital of music and musical “ReCreations” sponsored by Thomas A. Edison Incorporated. The audience was taking the challenge of distinguishing a live voice from a recording. According to a reporter for the Evening Mail , the evening proved the “indisputable fact” that the Edison phonograph could “perfectly reproduce” live music.

Thousands of Tone Tests, identical in format to that one, were presented to millions of Americans between 1915 and 1925. From Symphony Hall in Boston to the Keyler Grand Theater in Walla Walla, Edison recording artists and Edison Re-Creations (as the Diamond Disc records were known) performed in tandem, convincing—or attempting to convince—audiences that it was “impossible to distinguish the singer’s living voice from its recreation in the instrument.” While it is fascinating to ponder whether audiences really couldn’t detect any difference in sound quality, the true significance of the Tone Tests lies elsewhere. By explicitly asserting that live and recorded music were tonally equivalent, they implicitly equated listening to records with listening to live music. And they thereby transformed music-reproducing machines into musical instruments.

The idea that listening to records is as legitimate a musical activity as listening to live musicians seems so natural today that an era when the “musicality” of records was in question is hard to imagine...
Old 18th April 2014
  #3238
Quote:
Originally Posted by joeq View Post
I found THIS article. Apparently the tests were not even blind. You were just supposed to sit there and say "wow, they sound exactly the same!
That's what happens when salesman pretend to be scientists.

Not hard to see what was behind Edison's marketing onslaught -- the shadow of the electrical recording era were nipping at his toes. 1920 saw the first release of an electrical recording. By 1925 with the merger of Western Electric and AT&T engineering department, and the introduction of their electrical recording systems for disk and optical film, it was probably obvious to the 78 year old Edison that his technology had met its match and then some.

http://en.wikipedia.org/wiki/Sound_r...ical_recording
Old 18th April 2014
  #3239
Gear Guru
 

Quote:
Originally Posted by theblue1 View Post
That's what happens when salesman pretend to be scientists.
well our modern-day salesmen have an easier task. They only have to try to convince people that, for example, Pono is "as good as vinyl"!

Edison had to try to convince people that his phonograph was as good as a live concert.
Old 18th April 2014
  #3240
Gear Head
 
bandpass's Avatar
 

Quote:
Originally Posted by Ephi82 View Post
The math I provided shows that at only 2% adoption, PONO can be very successful from a business perspective and get a solid foothold. 2 people out of one hundred is what it will take.
It seems that Pono is in direct competition with HDtracks & Fiio—what's their percentage of the overall market?

For existing smart-phone users, there's also competition with this device which does a lot more than Pono for similar money (Yes! DSD! ).

I'm not sure the VC people will be impressed so much by the kickstarter—selling limited edition collectibles—as that's not what Pono is actually asking for funding for.
Topic:
Post Reply

Welcome to the Gearslutz Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…
Thread Tools
Search this Thread
Search this Thread:

Advanced Search
Forum Jump
Forum Jump