snr advantage at higher sample rates

[okydoky]

hi,

higher sample rate has an advantage with regard to signal to noise [quantization noise]. here is a quote from bob katz's book [page 63], with some specifics.

V. Dither at High Sample Rate.

"Moving to high sample rates automatically provides a signal-to-noise advantage, so 16 bits at 96kHz is 3.4 dB quieter than at 44.1, sonically equivalent to about 16 1/2 bits. Noise-shaping at high sample rates can allow shorter wordlength files with very low psychoacoustic noise floor - the noise can be made extremely low and flat in the audible band and the shaping moved above 20kHz. In fact, 16-bit noise shaped dither at 96kHz can sound as good as 24-bit/44.1, as I discovered one day when I accidentally left 16-bit dither on while working at 96kHz."

elsewhere it is explained simply that the "less quantization noise with higher sample rates" rule is because "the signal adds linearly, but the noise adds with a square root".

this is interesting to me, and i think it is worth consideration in the sample rate discussions.

this has been public service announcement.



right.

[Cellotron]

I remember reading in one of Dan Lavry's white papers that many converters in fact actually exhibit less usable dynamic range at higher sample rates. Can anyone confirm this?

Best regards,
Steve Berson

[24-96 Mastering]

Originally Posted by oky****

hi,

higher sample rate has an advantage with regard to signal to noise [quantization noise]. here is a quote from bob katz's book [page 63], with some specifics.

V. Dither at High Sample Rate.

"Moving to high sample rates automatically provides a signal-to-noise advantage, so 16 bits at 96kHz is 3.4 dB quieter than at 44.1, sonically equivalent to about 16 1/2 bits. Noise-shaping at high sample rates can allow shorter wordlength files with very low psychoacoustic noise floor - the noise can be made extremely low and flat in the audible band and the shaping moved above 20kHz. In fact, 16-bit noise shaped dither at 96kHz can sound as good as 24-bit/44.1, as I discovered one day when I accidentally left 16-bit dither on while working at 96kHz."

elsewhere it is explained simply that the "less quantization noise with higher sample rates" rule is because "the signal adds linearly, but the noise adds with a square root".

this is interesting to me, and i think it is worth consideration in the sample rate discussions.

this has been public service announcement.



right.

As I understand the text above, the perceived lower noise floor / perceived increase in dynamic range is only attributed to the fact that at high sample rates in a dithered system, significant portions of the required dithering noise can reside in the higher, inaudible frequency range. I.e. the dithering noise level within the audible band drops while linearization is maintained.

Note that the effect only takes place in a dithered system where there is no quantization "noise" (quantization distortion). I think the intro and outro to the quote, where you mention quantization "noise" are leading you on a wrong track. It seems to me you're mixing up observations about dithered systems and undithered systems.

[okydoky]

Originally Posted by 24-96 Mastering

As I understand the text above, the perceived lower noise floor / increase in perceived dynamic range is only attributed to the fact that at high sample rates, significant portions of the required dithering noise levels can reside in the higher, inaudible frequency range. I.e. the dithering noise level in the audible band drops while full linearization is maintained.

Note that the effect only takes place in a dithered system where there is no quantization noise (i.e. quantization distortion). I.e. I think the intro and outro to the quote (where you mention quantization distortion) are leading you on a wrong track.

hi,

there is a simple, unqualified improvement due to the fact that signal adds linearly and quantization noise adds with a square root. in a 4x system you have 4 times the amount of samples in a given time period as compared to a 1x system.

the effect is not limited to a dithered system. however, it seems that a dithered system also benefits.

it seems to me that the average of the 4 quantization levels in the 4x system is more accurate than the corresponding 1 quantization level in the 1x system. this is of course true without dither.


right.

[wado1942]

The text, in my understanding, makes no reference to quantization noise (i.e. quantization distortion) whatsoever.

There generally isn't any dither in a 24-bit signal. The principle works simply like this... The converters will always exibit a certain amount of noise/distortion. By doubling the sample rate, the distortion is spreak over double the bandwidth, which is partially above the human hearing range. So it simply SOUNDS like there's less distortion. When dither comes into the mix, you can actually shape the noise so that there's virtually nothing in the audible band so this rule doesn't exactly apply to dither.

I remember reading in one of Dan Lavry's white papers that many converters in fact actually exhibit less usable dynamic range at higher sample rates. Can anyone confirm this?

I can. I've tested several ADCs at different rates for my book. It all depends on the clock they use. A good ADC uses a fixed crystal clock to drive the sampling so it always runs at the same rate. To get different output sample rates, it has an internal SRC to change the sample rate to whatever you want. These kinds of converters exibit best performance at their highest output sample rates. A lot of ADCs use voltage controlled oscillators similar to the ones used in analogue synthesizers to create the clocks. Those converters use fixed DSP for all filtering etc and the actual clock itself if varied to get different sample rates. The oscillators have a certain amount of instability and the harder you push them, the more obvious it becomes. If you double the sample rate, the jitter, which in actuallity is constant, has double the influence on the signal. I can't get into the physics of it very well because I'm just learing about it myself. But one of the converters I tested yelded a 91dB S/N ratio at 24/96 but 94dB at 24/48. At 22.05K, you guessed it, 98dB. Another ADC I tested yielded 120dB S/N ratio regardless of the sample rate but SOUNDED cleaner at 96KHz, suggesting it used a fixed clock.

[24-96 Mastering]

Originally Posted by wado1942

The principle works simply like this... The converters will always exibit a certain amount of noise/distortion. By doubling the sample rate, the distortion is spreak over double the bandwidth, which is partially above the human hearing range. So it simply SOUNDS like there's less distortion. When dither comes into the mix, you can actually shape the noise so that there's virtually nothing in the audible band so this rule doesn't exactly apply to dither.

Right. That's what I'm saying. All this does not apply to a dithered system; which is what Bob K's quote actually refers to.


I am interested though how even in an undithered system, correlated distortion could possibly be perceived to be lower by being "spread out". How would that happen? Wouldn't the added frequency extension just allow for additional harmonics to be recorded? How would that decrease the perceived level of the lower harmonics? Am I missing something here?

[okydoky]

hi,

it seems that sampling faster reduces quantization noise, period. it does not merely "push it out of band". that is noise shaping [concerning dither].

the ratio of amount of signal to quantization noise is going to be better where you have more samples taken in a given time period [unless of course you have a system that has more noise than signal on a sample by sample basis, but that is not the system we are discussing].

the only reason you have noise "out of band" in a 2x or 4x system is because you have more bandwidth, and there noise in that bandwidth just as there is below.


right.

[24-96 Mastering]

Originally Posted by oky****

the ratio of amount of signal to quantization noise is going to be better where you have more samples taken in a given time period

Are we both talking about (undithered) PCM? Are we both assuming that both systems to be compared have sampling rates high enough to record the complete audible range?

[okydoky]

Originally Posted by 24-96 Mastering

Are we both talking about (undithered) PCM?

hi,

i don't know. are you?



right.

[Dan Lavry]

Originally Posted by oky****

hi,

elsewhere it is explained simply that the "less quantization noise with higher sample rates" rule is because "the signal adds linearly, but the noise adds with a square root".

this is interesting to me, and i think it is worth consideration in the sample rate discussions.

this has been public service announcement.

right.

The comment is for ADDING signals. But here we are not adding signals.

A converter with twice the bandwidth does not yield twice the signal amplitude. The signal amplitude is determined by factors other then bandwidth (such as supply rails, analog circuits and so on).

However, the noise does increase when you open up the bandwidth. True, it increases to the tune of the square root of the bandwidth, but it does increase.
If you double the bandwidth, you end up with 3dB more noise and 0dB more signal range thus the SNR drops by 3dB.

Dan Lavry

[It'sJoeAgain]

my understandin is that all hi quality converters have upsampling filters so they operate at high frequency rates (96kHz or higher) no matter what the incoming rate is...but that's bandwidth not dynamic range...

[It'sJoeAgain]

Originally Posted by Dan Lavry

However, the noise does increase when you open up the bandwidth. True, it increases to the tune of the square root of the bandwidth, but it does increase.
If you double the bandwidth, you end up with 3dB more noise and 0dB more signal range thus the SNR drops by 3dB.

do u reckon that in a world of 24 bit and up mastering, and even at 44.1kHz sampling rates, these specs are of great *practical* importance.??...

[Dan Lavry]

Originally Posted by 24-96 Mastering

Are we both talking about (undithered) PCM? Are we both assuming that both systems to be compared have sampling rates high enough to record the complete audible range?

OK, let’s leave dither and noise shaping out.

When talking about noise, one should not lump all the noise sources into one. Instead, one should examine the various causes of noise individually. This goes beyond a single post, but a few comments are in order:

Much of the limitations in dynamic range of both “analog only” and converter devices is due to noise generated by components (resistors, semiconductors and even small value caps). Take resistor noise, and you find that it increases with bandwidth (though not linearly, but as a square root of the bandwidth). If one provides more bandwidth, you end up with MORE NOISE. You do not deal with a fixed amount of noise spread over more bandwidth. This notion is plain wrong. The statement that noise goes up by a square of the bandwidth is correct, so double the bandwidth means 3dB more noise (not 6dB), but that is still more noise, not less noise…

One can look up and note that the lowest bandwidth converters (such as for weighing scales) offer the highest signal to noise, and as one increases bandwidth, the SNR and the number of bits drop. At 100MHz, you do not have 16 bits, and at 4GHz, a single bit (a comparator) is not an easy task… One can also look at OPamps and see a similar “curve”.


Another aspect (other then analog noise):
When it comes to converter design, one has to look at a more specific picture – what type of converter. Given that so much of audio conversion is based on sigma delta, there is a basic fundamental reason why more bandwidth means more noise. The factors impacting a “basic block diagram paper design” of a sigma delta are: 1. Number of modulator bits 2. Modulator filter order and 3. modulator oversampling rate. For example, one can have a 5 bit modulator with a 5th order feedback filter operating at 256fs.

Holding all things equal, the “amount of noise shaping” is fixed. The designer gets to remove noise from a selected frequency range (such as the audio range), and move that noise to a frequency range that is not in use. It is analogous to “digging a whole in the ground”. You start with 1000 cubic feet capability, and you may end up with 1000 feet deep hole that is only 1 square foot, or a 1 foot deep hole that has an area of 1000 sq feet. You may also end up with 10 feet deep and 100 square foot area…
The “area of the hole” is analogous to frequency bandwidth. The deeper the hole, the lower the noise floor. So given some fixed resources, you have to trade off bandwidth against noise! That is the ABC of sigma delta.

Note that the noise limitation and the tradeoff between noise and bandwidth take place at the modulator of the converter. The decimator block converts the high rate low bit modulator data (such as 5 bits at 256fs or similar) to a final format, but the trade off is already etched in stone. If a modulator is designed to accommodate near 96KHz audio (192KHz sampling rate), it offers the capability to digitize signals near 96KHz, but at cost to dynamic range. Making such a converter accommodate 96KHz sampling (near 48KHz audio) calls for one more stage of decimation (factor of 2) but the noise limitation is already "built in" into the previous modulator stage.

A converter designed for say 48KHz audio (96KHz sampling rate) offers better SNR then a 192KHz converter used at 96KHz rate. The 192KHz converter offers an extended audio range (48-96KHz) that you do not need, at the expanse of lower SNR at 0-48KHz range (some of that range you do need).

Regards
Dan Lavry

[It'sJoeAgain]

thanks Dan, i always felt there was a lot of misinformation floating round about this subject, but this single post of urs is one of the best i've read here at GS about converters...

I still am of the opinion that 44.1khz/24 bit audio with the right converters is all the sound u'll ever need......

[okydoky]

Originally Posted by Dan Lavry

OK, let’s leave dither and noise shaping out.

When talking about noise, one should not lump all the noise sources into one. Instead, one should examine the various causes of noise individually. This goes beyond a single post, but a few comments are in order:

Much of the limitations in dynamic range of both “analog only” and converter devices is due to noise generated by components (resistors, semiconductors and even small value caps). Take resistor noise, and you find that it increases with bandwidth (though not linearly, but as a square root of the bandwidth). If one provides more bandwidth, you end up with MORE NOISE. You do not deal with a fixed amount of noise spread over more bandwidth. This notion is plain wrong. The statement that noise goes up by a square of the bandwidth is correct, so double the bandwidth means 3dB more noise (not 6dB), but that is still more noise, not less noise…

One can look up and note that the lowest bandwidth converters (such as for weighing scales) offer the highest signal to noise, and as one increases bandwidth, the SNR and the number of bits drop. At 100MHz, you do not have 16 bits, and at 4GHz, a single bit (a comparator) is not an easy task… One can also look at OPamps and see a similar “curve”.


Another aspect (other then analog noise):
When it comes to converter design, one has to look at a more specific picture – what type of converter. Given that so much of audio conversion is based on sigma delta, there is a basic fundamental reason why more bandwidth means more noise. The factors impacting a “basic block diagram paper design” of a sigma delta are: 1. Number of modulator bits 2. Modulator filter order and 3. modulator oversampling rate. For example, one can have a 5 bit modulator with a 5th order feedback filter operating at 256fs.

Holding all things equal, the “amount of noise shaping” is fixed. The designer gets to remove noise from a selected frequency range (such as the audio range), and move that noise to a frequency range that is not in use. It is analogous to “digging a whole in the ground”. You start with 1000 cubic feet capability, and you may end up with 1000 feet deep hole that is only 1 square foot, or a 1 foot deep hole that has an area of 1000 sq feet. You may also end up with 10 feet deep and 100 square foot area…
The “area of the hole” is analogous to frequency bandwidth. The deeper the hole, the lower the noise floor. So given some fixed resources, you have to trade off bandwidth against noise! That is the ABC of sigma delta.

Note that the noise limitation and the tradeoff between noise and bandwidth take place at the modulator of the converter. The decimator block converts the high rate low bit modulator data (such as 5 bits at 256fs or similar) to a final format, but the trade off is already etched in stone. If a modulator is designed to accommodate near 96KHz audio (192KHz sampling rate), it offers the capability to digitize signals near 96KHz, but at cost to dynamic range. Making such a converter accommodate 96KHz sampling (near 48KHz audio) calls for one more stage of decimation (factor of 2) but the noise limitation is already "built in" into the previous modulator stage.

A converter designed for say 48KHz audio (96KHz sampling rate) offers better SNR then a 192KHz converter used at 96KHz rate. The 192KHz converter offers an extended audio range (48-96KHz) that you do not need, at the expanse of lower SNR at 0-48KHz range (some of that range you do need).

Regards
Dan Lavry

hi,

you are talking about a different thing [and i do not even believe it would be a salient issue in truth].

one does not increase quantization error by sampling faster. why lead people to that erroneous conclusion?

i'll post a more complete quote and reference below, because, respectfully, i think you may have misunderstood what was said.

"A converter where the input signal is sampled much faster than the Nyquist rate is called an oversampling converter. The signal bandwidth of the input signal is denoted by f(sub)b and the Nyquist rate, which is the minimum sampling frequency to avoid aliasing, equals f(sub)N = 2f(sub)b.
........
Compared to a 1x Nyquist-rate converter, the noise in the output signal is reduced by OSR [oversampling ratio].

The formulas to calculate the signal power are identical as for a Nyquist-rate converter. From the ratio of figures 2.6 and 2.11, the peak signal-to-noise ratio of an oversampled converter results as: SNR(sub)p = 3/2 times 2(sup)2B times OSR.

Expressing the equation in dB yields SNR(sub)p = 1.76 + 6.02 B + 10log(OSR) dB.

This clearly illustrates the advantage of oversampled converters. The SNR(sub)p improves by 3dB/octave, or equivalent, by 0.5 bit /octave of oversampling ratio. In other words, the accuracy of an AD converter can be improved simply by oversampling the input signal. This comes of course at the cost of increased sampling rates and the need for faster circuits. This shows that oversampled converters allow to trade speed for accuracy.

This improvement over Nyquist-rate converters can also be explained intuitively. By oversampling, more samples of the input signal are taken. The signal components add linearly, but the quantization noise components add with a square root. So, when the samples are averaged together, the quantization error is reduced.


It is possible to exploit the benefits of oversampling even further by employing noiseshaping. This results in a delta-sigma converter, as explained in the next section.

[Design of Multi-Bit Delta-Sigma A/D converters: by Yyes Geerts, Michiel Steyaert, Willy M. C. Sansen]"



its my understanding that the statement in mr. katz's book is correct.

and that is not even taking into consideration other advantages of higher sampler rates, such as better impulse response [amplitude], better results in processing the files, and, importantly, reduced latency [since we are making music with these things, not running sine wave tests, and timing / rhythm is an essential aspect of music].

frankly, reduced latency alone trumps any miniscule design issues you say you encounter in your designs at higher sample rates.

i also see no sigma delta modulators that would exclude a 192kHz sample rate. therefore, even under the theory that there is some sort of "trade-off" "etched in stone at the modulator", the 192kHz system rate would fare no worse in that regard.

always an interesting discussion.


right.

[Dan Lavry]

Originally Posted by oky****

hi,

you are talking about a different thing [and i do not even believe it would be a salient issue in truth].

one does not increase quantization error by sampling faster. why lead people to that erroneous conclusion?

i'll post a more complete quote and reference below, because, respectfully, i think you may have misunderstood what was said.

"A converter where the input signal is sampled much faster than the Nyquist rate is called an oversampling converter. The signal bandwidth of the input signal is denoted by f(sub)b and the Nyquist rate, which is the minimum sampling frequency to avoid aliasing, equals f(sub)N = 2f(sub)b.
........
Compared to a 1x Nyquist-rate converter, the noise in the output signal is reduced by OSR [oversampling ratio].

The formulas to calculate the signal power are identical as for a Nyquist-rate converter. From the ratio of figures 2.6 and 2.11, the peak signal-to-noise ratio of an oversampled converter results as: SNR(sub)p = 3/2 times 2(sup)2B times OSR.

Expressing the equation in dB yields SNR(sub)p = 1.76 + 6.02 B + 10log(OSR) dB.

This clearly illustrates the advantage of oversampled converters. The SNR(sub)p improves by 3dB/octave, or equivalent, by 0.5 bit /octave of oversampling ratio. In other words, the accuracy of an AD converter can be improved simply by oversampling the input signal. This comes of course at the cost of increased sampling rates and the need for faster circuits. This shows that oversampled converters allow to trade speed for accuracy.

This improvement over Nyquist-rate converters can also be explained intuitively. By oversampling, more samples of the input signal are taken. The signal components add linearly, but the quantization noise components add with a square root. So, when the samples are averaged together, the quantization error is reduced.


It is possible to exploit the benefits of oversampling even further by employing noiseshaping. This results in a delta-sigma converter, as explained in the next section.

[Design of Multi-Bit Delta-Sigma A/D converters: by Yyes Geerts, Michiel Steyaert, Willy M. C. Sansen]"



its my understanding that the statement in mr. katz's book is correct.

right.

You are mixing things up. The oversampling process is not the same as higher sampling frequency converter. A X256 oversampling converter is not an 11.2896MHz (44.1KHZ X 256) converter. Take an 44.1KHz oversampling DA, it does not contain signal above 22.05KHz. You upsample the data by say 256, and the signal itself is still only 22.05KHz. This is not a high sample rate conversion, it is oversampled conversion. The sample rate (thus the bandwidth) is limited and will stay limited. Up sampling and oversampling is great, but it is not high sampling rate.

The formula SNR(sub)p = 1.76 + 6.02 B + 10log(OSR) dB is correct when used in proper context, but you are starting out at the wrong end. Most sigma delta today operate at 128-1024fs oversampling and up sampling rates. That does not mean that they are 5.6-45MHz converters. In an AD, you start up with a few bits (typically 1-5 bits) at those very high rate, and given the few modulator bits, the noise is very high. Then you start trading off the bandwidth (sample speed) for better noise floor, hopefully with good noise shaping. In a DA, you start up with a lot of bits at lower rate, and you trade it for few bits at very high rate.

Oversampling and up sampling is not sampling, and understanding the distinction will clear up the misconception that higher sampling rate conversion yields better SNR. (Look up my paper "On sampling"for better explanation).

And for those that can not follow it all, look up specifications of converter products and you will find that the higher sample rate hardware offers less SNR.

Dan Lavry

[wado1942]

Ok, perhaps my choice of words is poor. You're correct in saying there's theoretically greater analogue noise at higher sample rates simply because there's wider bandwidth to contain more energy. Though you'd never hear that extra noise since it's beyond your hearing range. I'm saying, QUANTIZATION noise/distortion stays constant and by sampling at a higher rate, those artifacts are spread over a wider bandwidth. Thus they SOUND like they are lower because you do not hear the distortion spread into the 20KHz + range. I'm also assuming you're using stable cystal clocks instead of VCOs. I've verified personally that running VCO clocked converters perform worse at higher sample rates. However, a 96K converter with a fixed clock performs better at 96KHz than at 48KHz. Not only is the distortion lower but most of the "crap" from the LPF is out of hearing range.

my understandin is that all hi quality converters have upsampling filters so they operate at high frequency rates (96kHz or higher) no matter what the incoming rate is...but that's bandwidth not dynamic range...

They don't upsample, the data comming off of the quantizer is of very high rate. The filtering takes place before downsampling to your destination. Unless of course, you mean the DACs, in which case you're correct. The stream is upsampled before filtering.

That makes me wonder, is all this confusion over some of us talking about some DACs and others talking aboud ADCs?

[rogerbrain]

OK, I will bite.

I probably fall into the 'prosumer' range. and have followed this thread with much interest.

I believe I have been mixing/confusing dynamic range and SNR in my understanding, am I hearing that they should remain separate issues?

I have never been concerned with SNR in the digital realm since I came from the T A P E days of 60db (or so) down was the noise floor, AND very seldom do I record a single source so softly that 'media' noise (probably not the correct way to express) has ever been at issue with digital. I record enough noise in terms of circuit hiss or ambient noise to mask any 'noise' that is produced by my DAW...(or so I thought). soft signals can suffer from sounding whimpy as the amount of data can be low (again ..or so i thought)

I have also never been overly concerned with dynamic range as TAPE seemed to do a fine job with this aspect with less than I have available in my DAW.

I have always 'believed' that resolution, linearity and frequency response were the issues with digital.. as softer signals are 'recorded' there was not enough 'bit depth' to get a very good sound.. so higher resolutions (which include both bit rate and sample rate) help take care of this.

I have a system that will support 24/192 that claims 120db dymanic range (I will have to back and look to see if any SNR specs are even listed) and I record at that resolution.

I understood that there is a filter that can cause artifacts (not meaning noise per say) that can degrade the 'sound' that is placed at about half of the freq of the sample rate -or- about 80k in the case or 192, about 22k @ 44.1, aout 40k at 96.. I alway thought that having the increased frequency response was a good thing, moving any artifcats up out of hearing range. ALSO that the faster sampling speed made the data more dense much like faster IPS on a tape deck.

I understand that there is not substantially more data at 24/192 than at 24/96 or 26/48 etc but I have always thought the increased frequency response was desirable so I always use the highest sample rate I can.

if you guys could say if I am correct about the above and help me connect/apply what issues actual SNR or quantization noise are causing me that I may have been attribuiting to something else. OR- how do these artifacts show up and in what quantity.. Hopefully this is a real Question. thanks

[Dan Lavry]

Let me try for a better explanation of my last post , regarding the difference between sampling and up sampling:

Say you have a 48KHz DA, which in theory can yield up to 24KHz signal bandwidth. Say the DA is 16 bits, 10V peak to peak device. That means 153uV per bit. The quantization “grid” is made of horizontal lines that are 153uV apart, and all the samples fed in fall on one of the available grid lines.

Say we want to up sample by X2. That means, we are looking for data at times “between the original samples”. Such data may not fall on one of the available quantization “grid lines”. When one “connects the dots” between the original samples, the line does not have to fall on a grid line.

The up sampling block is a computational engine, and let us assume it is a perfect up sampler. So now we have twice as many DIGITAL samples values, where every other sample is off the quantization grid.
In fact if we up sample by say X128, we have a situation where 127 out of 128 samples can be off the quantization grid. Indeed, in theory we have a lower noise. The original signal “jumped” from sample to sample by multiples of 152uV. The sample errors are up 152uV in magnitude. But the up sampled signal is computed in the digital world to much less error, because the computed up sampled values fall between the grid lines.

That is all great, but we still need to convert the digitally computed very precise samples from digital to analog. Our 16 bit DIGITAL data may be 20 bits in terms of noise, or even 24 bits, but all that is useless if the circuitry that converts digital to analog is still only 16 bits, thus the conversion is restricted to 152uV steps.

In order to take advantage of the digital compute block ability to place sample values BETWEEN the original grid lines to say 20 bit grid, we ALSO NEED a DA that can convert the computed digital to an analog signal that is 20 bits accurate (thus 9.5uV steps).

Does the up sampled data offer more bits? The up- sampled data DEMANDS a better DA converter, one capable of 9.5uV steps instead of 152uV in our example.

Note that getting the DA circuit to provide more accuracy (smaller steps) get MORE DIFFICULT when increasing the up sampling ratio because more samples demand higher bandwidth, and more bandwidth brings in more noise. Not to mention that up sampling tends to do nothing for improving distortions.
The old non up sampling DA’s offered near 96dB noise. What does a X1024fs DA’s do not offer? The square root of 1024 is 32 so why do we not end up with 96 + 32 *3dB = 192dB SNR? The reason is, we can not make a circuit that will convert 32 bits digital data to analog signal ( 192dB/6dB = 32 bits). So where is the noise limitation? It is in the what I mentioned in the previous post – resistor noise, semiconductor noise and so on.

So why do we up sample so high? We do so for various reasons such as to make the design of good analog filters possible, to accommodate effective noise shaping and more.

It is important to note that up sampled data is not to be confused with high sampling rate. In the case of up sampling, one gets to use a lot of samples but they “carry” bandwidth information. There is no need or point in storing up-sampled data. We get to generate that data at the converter, because all the signal information is already contained in the data BEFORE the up sampling takes place.

When up sampling DA’s came out, the sales guys knew that the ordinary consumer will not appreciate terminology such as “better analog filter”, so they talked about 16 times oversampling 20 bit converter. The computational block offered 20 bit, but they did not have a 20 bit DA circuit. They needed a 20 bit machine to take advantage of up sampling. But calling a device 20 bit up sampling DA sounded like more bits, so folks got confused. And apparently some are still confused.

Regards
Dan Lavry

[24-96 Mastering]

Originally Posted by Dan Lavry

Much of the limitations in dynamic range of both “analog only” and converter devices is due to noise generated by components (resistors, semiconductors and even small value caps). Take resistor noise, and you find that it increases with bandwidth (though not linearly, but as a square root of the bandwidth). If one provides more bandwidth, you end up with MORE NOISE. You do not deal with a fixed amount of noise spread over more bandwidth. This notion is plain wrong. The statement that noise goes up by a square of the bandwidth is correct, so double the bandwidth means 3dB more noise (not 6dB), but that is still more noise, not less noise…

Thanks for the reply Dan, that's very interesting. Makes sense that if one assumes there to be self noise in a system, sampling bandwidth has an influence on the amount of noise recorded. I have no doubt that such real life limitations are very much of importance.

However, I'm also interested in the theoretical, generic claim made in the original post: That in any (non-dithered) PCM recording system, sampling rate supposedly has an influence on SNR. First, I don't understand how that is possible because a generic undithered PCM system has no noise floor. All perceived "noise" will be quantization noise, i.e. harmonic distortion, so we're not talking about SNR but amount of distortion. If that's what the original post is referring to, then I'm asking how the relative amount of (perceived) harmonic distortion can possibly change with sampling rate (assuming both system's sampling rates are sufficient to cover the audible range).

If the original post is referring to a dithered system, then there can of course be a connection between sampling rate and perceived SNR because of the dither spreading over an increased bandwidth (or intentionally being shaped) into the inaudible range. But the OP is specifically saying that this is not what he's talking about. So I'm wondering what exciting new development in PCM I missed in the last decade.

[Jon Hodgson]

Hi Dan,

Thanks for your input on this thread, well thought out and informative as ever.

As for Oky****, at least he seems to accept you're the real Dan Lavry, he's directly accused me of being an imposter in another thread, (though typically has yet to respond when questioned directly as to who or what he thought i was pretending to be that I was not).

Please don't let him ruin things for everyone else here, I'm sure I'm not alone in appreciating you sharing your knowledge and experience.

[It'sJoeAgain]

Originally Posted by Dan Lavry

I stayed out of GS for a long time, and folks, you can see why. If you all (and the moderator) want to read my technical input, you have to figure out how to put a stop to such comments.

I think ur gonna fair a lot better than a ME that used to give his views on things such as analysers, narrow band compression, etc and no longer post round here...sorry Dan...u really are knowledgeable and it would be a bummer to have u also gone from the forum....

[wado1942]

Originally Posted by rogerbrain

I believe I have been mixing/confusing dynamic range and SNR in my understanding, am I hearing that they should remain separate issues?

They always have been and should still be considered separate. For instance, my 1/4" deck has about a 60dB nominal S/N ratio without noise reduction. That's not debatable because it's a tested hard fact. Now, dynamic range is somewhat subjective because you can hear detail over 20dB below the noise floor. It's a matter of simply when the noise is bad enough to mask the signal which is dependant on a lot of things. For instance, most of the noise on a tape-based analogue medium is in the extreme low frequencies which we don't hear very well anyway. So once you take that into account, it makes sense that the medium sounds less noisy than it really is. So there's less masking in the critical range than one might expect. So do you consider dynamic range the entire range of the absolute maximum peak to the noise floor like many do? Or you might consider the dynamic range by the difference between a hot, undistorted signal and low enough that the noise becomes intrusive in the program material. Or you could consider it the range between the highest percieved level and the lowest level where you can no longer pick out detail?

With an undithered 12-bit audio source, there's no doubt it's less noisy than 1/4" tape but all the low-level detail is missing so it has a smaller usable dynamic range. This is one that even engineers have a hard time defining. On paper, an engineer might say a 12-bit audio tape has a better dynamic range than 1/4" tape but I'll take the 1/4" any day. There's a lot of reasons for that, but it just goes to show that dynamic range is highly subjective and not necessarily dependant on S/N ratio.

I have never been concerned with SNR in the digital realm since I came from the T A P E days of 60db (or so) down was the noise floor,

I'm still not concerned with S/N ratio even on tape. I can count the number of times I used NR on my 1/2" 8-track with one hand and I've never used it on my 1/4" deck. I remember somebody telling me that 1/4" isn't good enough by todays standards because of noise and distortion. I did a test master, passing the music through my deck and taking the feed to the ADCs right off the monitor head. He had no clue what I did till I told him.

I have a system that will support 24/192 that claims 120db dymanic range (I will have to back and look to see if any SNR specs are even listed) and I record at that resolution.

Double check your specs and see the sample rate at which that test was made. If it has a VCO clock I'll bet the tests were done at 44.1KHz and the actual 192K performance is less.

I alway thought that having the increased frequency response was a good thing, moving any artifcats up out of hearing range. ALSO that the faster sampling speed made the data more dense much like faster IPS on a tape deck.

That's a huge debate inof itself. I agree with you on that. But there's guys that say "if your filters are perfect, then you don't need those higher sample rates"....yeah, well, there ARE no perfect filters so higher rates seem to work better, at least to my ears. I really wish people would just accept the fact that designing these utopian filters is not realistic and that we should just adopt 96KHz for all standard media. Even if it's supposedly noisier, I'd take the noise and push the ringing harshness out of the way. Since we're on the tape trend here, you know the tape has about the same noise floor at 15I/S as it does 30I/S. The noise is just smoother sounding at 30I/S so it's less intrusive. Sometimes hard facts are less important than perception.


All that aside, I read some studies last year that talked about a scientific study with audio recordings. They recorded an instrument to DSD, full bandwidth, directly off of the mic preamp. They put a 4th order crossover at 20KHz on the playback system and gave 100 subjects PET scans while listenig to the playback. So no matter how they played back the recording, there would always be the same filters at work in the audio so it was simply a matter of bandwidth. At first, they'd take a baseline reading with nothing being played. Then they'd randomly playback the full bandwidth audio, the same recording with the HF channel disabled and the same recording with the LF channel disabled. The results were frighteningly similar. The HF only playback was always the same as no recording at all. Now here's where it gets fun. The hearing center of the subjects' brains was the same whether using the LF only or full bandwidth audio. So that shows the subjects did not hear any notable difference between the recordings. Now, when listening to the full bandwidth playback, ALL SUBJECTS showed greater activity in the pleasure centers of the brain. So even though they can't consciously hear the difference, they can feel it. Now that's a 4th order filter which is way less steep than anything used in conventional digital audio and even that showed some pretty suprising differences in the subjects. I saw the normalized PET scans and even an amateur like myself could see the difference. I'll try and see if the papers are online.

Engineers should probably work hand in hand with some psychologists before comming out with the next revolution in digital audio.

[Alexey Lukin]

Originally Posted by 24-96 Mastering

However, I'm also interested in the theoretical, generic claim made in the original post: That in any (non-dithered) PCM recording system, sampling rate supposedly has an influence on SNR. First, I don't understand how that is possible because a generic undithered PCM system has no noise floor.

Robin, I believe that the original quote by Bob Katz is made about a dithered PCM system, however it still holds true for a non-dithered PCM system. When quantizing a signal at a higher sampling rate, the quantization distortion (or dithering noise) has the same power (defined by the amplitude of the least-significant bit), but their spectrum is spread over a wider frequency range. If we are only interested in noise (or distortion) levels in the limited audible range, their levels (and power spectrum density) drop when higher sampling rate is used.

I believe that both Dan Lavry and oky**** are correct and there's nothing to argue about.
Dan's argument holds true when the dominant noise in the system is an analog noise (e.g. when recording at 24 bits). In this case, doubling the sampling rate captures a wider range of analog noise. Since analog noise is typically white, this doubles the power (+3 dB) of the recorded noise. However the noise level in the audible band remains the same.
Oky****'s (and Bob's) argument holds true when the dominant noise in the system is quantization or dithering noise (e.g. when recording at 16 bits). In this case, analog noise has relatively little impact on the recording, and quantization (or dithering) noise is spread over a wider frequency range, which results in decreased noise levels in the audible band.

Another related observation is that with many noise-shaped sigma-delta A/D converters full-band noise levels dramatically rise with higher sampling rates because noise shaping curves are only crafted to remove noise below 20-40 kHz, while high sampling rates reveal much of the ultrasound noise that is "shaped out" of the audible band.

[Jon Hodgson]

Originally Posted by Alexey Lukin

Robin, I believe that the original quote by Bob Katz is made about a dithered PCM system, however it still holds true for a non-dithered PCM system.

Actually it doesn't hold true for non-dithered systems, certainly not with the ideal 4 x oversampling gives 6dB better resolution relationship, the actual improvement will be signal dependent. I could quote the relevant bit from John Watkinson's "The Art of Digital Audio", but I think a simple example will work better.
If you have a DC level a third of the way between two quantization points, if you have no dither then it doesn't matter how many times you sample it, it will always quantize to the same level, oversample as much as you like and you won't improve the accuracy.
However if you dither it, then on average one third of the time it will quantize one way, and two thirds of the time it will quantize the other, the more points you average together, the less each average point will deviate from the correct value.

I believe that both Dan Lavry and oky**** are correct and there's nothing to argue about.
Dan's argument holds true when the dominant noise in the system is an analog noise (e.g. when recording at 24 bits). In this case, doubling the sampling rate captures a wider range of analog noise. Since analog noise is typically white, this doubles the power (+3 dB) of the recorded noise. However the noise level in the audible band remains the same.
Oky****'s (and Bob's) argument holds true when the dominant noise in the system is quantization or dithering noise (e.g. when recording at 16 bits). In this case, analog noise has relatively little impact on the recording, and quantization (or dithering) noise is spread over a wider frequency range, which results in decreased noise levels in the audible band.

I agree, I would also add that the important thing is to understand the implications of these factors. Since with 24 bit converters the real world issues of analogue noise outweigh the theoretical advantages of oversampling it's not productive to take a blanket "faster is better" attitude, select your input sample rate for bandwidth.

Another related observation is that with many noise-shaped sigma-delta A/D converters full-band noise levels dramatically rise with higher sampling rates because noise shaping curves are only crafted to remove noise below 20-40 kHz, while high sampling rates reveal much of the ultrasound noise that is "shaped out" of the audible band.

Sigma delta converters rely on that noise shaping in order to work, even with oversampling their signal to noise ratio would be too high otherwise.

[Alexey Lukin]

Originally Posted by Jon Hodgson

Actually it doesn't hold true for non-dithered systems, certainly not with the ideal 4 x oversampling gives 6dB better resolution relationship, the actual improvement will be signal dependent.

Agreed, the improvement for non-dithered quantization will be signal-dependent.

[24-96 Mastering]

Originally Posted by Alexey Lukin

Robin, I believe that the original quote by Bob Katz is made about a dithered PCM system, however it still holds true for a non-dithered PCM system. When quantizing a signal at a higher sampling rate, the quantization distortion (or dithering noise) has the same power (defined by the amplitude of the least-significant bit), but their spectrum is spread over a wider frequency range. If we are only interested in noise (or distortion) levels in the limited audible range, their levels (and power spectrum density) drop when higher sampling rate is used.

Hi Alexey,

thanks for trying to help me out here. I still don't understand. Wouldn't the harmonics build up evenly across the frequency range, generating the same energy in a given frequency range (such as the audible spectrum), regardless of sample rate? Or am I confusing cause and effect?


Furthermore, I tried to test this using the following set up:

- start with a (complex audio) 24/96 recording
- applied a 4-pole low-pass at 1kHz and dropped audio level by 60 dB (to make subsequent harmonic distortion more obvious)
- made two versions, one SRed to 192 kHz, one SRed to 44.1 kHz.
- both versions were then saved (truncated) to 16 bits.
- applied 60 dB of gain to both versions (to make results audible)
- applied a 4-pole high-pass filter at 5kHz, so that pretty much only distortion products are left.
- measured levels

The resulting quantization distortion measures (near as makes no difference) the same in both files. Actually, the 44.1 version even measures a teeny bit lower. If I read your post correctly, the higher sample rate file's quantization distortion should measure considerably lower. Is there a flaw in my test setup?

EDIT: I just realized the flaw in my setup... I didn't downsample the 192kHz file after truncation. Doh. After doing so, there is now indeed a difference in total distortion level. Albeit significantly less than expected, nowhere near 6 dB. But it comfirms there is or can be AT LEAST SOME difference depending on SR.

[Dan Lavry]

Originally Posted by Alexey Lukin

Agreed, the improvement for non-dithered quantization will be signal-dependent.

The thread title is about higher sample rate and SNR. I do not see how dither got in here.

SNR is limited by analog noise. You choose a bandwidth and sample rate and you get the SNR that is limited by analog noise. One can later reduce the SNR by word length truncation, or by adding dither, with or without noise shaping. But what does it have to do with sample rate?

If one has say 96KHz system, and you want to keep it at 96KHz, the 96KHz formats offers 24 bits so why bother to dither or noise shape or truncate?

Also, take an FFT of a noise floor of an AD. What you see is noise over the frequency range. Say the noise level between 1-3KHz is 120dB (for example). That is as low as it will ever get. In that example when one noise shapes, one can not have more then 120dB between 1-3Khz, no matter what you do. Noise shaping can not yield better then what you started with. It is about trying to "keep as much as possible" when going to less bits.

So the noise limitations are due to analog components and circuit noise. I see one of my posts was deleted, so let me restate - the noise limitations are due to component an circuit noise, and that is what determine the quantization noise (how many real bits) a converter has. Faster sampling means more noise and lower SNR.

If dither and noise shaping is the subject of interest, then I recommend reading my paper "Do you need 20 bits" on my company site.

Regards
Dan Lavry
Lavry Engineering

[24-96 Mastering]

Originally Posted by Dan Lavry

The thread title is about higher sample rate and SNR. I do not see how dither got in here.

Whether the initial post is about SNR as defined by analog noise floor (as would be practical in the real world), SNR as defined by dithering noise floor (as Bob K's quote in that post seems to suggest) or quantization distortion "noise floor" as the first sentence in the post seems to suggest... who knows.

Confusion over that those different aspects of the theme is pretty much what this thread has been about so far... Now that they are untangled, we can actually get on with it and share / learn. Actually, seeing them get untangled has already been quite insightful to me. Thanks!

[Alexey Lukin]

Originally Posted by 24-96 Mastering

Wouldn't the harmonics build up evenly across the frequency range, generating the same energy in a given frequency range (such as the audible spectrum), regardless of sample rate?

No. The overall power of harmonics is limited by the power of quantization distortion (which is independent of the sampling rate). But since they can spread to a wider frequency range, the audible power is reduced.

P.S. Dan, I hear you. I think that the original post mentioned a different type of noise: not the one from analog components, but the one theoretically achievable in a given digital audio format.