How do DAWs and plug-ins work?

[nikki-k]

Hi!
I have about had it with the whole plug-in/delay compensation thing...but, part of this frustration is due to my own misunderstanding of how things work. So, I am here asking for some help please.

I understand this (and hope I got it right):
The hardware has latency. ADC/DAC- take time to "work." Inevitable, basically unavoidable (without monitoring seperately...).

All audio apps work with large chunks, or packets, of audio data. The size of these chucnks is determined by the buffer size the user sets. At each sample rate (44.1k, 48k, etc), the buffer size will yield a specific, static time. During tracking, this can be bothersome (at best). However, during mixing, since EVERYTHING is delaed by this set amount, no issues should arise...disregarding the use of external, hardware effects or "live" sources, such as MIDI triggered synths, etc.

Here is where I am lacking in understanding, and have received conficting explanations, or ones that avoid a full answer. Plug-ins.

In my own little world of deductions, I have figured out that if the host retrieves a buffer, before it can retireve the next chunk, that retrieved chunk has X amount of time that it could have some processing done to it...via plug-ins perhaps. Thus, if a plug-in requires 70 samples worth of time to process, and the buffer is at 128 samples, then it would introduce no additional delay or latency. Further, the track with this plug-in would then remain "time aligned" with the rest of the tracks present. Correct? Or no? Please correct here!

Certain plug-ins will always require "X" amount of time beyond ANY buffer size due to their nature. An example would be plug-ins utilizing a "look ahead" feature, such as certain dynamics plug-ins. Correct?

Bottom line is, I am trying to understand how the data flow in a DAW "works." Where does the delay come from that needs to be compensated for (auto in most apps)? Why can we not simply choose a higher buffer size to compensate?

I am hoping this thread will allow some technical explanations, while also allowing some people to try and explain things with enough technobabble and "layman's terms" to allow the rest of us to better understand the tools we are using. Ignorance sucks...

[manning1]

nikki......
many moons ago i explained how daws worked at a basic level...
(useing the concept of "buckets" of data) to keep it in laymans terms.
thus you might like to search under my name and add the word "buckets" to save lots of retyping by me.
ive added some notes as follows for you to consider.
1. todays level of personal computers were originally designed years ago for "office applications". over time it was realised i guess by OS architects that many added audio features were needed.
initially the problem was however that....an audio program (normally written in C++) was divorced from the sound card device.
thus this is what happened.
user says record a track>>>>OS says "okey dokey">>>>tells sound card to start capturing audio stream from someone like yourself.
now.as is readily apparent in this situation the OS is like a judge sitting in the middle of things. ...and delaying if only for a few milliseconds.
its rather like you issuing an edict to your kids...but instead of telling them directly....a couple of other folks are involved..thus creating a delay if u see what i mean. thus the concept of "latency".
now of course latency of this kind is anathemia for audio engineers cos audio is a real time process. thus some smart audio software companies found ways to mitigate this OS sitting in the middle biz..by finding low level direct ways of talking to the sound device to minimise latency.
2. computer engineering is not perfect also. there are issues like how long a disc drive takes to read and write data....and issues like internal busses getting clogged.
like your waterpipes at home if you try and slam too much through them.
if you want to see this effect in action put lots of devices plus your audio device
on the pci bus. this is why ive mentionedmany times on GS that if useing
a pci sound device ....try to limit the useageof other pci devices so that ....
the audio sound card has free rein over the pci bus. ie,.....ITS FULL BANDWIDTH rather than being bandwidth limited.
the aforementioned is just a couple of examples to show that personal computer technology still has some imperfections to overcome within computer architecture as it stands. now haveing said that audio software developers that ive talked to try and attempt thru various tedchniques certain problems that arise.
to maintain the user illusion that everythings occurring in real time.
in summary.....there are many techniques programmers can use to minimise
latency. in their programming code. not only that butanother factor is the sound device itself and as you alluded to its own ADA architecture.
also lets not forget the drivers themselves in the sound device.
if they are not programmed properly then problems can arise in this area.
you can see this on a lot of pc's where someone trys to record useing the standard on board sound chip. tho there are some smart audio programs that
can sometimes mitigate problems with on board sound chips.
3. lets look at plug ins nikki....once again a LOT depends how plug ins are programmed. when you as a user tells your audio program to load a program there are lots of things that take place "under the hood".
its like when i start up my chevy van that has an on board computer.
it goes thru a check cycle. if the abs isnt working or oil is low a warning light goes on.
similarly with plug ins. there is a "finite time"....for the plug in to be loaded
i suspect plus computer resources allocatted (eg memory.) and of course the OS is involved in this. then i suspect the host program has top monitor the performance of the plug in. thus there are lots of "housekeeping" activities that occur unknown to you as the user. and when you tell the program to release it cos you dont need it on a particular track....then resources will be released back to the OS prolly etc.
one thing to note. EVERYTHING is a task or process ....and the OS is always keeping a watchfull eye over running tasks/processes.
if your useing XP you will see this if you bring up the xp task monitor.
the aforementioned is why i purposely use only plug ins that are what i call "low resource useage" plug ins. i make sure i check plug ins v carefully before i use them. its no good haveing a great sounding plug in if it locks out all other activities on a pc. and herin lies the rub. the more complexthe dsp algorithm in the plug in to produce say that lush effect you lust after prolly the more computing power is needed due to the coding complexities. ...ie...no free lunch.


nikki .......this is a very complex subject to explain fully and would take pages.
so ive only touched on a few things and tried to keep it simple.
which is not easy in laymans terms. all the best.
ps...maybe as this is such an in depth subject....just ask one question...and i'll attempt to answer in laymans terms,
useing examples...then you ask another etc etc.

[nikki-k]

Hi!
Thx manning1!
I have read your posts before, and the basic concept at that is cool then...I have it "right" in my head for this much of the chain: source->ADC->"soundcard"->OS/DAW->*storage*. I am concluding from various readings that the reverse path- sub DAC for ADC, of course- is much the same on this basic level, save for the intermediate steps of the DAW applying it's own calculations, such as summing and such. It is here I am most concerned with I think..

Let me create a scenario: I have a DAW running via my computer. It is a natively powered 32-bit app. I have a doc loaded that has 8 audio tracks playing back material. I decide to duplicate one of those tracks- let's say, a bass track- so I can apply some effects processing to each version. On the first bass track, I place an EQ. It is a non-FFT EQ, and has no need for "look-ahead" or anything like that. On the second bass track, I place a different EQ, still not using "look-ahead." I also place a compression plug-in that DOES use look-ahead. The amount for this look-ahead is 1024 samples. I am working at 24 bit, 44.1k, and summing all 9 tracks to a single stereo output, routed to my *soundcard* 1 + 2 output for monitoring.

The app I am using has no delay compensation routine for time aligning all the material in question.

Here is where my main interests are:
All plug-ins require time to process- correct? How do I figure out how much time it takes for a plug-in to process?

In the above scenario, the two bass tracks will be out of phase...not time alinged..anymore- correct?

If I have a buffer set of 64 samples, and if the EQ's being used BOTH require less than the time afforded by the buffer (64 samples in this case), then they will incur no additional delay or latency to the tracks they are placed upon- correct?

If I place multiple plugins inline on a track, do the process times for them add together? IOW, if I use 5 plug-ins on a track, and each requires 20 samples of time to process, would they total to 100 samples of time required to process? If yes, what would then happen at a buffer of 64 samples? 128 samples?

Why do certain plug-ins incur a static delay/;atency amount, irregardless of the buffer setting?

Does the buffer setting have anything to do with the time "allowed" for a plug-in to process? If the plug-in process time is greater than the buffer setting, will it produce errors? Or will it simply add latency to the aduio it is processing?

I hope I narrowed it enough- lol! It is quite a bit I am attepting to grasp, but I am tired of being ignorant an d oblivious to this stuff. Wish I knew coding and such better...I'd be sorely tempted to begin dveloping my own little DAW of happiness...

OH! BTW! Any suggestions for resources to learn more? And what if I decided to "code" my own VST plug-ins...good starting points? Is it really THAT difficult?

thx!!!

[manning1]

nikki.....
your asking a ton of stuff there.....lol.
if you would break your questions up into manageable chunks...it would help me more easily answer
in laymans terms.....lol.
ie...Q1 ....what is ??.....blah blah.
anyway i'll try.
your main concern seems to be latency in plug ins. correct ??
if so ...this is not a trivial answer. cos there are lots of factors.
lets attack this one point at a time.
by looking at the factors that might cause plug in latency.
1. each plug in has code executing in your computer of course as a task.
therefore point one is....how efficient is the programmers code ??
only the programmer knows this as well as any latency effects the code might produce. it could vary from plug in to plug in.
maybe the code is very complex ?? for example.
2. how much pc resources does the plug in consume ??
some might consume more than others depending how they are coded.
eg memory.
3. how long does the plug in take to process audio data ??
once again there are many factors involved.
depending on the complexity of the coded dsp algorithms by the programmer.
once again only the programmer of the plug in truly can answer this.
also remember there are other factors that might impact performance like
the architecture of the computer the plug in is running on.
for example processor speed , memory speed and internal computer bus speed factors.
4. then we have to consider the interprocess communication of the host and the plug in communicating with each other.
once the plug in is operating in real time in conjunction with the host multitrack software i believe how you might look at it is in terms of "buckets" of audio data...
being sent from the host to the plug in and vice versa.
ie....like this...in real time....
a... host prepares "bucket"
b...plug in receives "bucket"
c... plug in applies dsp algorithm to "bucket".
d. plug in sends processed "bucket" back to host.
e. host receives "bucket".
its a bit like a "water purifier" ......lol in your house.
a water purifier will only take so much water...thus you have to load several buckets into it....if you have a LOT of water to process...lol.
now let me explain what is happening in real terms.
you need to familiarise yourself with a term called "array".
the ARRAY is crucial to many computer apps and definitely audio applications.
question..what is an array ??
answer....normally an area of memory set aside to hold some type of data thats predefined.
in audios case....we would be talking about audio samples of course.
thus those points a thru e i mentioned above ie..the "buckets" are little areas of memory that hold the audio samples.
so in point a for example ...the host prepares an array of samples to send to the plug in. ie.the "bucket".
now heres where it gets interesting. there is of course a finite (tho small ) time to set up the arrays of course. thus another factor comes into play.
ie..how quick is it to set up the arrays in memory.
as i said this stuff gets a bit complex. ..as you can see from the aforementioned.
i'll try n cover more next time. i'm tired lol.

[manning1]

nikki....part two.
i hate to leave things unfinished....lol.
so follow my logic.
it is too simplistic to say well lets just increase the number of "buckets" or arrays of memory. or the size of the buckets.
cos the problem the programmer has remember is that the code in both the host and the plug ins have to operate on lots of different pc configs from powerfull to less than stellar. or even downright awfull pc configs..lol.
cos not everryone runs dual or quads or xeons or the new intel core duo 2's.
thus you might appreciate the quandary of the host and plug in programmers....
who are human beings...lol....and dont know what crazy configs some folks might be trying to run. thus they prolly try to optimise for the broadest market possible.

in this message i'll try and cover one of your examples.
remember the points a thru e ?? and the buckets or arrays of memory ??
lets say we have 3 plug ins on a track.
heres what i suspect is happening.
(note..array means a bucket of audio samples in memory.)
a. host prepares array1 (eg 2 secs.)
b. plug in one processes array1..and creates output array2.
c. i suspect the host then passes array2 to plug in two.
d. plug in two processes array2 and outputs array3.
e. host passes array3 to plug in three that then outputs array4.
f. host receives array4 and plays it back of course.

where i think you might have difficulty in understanding is that the host itself
is "looking ahead" prepareing the arrays before they are prolly needed.
thus in this virtual world...lol....it seems to the user ie..you that everything is operating in real time.

as you can see nikki...in this context ......there are lots of opportunity for screw
ups. maybe the processor isnt fast enough, or what happens if the user doesnt have enough memory ?? or useing slow memory ?? or a plug in encounters an unexpected error ?? or the plug in causes problems for the host or vice versa ??

i dont know how much ive answered as i'm very tired after recording today.
but if yopu just ask one question at a time i'll try n answer.

[Nerve Nickels]

Oh,oh,oh,oh, I have a two part question? lol

When just monitoring thru a plugin or 'software monitoring' does the HardDrive affect latency?

What is the weakest link in 'software monitoring' in relation to latency? ADC-DAC/BUSS/RAM/CPU/DRIVER/OS/HOST/PLUGIN ... ???

GOOD STUFF thanks for posting....

[microstudio]

Originally Posted by nikki-k

Hi!
I have about had it with the whole plug-in/delay compensation thing...but, part of this frustration is due to my own misunderstanding of how things work. So, I am here asking for some help please.

I understand this (and hope I got it right):
The hardware has latency. ADC/DAC- take time to "work." Inevitable, basically unavoidable (without monitoring seperately...).

All audio apps work with large chunks, or packets, of audio data. The size of these chucnks is determined by the buffer size the user sets. At each sample rate (44.1k, 48k, etc), the buffer size will yield a specific, static time. During tracking, this can be bothersome (at best). However, during mixing, since EVERYTHING is delaed by this set amount, no issues should arise...disregarding the use of external, hardware effects or "live" sources, such as MIDI triggered synths, etc.

Here is where I am lacking in understanding, and have received conficting explanations, or ones that avoid a full answer. Plug-ins.

In my own little world of deductions, I have figured out that if the host retrieves a buffer, before it can retireve the next chunk, that retrieved chunk has X amount of time that it could have some processing done to it...via plug-ins perhaps. Thus, if a plug-in requires 70 samples worth of time to process, and the buffer is at 128 samples, then it would introduce no additional delay or latency. Further, the track with this plug-in would then remain "time aligned" with the rest of the tracks present. Correct? Or no? Please correct here!

Certain plug-ins will always require "X" amount of time beyond ANY buffer size due to their nature. An example would be plug-ins utilizing a "look ahead" feature, such as certain dynamics plug-ins. Correct?

Bottom line is, I am trying to understand how the data flow in a DAW "works." Where does the delay come from that needs to be compensated for (auto in most apps)? Why can we not simply choose a higher buffer size to compensate?

I am hoping this thread will allow some technical explanations, while also allowing some people to try and explain things with enough technobabble and "layman's terms" to allow the rest of us to better understand the tools we are using. Ignorance sucks...

Nikki I thought your post was interesting so I posted it on my DAW forum SAWStudio and asked the developer of SAWStudio Bob Lentini to try and answer your question. Remeber this is the guy who made SAWStudio.. he wrote the code.

I hope it helps you out.

" Well... for SAWStudio... the VST latency compensation is a static thing... the plugs will generally absorb x number of samples (its reported latency) before returning that same data... this allows the plugs to always have some look-ahead data in memory to use in their algorithms... the Mulitrack compensates for this by looping around multiple times for that one track feeding new data, out of sync and ahead of the other tracks and throwing away the returned zero data buffers until the plugin starts returning the original buffers... then these are mixed into the rest of the summing stage.

With DX plugs, the data could be handled the same way, or portions of buffers may be returned, constantly altering buffer sizes on the fly as the plug needs more or less data to process its math. This gets harder to compensate for.

Either way... you can see very quickly that with latency causing plugs in the loop, tracks all get very out of sync internally and there are hundreds of extra memory pointers that must be maintained to keep each track in sync in the final mix.

As far as recording... concerns about recording overdub syncing is not a problem in SS because the recorded buffer will be dropped at the proper offset position in the mulitrack when the regions are created. This is explained in the Ready To Record section of the helpfile.

I have no idea how the other DAWs do it.

Bob L "

[nikki-k]

Wow!
This is very cool- thx for the responses so far!

I think I might take a venture over to the SAW forum then to see if I find more; it sounds like the guy who coded it actually hangs there- wild!

OK...to singularize a little...

1) Plug-ins require time to process- correct?
2) Question: is this a finite time? I thought it should be, since I always hear about "plug-ins reporting delay numbers" to hosts.

3) When a plug-in takes more time than the buffer set, will this produce errors? (halt, severe glitching, etc) Or will it simply delay the audio passing through it the extra time required for it to be processed for this effect? This is the area I am most curious about, but realize deeper understanding overall would probably be required.

Oh- one last thing )for now): If one were to use plug-ins that do not require "extra time" - such as FFT, look-ahead, etc- and if we stayed compeltely ITB, would we need delay compensation? Due to conflicting answers, I am unsure of this answer. Deductively, with limited knowledge, I would think not. If 3 plug-ins required XX time to process, raising the buffer would allow for the time thos eprocesses required, and thus no "extra latebcy due to plug-in processing" would be added. Correct? Or no?

Thx again- this is really nice of you guys!!!

[microstudio]

Originally Posted by nikki-k

Wow!
This is very cool- thx for the responses so far!

I think I might take a venture over to the SAW forum then to see if I find more; it sounds like the guy who coded it actually hangs there- wild!

Yes Bob is on the forum all the time. He responded to my post in less then 10 minutes...Crazy... he is the one who wrote the code... talk about tech support.

[manning1]

nikki....
bob lentini i highly respect and would suggest you would get a lot of info thats usefull from the dialogue. particularly as he is familiar with the code internals of his app.
nerve.....
there are just too many variables to say which is the weakest link in the chain.
in fact its a whole smorgasbord of factors as i tried to elucidate above.
for example .....now ive had a little nap....its important to understand that the host audio software "looks ahead" before it actually needs the data from disc for mixing. for example while playing the current 4 secs of mix....it will already be preparing a number of mix buffers for future play to maintain the illusion of real time mixing. simplistically....the audio software plans ahead what it needs... to maintain the illusion that everything is occurring in real time.
NOTICE THE WORD ILLUSION.

its very important you guys understand this MAIN POINT.
that is..in digital audio .......things are done/prepared before the audio mix time segments are needed by the user ..to keep it simplistic....thus overcoming such factors as the time it takes for a disc drive to read data.
all modern multitrack software does this. to overcome the many possible limitations of computer architecture.
now haveing said that if the disc drive is goddam awful it might have an impact.
a goddam awful processor or lousy memory also.
try running an old celeron sometime with lousy memory and youll see your track counts and plug in counts severely impacted compared to say a dual core modern processor.

going to bed now ....nitey nite. interesting thread you started nikki.

[Nerve Nickels]

If I have a software mixer with plugins loaded and live audio coming in/out, is the harddrive in use? if so for what?

no recording, no plaback, no samples....
Just effect plugins and real audio A/D/A

[adzski]

Originally Posted by manning1

nikki......
many moons ago i explained how daws worked at a basic level...

thus you might like to search under my name and add the word "buckets"

http://www.simpsonstrivia.com.ar/sim...ck-riviera.gif

[manning1]

nikki....
ok ive had a sleep....so lets look at this all in further detail.
play this game with me nikki ....(simpson trivia notwithstanding...lol.)
you know "pimp my ride" ??
well this ones called "trick my daw" into coughing. ...lol.
and i dont care the daw or platform...pc or mac....or audio software.....
all daw programmers/software will use the same buffer/bucket/array tricks to
let the user think everything is being done in real time. ..as i explained before.
heres the scenario.
1. lets take a typical 3 minute song.
you play it back starting at zero marker for the whole 3 minutes.
through the whole song.
what do you think is happening...??
well ..as i alluded to earlier the mix arrays are being prepared in advance of them
being neded. thus when your playing say at 5 secs....mix buckets/arrays/buffers....
call them what you will....are being prepared in advance. ie..forward in time.
to maintain the illusion of real time mixing.
now to the trick my daw part.....
BUT WHAT HAPPENS IF...
2. you click on the timeline and start playback other than zero. ??
which some daw software lets you do.
now this is where things get a LOT TRICKIER for the programmer.
HE DOESNT KNOW WHERE YOUR GOING TO START THE MIX FROM.
but THE RUB is that all those forward buffers/buckets/arrays must still be prepared includeing the first one to play.
thus there are a whole bunch of "housekeeping activities" that have to be done
prior to you hearing mix playback. for example lots of arrays in memory have to be initialised. and the samples have to be dragged from disc to set up the mix memory arrays.
thus if you want to see your daw cough.... you might have to wait a second or two after clicking on the time line at a non zero point...say 1 minute for example where you have 50 tracks playing with lots of plug ins loaded.
now you might wonder....what are the factors that govern how long i have to wait
after clicking on the time line at a non zero point ?? for mix playback ??
good question...so anticipating this...i'll answer it.
on some powerfull pc's youll notice the lag is very small. pretty instantaneous.
while a less than stellar pc the lag will be longer.
lots of factors come into play here. processor speed. memory speed. the hard drives, internal busses, and a myriad of small details.
thus before buying a new pc its a good idea to load a tester pc up with your typical project with lots of tracks and plug ins .....click on its timeline at say one minute for example...or somewhere you know there would be lots of tracks and/or clips playing back with their plug ins....and see how long the lag is from the moment you clik with the mouse the play button in the daw software.
some pc's will cough....others wont. depending on the configuration.
3. now just to round things off. to provide further details for you to think about.
lets go back to when mix is played back from zero.
on some systems in this scenario youll possibly notice that mix playback is pretty rapid. why is this ?? the answer is simple.
in the point 2 above the programmer cannot pre-prepare the buffers cos he doesnt know where you might click in the timeline.
thus he is in a bitof a bind...lol.
BUT....for playback from zero..some smart daw audio software programmers ...
on the previous halt of the multitrack software will already have pre-prepared buffers in anticipation that you might start playback from zero.
do you see ??
4. now lets turn to plug ins themselves and your quandaries.
my personal story.
when plug ins first came out.....years back....lots of my daw friends were just gung ho on them. then the freebies arrived .....and they went like mad maniacs after them. laughed at me for not following the herd. THEN THE PROBLEMS STARTED....
i DID TRY TO WARN THEM of course. often unheeded. then they found out their pc's were woefully inadequate to run all the plug ins they wanted to run.
this is why i didnt rush into them. now this isnt to say that i dislike plug ins.
i just felt...and i still do that computer architecture had not matured to the right point. which they subsequently found out.
i stayed with useing the plug ins built into the multitrack software i use.
due to their efficiency. only now with the coming of dual and quad cores do i feel happier running lots of plug in instances.
in SUMMARY....i felt at the time that plug ins were coming out in advance of the
computer hardware power needed to run them.


nerve......
youll have to go into a lot more detail...on your set up and what your trying to do.
cos there are lots of answers.
for example.....is this your scenario ??
people are playing out in a recording room.....and without recording to the hard drive... you have some set up where the mic signals are coming into the daw and then effected in some way
then pumped out to the monitors ?? i dont understand what your asking. if its this scenario..still latency issues might occur depending on your particular daw. there are many factors that can impact once again. the convertor design/sound card engineering....the plug in design themselves etc....etc...

ok..i think i see what your getting at. re hard drive.
theoretically if not recording and just passing thru i doubt the harddrive is involved. but its difficult to say.
the host software your useing to load the plug ins might be useing the hard drive for something. only the programmers of your host software can truthfully answer this. as well as the programmers of your plug ins. your useing.
all i can give you is conjecture. cos i dont know the internal code workings or design of your host and plug ins.
in this scenarion iwould suspect that on start up memory is allocatted to your host program....then also for each time you add a plug in. but bear in mind...depending on programmer design....
some code modules/subroutines might reside on disc and only
be loaded as and when needed. difficult to answer as it depends on knowledge of the internals of the host and plug ins your useing.
thus i would ask the developers of such.