HOWTO: Low DPC latencies (<100 us) on bootcamped Macbooks (Pro)

[Timur Born]

Here is a small HOWTO for getting the lowest possible DPC latencies (<100 us) on bootcamped Macbooks Pro (late 2008):

Disclaimer: I did all tests on my late 2008 Macbook Pro Unibody 2.8 GHz model with NVidia chipset and graphic. Most of the following suggestions should apply to standard Macbook models and likely older generation as well.

First of all Intel Speedstep leads to dropouts and higher DPC latencies on small load! Unfortunately all tools that are supposed to manually switch Speedstep off don't seem to run on the late Macbooks (Pro) while on OS X you can use "Coolbook".

Your only way to make sure your processor is clocked high enough and not dynamically switching is to put up a constant load (like running your DAW pretty hot or running Prime95 at "Idle/Lowest" process Priority in the background). I will keep investigating if I can find a tool to switch Speedstep off.

Most importantly (to get rid of really bad DPC latency spikes):

Kill the process "KBDMGR.EXE"!

That's Apple's driver for controlling brightness and keyboard lighting via the function keys and setting tap options for the trackpad. It seems to have broken multithreading! You can also the CPU affinity of KBDMGR.EXE to CPU1 (not CPU0!) which will help decreasing DPC Latencies alot, but there will still be Audio dropouts.

Here's a small toolkit I put together that allows you to conviniently enable/disable Apple's "Boot Camp" tray application (KBDMGR.EXE) via an icon link and/or keyboard shortcut. Optionally it will switch the function of the F-Keys automatically for you depending on whether Boot Camp is loaded or not.

Furthermore it automatically turns Boot Camp's CPU priority to "Idle" and CPU affinity to CPU1 in order to turn down the bug induced DPC Latencies and prevent dropouts with Windows sounds and Media Player playback. Professional Audio users will find that only turning off Boot Camp will allow low audio latency usage. Installation instructions are included in the README.TXT for your convinience.

Boot CampED - Download Page

Turn off the Broadcom 802.11N WLAN driver via Device-Manager or update to the latest drivers via Microsoft Update Catalog.

Like on OS X the Airport module can lead to audio dropouts. The DPC Latencies produced by the Broadcom driver are less regular than the KBDMGR thing, alot higher in value. Best thing is to try for your own needs.

Update:Meanwhile a new Broadcom drivers was published via Microsoft's Update Catalog named "Broadcom - Network - Broadcom 4322AG 802.11a/b/g/draft-n Wi-Fi Adapter " (4322 is the chip used). This one comes with both low DPC latencies and finally the ability to use the full rate upto 300 mbit/s. Go get it! For safety you might still want to turn WLAN off during critical audio work though.

Change the graphic-card driver to "Standard VGA Driver" via Device-Manager or use RIVATUNER to enforce a fixed clock-rate and performance mode.

Update:The dynamic clock-rate switching happening with NVidia drivers in order to save power and keep temperatures low leads to extreme DPC spikes for each switch and constantly high DPC latencies when it settles in low performance 2D mode. RIVATUNER's "Enforce Performance Mode" option can be used to set the card to a fixed clock-rate. I recommend using "Low Power 3D" for audio work.

User of XP might think that they don't need this, but be aware that on XP the NVidia driver keeps running at highest clock-rates in "Performance 3D Mode" all the time. Via RIVATUNER you can switch to "Low Power 3D".

Turn off the ACPI compliant Battery driver via Device-Manager

This driver polls the battery for its current load status and produces a small, single, short spike exactly every 15 seconds. In my own tests I found that it doesn't seem to affect low latency audio performance. Furthermore turning it off will remove monitoring of your current battery status. But if you are running on power-chord anyway and want to make absolutely sure you can turn it off.

All other devices don't add much if anything to DPC latencies, but can savely be turned off if you don't need them (like Nvidia LAN, Bluetooth, Onboard High Definition Audio).

Attention: Removing the Battery while the power chord is connected results in permanently reduced CPU clock (downto the lowest clock setting possible). According to Apple this is done to prevent overloading the power-supply during heavy load as it needs the assistance of the battery from time to time.

[Timur Born]

Unfortunately I just found out that for low latency performance you need to kill the KBDMGR.EXE process completely!

Even when there is no increase in DPC and IRQ cycles and near to no CPU load happening from KBDMGR it still interrupts the audio of my Kore 1. Interestingly the buzzing/zipping noises that are happening sound very much like the noise I get on OS X (eventhough at higher load). So something about that Keyboard/Screen lighting control is fishy.

[Robert Randolph]

Excellent information. I did this a few months ago and had to figure this all out and find the info myself.

Nice to have it all in one place thumbsup

More information here: RME User Forum / FF400, Macbook Pro, Bootcamp - weird buzzing noises

Note: Nvidia powermizer should be turned off as well. It has great effect on DPC latency.

[Timur Born]

Thanks for the flowers!

There are no Powermizer settings in Vista, so nothing to change there.

[hachre]

CPU Throttling
You can change the CPU throttling behavior in both Windows XP and Vista without the help of external software.

On Vista you go to Power Settings and then advanced settings. In that tree you will find values for minimal and maximum CPU frequency in %. Just put both on 100% and the CPU will no longer reclock in Vista.

On Windows XP you also go to Power Settings but you will see that there aren't any obvious functions provided to change the CPU power saving. Instead it is hardcoded into the default power profiles. "Always On" does not do any power saving, while "Maximum Battery" does for example. Just set it on "Always On" and the CPU should no longer reclock in XP.

Permanently Setting Affinity For kbdmgr.exe
If you don't trust patched versions of the kbdmgr.exe that random people provide, you can patch the file yourself!

First you need to download a program called imagecfg.exe from Robpol86.com - ImageCFG.exe.
You should put the file into the same folder you have the kbdmgr.exe in, that would normally be Program Files\Boot Camp\.
Next you have to kill kbdmgr.exe via Task Manager (CTRL ALT Delete, Processes, Kbdmgr.exe, End Process).

Now you need to start cmd (Start -> Run, Windows Key + R).
Change to the directory kbdmgr.exe is in (cd C:\Program Files\Boot Camp)
Now if you have imagecfg.exe in the same folder you can simply type "imagecfg -a 0x2 kbdmgr.exe".
It should say something about using 0x00002 or similar as a new affinity.

Once you restart Windows or restart kbdmgr.exe manually it will run on CPU 1 (not CPU 0) and produce no latency issues.

Good Luck!

[Timur Born]

Originally Posted by hachre

CPU ThrottlingYou can change the CPU throttling behavior in both Windows XP and Vista without the help of external software.

This does not work for Intel Speedstep!

Permanently Setting Affinity For kbdmgr.exe

While this leads to much better DPC Latencies you still have to kill kbdmgr.exe in order to prevent audio dropouts. That drivers is broken and I hope Apple will fix it anytime soon.

[Timur Born]

Originally Posted by Robert Randolph

Note: Nvidia powermizer should be turned off as well. It has great effect on DPC latency.

On XP (in contrast to Vista) you can access PowerMizer settings, but they don't have any effect on the late 2008 MBP, neither DPC wise nor battery/power wise.

Furthermore the NVidia driver produces no relevant DPC/load spikes on XP, but it's a good idea to deactivate "Cursor Acceleration" (set the Troubleshooting setting one tick down). The standard VGA driver on XP (only accessible by uninstalling the NVidia driver) shows lousy performance and should be avoided altogether.

It's also not necessary to deactivate the ACPI Battery service on XP, because it produces smaller spikes in irregular intervals only.

All in all the lowest DPC latencies I could measure on XP are downto 2 us, but with the tweaks listed in this thread performance of Vista is roughly on par with XP if not even slightly better on my MBP 2.8 GHz.

[Timur Born]

There is some good news: Because of the Windows 7 Beta that will be published in the near future Microsoft has put up a new "Microsoft Update-Catalog" at: Microsoft Update Catalog

This includes many updated drivers for hundreds of hardware interfaces including Broadcom WLAN drivers and Nvidia 9600M GT drivers that are found on the current Macbook (Pro) models.

Especially the Broadcom driver is very interesting:

1. It finally enables full 300 mbit/s 802.11n support, whereas older drivers (including OS X) only connect upto 130 mbit/s with many routers. Whatever the driver does to switch 300 mbit on it even stays on when you reinstall the older drivers!

As a drawback you cannot use channels 12 and 13 anymore (in Germany) as this driver seems to be made for the USA only.

2. It dramatically improves DPC latencies

From this (Bootcamp Broadcom driver version 4.170.77.3/03-21-08):



To this (Broadcom driver version 5.10.38.26/10-22-08):



I would still advice to turn off WLAN when doing critical Audio work, but at least you only have to do it for special circumstances now.

The Nvidia 9600M GT drivers are version 177.48 and thus slightly newer than the Bootcamp ones. Unfortunately they shows about the same DPC Latency behavior as before. So for critical Audio work you still have to fall back to the "Standard VGA Drivers".

PS: Curiously the new driver version of the Broadcom drivers resembles what OS X reports as "Firmware Version: 5.10.38.24". Even more strange Windows 7 Beta comes with a driver version 4.176.75.11/10-23-08 that is one day younger, but less performing than the version 5 drivers.

[Timur Born]

Some Updates:

1. NVidia now officially offers graphic-drivers for their mobile line of GPU including the 9400M/9600M that are found in the Macbook (Pro). So you can just download the latest drivers directly from NVidia now without any need to mess with INF files or any other hack.

2. I have found the source for the DPC Latency spikes that are happening with NVidia graphic-drivers installed. Actually there are two reasons for this happening.

a) The Nvidia driver dynamically changes the GPU clock-rate (including GPU memory) up and down depending on current graphic-load. Everytime the clock-rate changes (especially when it changes down) a huge DPC Latency spike happens. This has been reported with Geforce-8 based GPU on other forums and unfortunately the 9600M is more of a Geforce-8 than Geforce-9.

b) When graphic-load is low (like with most 2D based DAWs) the GPU memory is clocked down to only 100 MHz (from a max of over 700 MHz), this leads to regular and ongoing DPC Latency spikes when many dymanic elements are displayed by a DAW.

I have already managed to temporarily get rid of NVidia based DPC Latency spikes and I'm working on finding a solution that allows to permanently get rid of those during DAW work. I will report back on that.

Until then the best performing option that leads to the lowest average DPCs is still to switch the graphic-driver to "VGA Standard Driver" (minimum power draw will be upto 5W higher though).

3. Nvidia also offers an application called Ntune at version 6. NTune allows you to overclock/underclock the Frontsidebus (and thus the CPU and RAM), modify RAM timings and overclock/underclock the GPU. Only experienced people who know what they are doing should mess with it, but it's nice to have for those of us who like messing around anyway.

[Timur Born]

I have one bad news and two lightly good news:

Bad news: I was not able to keep the Nvidia 9600M GT from switching down to 2D mode sooner or later. Even when it is possible to keep it from switching clock-rates in performance-3D and low-power-3D by setting those to the same clock-rates once it goes down to 2D the performance is rather bad aka DPC latencies are regular high enough to make it unusable for low latency audio work. It seems that the lower the memory clock the higher the DPC latencies.

1. good news: "For a limited time, provide your feedback on Rel 179 notebook drivers."

Feedback Form for NVIDIA Notebook Drivers

So for the first time they offer us end-users to tell them what we think about the performance of their drivers. I will use that opportunity to tell them about bad DPC behavior!

2. good news: NVidia not only officially offers drivers for the graphic-card but also for the system chipset. You can find it under "NForce -> NForce7 -> 730i/Geforce 9400m/9300m mGPU". The current version is already a bit more up-to-date than the ones on the Bootcamp disc.

3. good news: I forgot what it was, but I'm sure there was a third good news.

[Timur Born]

Update: I didn't give up and finally found a way to keep the NVidia graphic from producing DPC Latencies on Vista/W7 while NVidia drivers are used (instead of just using VGA Standard drivers).

Furthermore I found that the only reason why Windows XP is not affected by NVidia induced DPC Latencies is because the graphic card's clock-rates are never changed but keep running at the highest settings (Performance 3D), not even when installing the latest drivers. This is a bug! The obvious drawback is that power drain and likely GPU temperatures are somewhat higher compared to Vista/W7.

Actually I found two ways to stop NVidia DPC Latencies from happening on Vista/W7. Since the reason for the DPC Latencies are:

1. dynamically switching clock-rates
2. running at low performance 2D mode most of the time

We need to get rid of both! One way is to switch to the Microsoft VGA Standard driver which makes the 9600M run at fixed "medium" clock-rates for both GPU and memory. But since this not only means to abandon Aero, but also Nvidia video-playback acceleration and other driver goodies it's only the second best option.

The best option I found is to use an application called Rivatuner that offers a somewhat hidden option called "Force Performance Mode". This allows to set the NVidia driver to one of it's three modes: "Standard 2D" (we don't want that!), "Low Power 3D" (comparable to Standard VGA driver clock-rates), "Performance 3D" (that's what XP gets stuck at).

I recommend enforcing the "Low Power 3D" mode for audio work. You need to restart the computer in order to switch the "Force Performance Mode" on and off in case you need to get the old behavior back or need the full "Performance 3D" mode for gaming, but that's not too much of an inconvenience. It also works on XP by the way.

The second method I found is a bit too geeky and needs too much work to setup so I wont explain that in detail. The general idea of that second idea is that once a DAW GUI is loaded any switching of GPU clock-rates forces the GPU back to "Performance 3D" mode (even when the rate doesn't actually change). So a regular occurring switch every few seconds would keep the rate constant. I prefer method 1 even when that means to reboot for changing it.

[Schaap]

Great work! I had to do similar 'work' about a year ago when I had errors and glitches etc. The same components were responsible for it.
An WLAN card that raised the DPC from 80 to 900 and after a lot error and trial it seemed to be the Nvidia driver that caused the errors.
I had to go back to Nvidia drivers that were 2 years older than the current version at that time.
You should check components that are not present anymore in the computer but still registered in your OS in a way. Can also be a source of trouble
I'm on a PC with windows XP btw

[Timur Born]

I spent some time to get more convinient control over KBDMGR and the corresponding settings application (the one you configure the trackpad and F-keys behavior besides other stuff).

Right ALT-. will switch KBDMGR on and off now. Additionally the F-keys are automatically switched to their special Macbook functions (like controlling brightness) when KBDMGR is on and to normal F-keys function when KBDMGR is off.

Furthermore when Windows boots up and when enabling it via ALT-. then KBDMGR is set to "Idle" priority and CPU1 affinity. That helps for normal desktop audio output, but for professional audio work it still needs to be turned off.

Right ALT-, will open up the settings dialog now. It also works with KBDMGR being disabled including setting brightness via the slider.

PS: While I was at it I also remapped the key left to the backspace to act as DEL (it's the never used key `´ on german keyboards) and the right CMD key to act as CTRL (so I can use it like on the OS X).

[Timur Born]

Here's a small toolkit I put together. Installation instructions are included in the README.TXT for your convinience.

Boot CampED - Download Page

[Timur Born]

I put up some better download links!

[sonare]

I get a 404 error with both those links-- anything newer?

Rich

[Timur Born]

Sorry, I thought no one was reading this thead here anyway, so I didn't update it. Now the original post is updated with the latest findings and there is a working link to v1.1 and all future versions of my scripts.

Boot CampED - Download Page

I've already written a version 1.2 that tries to automatically get rid of the tray-icons once KBDMGR is killed (not needed on Windows 7). It also allows you start KBDMGR later if you want to start Windows without it (some genius at Apple came up with the idea trying to prevent that). I didn't rewrite the Readme yet, though. Once I've done that I will post it via the same link.

The latter is especially useful when you are using VMWare Fusion or Parallels, because KBDMGR causes loooong boot-times when booting the Boot Camp partition via the virtual enviroments.

[sonare]

I am in Boot Camp 2.1 in XP Pro on a MBP 2.33 17' and am happy to report that by disabling the Atheros Wireless Network Adapter I was able to reduce latency to about 100ms with a peak of 137.

What you are doing is a very needed help in getting bootcamp to behave-- thanks for your efforts-- even though I didn't actually have to download your fixes.

Rich

[Timur Born]

Originally Posted by sonare

I am in Boot Camp 2.1 in XP Pro on a MBP 2.33 17' and am happy to report that by disabling the Atheros Wireless Network Adapter I was able to reduce latency to about 100ms with a peak of 137.

You should take a look at Microsoft's "Update Catalog". As far as I remember there are also new drivers for the Atheros WLAN chipsets. Maybe these fix the DPC problems just as much as the new Broadcom drivers did.

What you are doing is a very needed help in getting bootcamp to behave-- thanks for your efforts-- even though I didn't actually have to download your fixes.

Just sharing the knowledge I obtain while making it run for myself! It's been more work to publish than I thought though.

You should check for differences between enabling and disabling KBDMGR though. Maybe it's behaving better on older Macbooks, but on mine (unibody) even after using the "affinity" trick and with low DPC Latencies being reported it leads to audio-dropouts as long as KBDMGR is running.

[theblue1]

Originally Posted by Timur

Sorry, I thought no one was reading this thead here anyway, so I didn't update it. Now the original post is updated with the latest findings and there is a working link to v1.1 and all future versions of my scripts.

Boot CampED - Download Page

I've already written a version 1.2 that tries to automatically get rid of the tray-icons once KBDMGR is killed (not needed on Windows 7). It also allows you start KBDMGR later if you want to start Windows without it (some genius at Apple came up with the idea trying to prevent that). I didn't rewrite the Readme yet, though. Once I've done that I will post it via the same link.

The latter is especially useful when you are using VMWare Fusion or Parallels, because KBDMGR causes loooong boot-times when booting the Boot Camp partition via the virtual enviroments.

I'm reading it and I don't even have a MBP. I just think it's fascinating. And it also gives me insight into issues that might effect me in the future with W7. (Hopefully, folks will have their ducks in a row, by then, properly functioning drivers, etc.) And it encouraged me to fire up the DPC Latency Checker I'd DL'd some time ago and take a look at my own WLAN overhead. It's nothing crazy, but it does push the 'current' latency numbers from mostly single digits into the 35-65 μs. Since I have the option of ethernet or WLAN, it helped push me to switch to the wire for my desktop.

I hope people appreciate the work you've done here!

[Timur Born]

Don't worry about your DPC Latencies, anything around 100 us is premium well. On the other hand you can well suffer from drop-outs and issues even when DPCs are reported to be low. It's a very useful tool to find driver culprits, nothing more, nothing less.

And thanks for the appreciation!

[Robert Randolph]

Awesome work, thank you.

[theblue1]

Originally Posted by Timur

Don't worry about your DPC Latencies, anything around 100 us is premium well. On the other hand you can well suffer from drop-outs and issues even when DPCs are reported to be low. It's a very useful tool to find driver culprits, nothing more, nothing less.

And thanks for the appreciation!

I have a very modest machine (considering the work I do with it) so keeping it lean and mean is important. I realize the DPC Latency Checker is just one measure of performance and overhead, but it is a handy one and the time-based display is quite helpful in seeing the timing of processing spikes.

By analyzing the issues you were having in a pretty methodical manner you've done a real service to others with similar set ups, but the insights gained are valuable for others, too.

Thanks, again!

[Timur Born]

Just to mention it:

Currently I'm testing VMWare vs. Parallels vs Virtualbox, that's why I worked on my scripts to allow booting without KBGMGR. Apart from the obvious office work I'm mostly interested in using Cakewalk Sonar from time to time without having to reboot (bootcamped) and will likely report about the results in a new thread once I'm done with testing.

[TheRealRoach]

Hey man,

I read your posts, thanks for the all the research made public. I've been battling with my 9400GT video card chewing through my PCI bus. Let's cut to the chase:

What you see in this screnshot is Cubase 4 running, with 3 Waves SSL GUI's behind the project window (i.e. not visible, but open). DPC latency checker is running and you'll notice spikes of up to 946 microseconds in the timeline between -30 and -15.

Between -15 and 0 I closed the Waves SSL GUIs.

What should I do in RivaTuner to address this? I've tried a bunch of things but have gone back to default settings because nothing seemed to make a difference. Maybe I didn't do it right?

[Timur Born]

Hey Mr. Rocha,

usually spikes below 1000 us (1ms) shouldn't be much of a problem. If you are still experiencing dropouts eventhough DPC latencies are low then make absolutely sure that the KBDMGR.EXE process isn't running anymore (check via Task-Manager).

The following screenshots should help you find the options in Rivatuner that allow you to set the graphic-card to a fixed performance level.

First you need to unlock the option:



Then you go to the System Tweaks settings:





Then you have to set a desired performance level (I advice to use lowpower 3D unless you want to play some performance 3D games):

[TheRealRoach]

Many thanks, I will try when I get to the studio later this afternoon!

I noticed that there were a couple broken images in your post, but there didn't seem to be any missing information.

I noticed in your first image, RivaTurner->Power User-> "EnablePerLevelForcing", that you have Enable LowPower 3d Control Enabled as well. I believe I enabled "EnablePerLevelForcing" only. On reboot the video card clocks never changed speeds, but hopefully this will do the trick.... the clocks SHOULD change speeds when activating Low Power 3d right?

By the way I am on a PC/Windows machine. Also, my sub-1,000 microsecond latency spikes are in the "green", but I do get snaps and ticks in playback and also in recording. Occasionally the ASIO overload light with register an overload, but I don't hear any drop out. The amount of latency being spiked seems to be right on the fringe of unnoticeable and noticeable.

[Timur Born]

Originally Posted by TheRealRoach

I noticed that there were a couple broken images in your post, but there didn't seem to be any missing information.

Don't worry about the "seemingly" broken images, nothing is missing. It's tricky to put images into posts on this forum and for some reason Internet Explorer and Safari show those "broken image" icons in front of the not so broken images. Neither Firefox nor Opera seem to do that.

I noticed in your first image, RivaTurner->Power User-> "EnablePerLevelForcing", that you have Enable LowPower 3d Control Enabled as well.

That's not needed. I use it only for my own geeky control over all clock-rates. It allows to set clock-rates for that LowPower 3D mode via Rivatuner which normally would not be manually changeable (only 2D and Performance 3D are changeable by the default NVidia behavior).

I believe I enabled "EnablePerLevelForcing" only. On reboot the video card clocks never changed speeds, but hopefully this will do the trick.... the clocks SHOULD change speeds when activating Low Power 3d right?

Actually the clock should be fixed to one speed after the change and never change neither when idle nor when active. The performance level chosen dictates at what clock the card runs after the reboot. If you want to define yourself what clocks are used then switch on the "geeky" setting mentioned above.

By the way I am on a PC/Windows machine. Also, my sub-1,000 microsecond latency spikes are in the "green", but I do get snaps and ticks in playback and also in recording. Occasionally the ASIO overload light with register an overload, but I don't hear any drop out. The amount of latency being spiked seems to be right on the fringe of unnoticeable and noticeable.

The dropouts don't necessarily have to be based on DPC Latency though, it could also be lacking internal priorisation of the DAW (Ableton Live is a good example of how priorities can affect performance alot).

Something you also might want to consider is that VST Plugin Buffers may be linked to ASIO Audio Buffers with your DAW and that some plugins needs these buffers to be multiples of 16 or even 32 to work properly.

[TheRealRoach]

I've followed the steps and it is not having any effect on the DPC readings as mentioned above. I'm currently trying an older video card with older drivers.

Edit: No dice. Older card and driver performed the same.

[Lawrs]

Uninstall all .net framework, and reinstall .net framework 1.1 and 2.0 ONLY. Helps some cases of asio chokes. Thanks timur, your advises helped me alot.