Logic Pro Benchmark Test: Live Instrument

[speakerfood]

- This is an extension to the Benchmark test from Evan -

Although it's important to find out how many tracks you can run simultaneously on one machine (using one or more cores), I also wanted to find out how much one single processor in a single- or multicore machine can handle.

The reason for developping this test, is that although I have 8 cores in my Mac Pro, the audio sometimes still cracks/pops because it can't handle the processing chain of the software instrument I wan't to record, with 'software monitoring' enabled.

I created session with one software instrument (ES1) that is routed through several busses with Space Designer instances on it. This forces Logic to handle all processing on a single core. You can download the Logic song here:

www.speakerfood.nl/logic/SpeakerfoodSingleCoreBenchmark.zip

- Open with Logic 9.1.3 or higher
- Set the I/O buffer to 64
- Set Process Buffer Range to small
- Set the Track Mute/Solo to CPU-saving (under the General tab).

Play the song, and mute/unmute the Space Designer instances in the mixer untill the audio stops to crack/pop. Count the number of Space Designer instances that are unmuted (each track/bus has 14 instances).

Results with buffer @ 64 (updated 05-04-2012)

Hackintosh 4.6GHZ i7 2600k, RAM @ 1866MHz: 37
Mac Mini Server 2Ghz i7, RAM @ 1333MHz: 23
Hackintosh 4.6GHZ i7 2600k, RAM @ 800MHz: 18
Mac Pro 8 x 2.8Ghz 2008, RAM @ ?: 14
MacBook Pro 2.53GHz i5 2010, RAM @ 1066MHz: 11
Mac Pro 2,66 2006, upgraded with 2 x 2,66 Intel Xeon X5355 Quad-Core, RAM @ 667 MHz: 8

[gdoubleyou]

62 Space Designers. interesting.

2GHz quad i7 MBP 4GB RAM OSX10.6.8

[The Beatsmith]

Do you mean that you have a 2.8GHZ Octo Mac Pro from 2008 or do you have the 2.66 from 2009?? I have the 2008 model... from your results i'd imagine you have the same?

I run out of cpu on one core relatively frequently. I run 64 buffer and use software monitoring (no hardware solution). Things like running Kontakt with large sample libraries can easily make it crap out. Try routing these to multi out (stupidly handled on the same core in logic), it's a no go.

From evans test it was clear that the current/2010 2.8ghz mac pro's can run the same number of es2's overall vs my 2008 mac pro, but with half the number of real cpu's. So i guess it figures that per core, they are about twice as fast. Gdoubleyou's results seem to confirm this... not sure how hyper threading factors in/helps.

Either way... damn...

PS... i think it's worth mentioning which audio interface is being used, too. Unless everyone just runs core audio with internal audio

[Kane]

i7 2600k @ 4.8, Lion

Can run all 84 in test without problem. Impressed with the Macbook Pro i7 performance!

At 32 buffer I can get only about 16. OsX really is poor at low-latency.

[TAFKAT]

Originally Posted by speakerfood

Although it's important to find out how many tracks you can run simultaneously on one machine (using one or more cores), I also wanted to find out how much one single processor in a single- or multicore machine can handle.

Hey,

Multiprocessing is assigned on a per track basis in the host DAW's tho, so activating the plugins on individual tracks will load balance across multiple cores.

What am I missing here ?

It would be great to get some clarification on what exactly is happening when enabling the software monitoring in regards to the actual playback latency,.

Going on what Kane posted its sounds to me as its shifting from the Hybrid 1024 to the respective buffer setting assigned on the audio interface.

That could definitely cause some issues at lower latencies when needing to live play extra VI tracks to heavily loaded sessions for example, that had the benefit of the hybrid playback engine smoothing things out in the back end.

[Kane]

Logic assigns all "live" parts (effects, instruments etc.) to a single core.


> Going on what Kane posted its sounds to me as its shifting from the Hybrid 1024 to the respective buffer setting assigned on the audio interface.

Yes, that's exactly what it does. Problem is, the myth of Logic's performance is revealed to be just that; it performs poorly for live parts and horribly at low-latency. Irrespective of buffer-size, all live parts get to share a single core.


>That could definitely cause some issues at lower latencies when needing to live play extra VI tracks to heavily loaded sessions for example, that had the benefit of the hybrid playback engine smoothing things out in the back end.

Yes, you get situations where you can't reliably record-enable tracks. To be honest though, I think (barring the poor performance) that this is the best possible system as it gives you more mixing power without reducing live power (barring the poor performance!).

It's really disappointing how OsX performs at low-latency and I can't believe that nothing has been done about. Windows literally wipes the floor (sometimes by order of 3x+ on the same hardware) with it on ANY application you choose. Still prefer OsX by quite a lot (don't mind Windows but Lion does it for me), but the DAW performance is horrible. Not only that, but try actually doing anything at these low latencies. Got two 24" monitors (at 1200x1920) here; in Windows I can move windows around/resize without any audio problems. Under Lion, as soon as I start resizing large windows, click hell...

[valis]

Are you using a Mac with Bootcamp or a Hack, and what gpu & audio interface are you using Kane?

[florent]

do you guys have a RAM problem with Logic ?I can run more VI with cubase in a same mac !Logic chocks very fast with VI's

[TAFKAT]

Originally Posted by Kane

Logic assigns all "live" parts (effects, instruments etc.) to a single core.

Irrespective of buffer-size, all live parts get to share a single core.

Hey Kane,

That makes sense as load distribution is per track as I noted earlier, so when live playing a respective instrument, load is placed on a designated core.

My question was more in reference to how that relates to the actual benchmark here that has multiple tracks that the plugins are assigned to.

The only significant change that seems to be apparent on this test over the previous is that by enabling a live track , playback latency is assigned to the actual audio interface buffer setting , which is very useful in itself in getting comparative performance against the other hosts that do not have a hybrid engine approach.

I have been asked numerous times with my own work in this area to compare Logic to the other hosts ,but until now I wasn't sure how to get a fairer head to head, now at least I have some avenues to explore.

[Kane]

Originally Posted by TAFKAT

Hey Kane,

I have been asked numerous times with my own work in this area to compare Logic to the other hosts ,but until now I wasn't sure how to get a fairer head to head, now at least I have some avenues to explore.

Yes, this is the way to see performance at different buffer sizes (at least on a single core).

One thing you can do to compare across systems is to limit the system to a single core only (utilities to do this; although some DAWs have the option to disable multi-processing, you still get benefits from the extra cores). That way you can compare Nuendo/Live/Logic etc across a single core.

Things aren't pretty though, performance is WAY below the levels of Win.

[Kane]

valis

Using an optimised Hack (4.8ghz 2600k) with power saving completely disabled.

Radeon HD6450 for GFX. Have tried with others though (across a range of frame buffers) and performance is same. Using genuine Apple WLAN and Bluetooth, although disabling them doesn't improve performance greatly (disabling WLAN does have an effect however). USB3, Ethernet etc. all disabled. Custom DSDT file.

Audio interface is an SSL Madixtreme. Again, tried with others (including onboard and RME) with similar results; OsX just doesn't perform well for low-latency audio.

[valis]

Originally Posted by Kane

valis

Using an optimised Hack (4.8ghz 2600k) with power saving completely disabled.

Radeon HD6450 for GFX. Have tried with others though (across a range of frame buffers) and performance is same. Using genuine Apple WLAN and Bluetooth, although disabling them doesn't improve performance greatly (disabling WLAN does have an effect however). USB3, Ethernet etc. all disabled. Custom DSDT file.

Audio interface is an SSL Madixtreme. Again, tried with others (including onboard and RME) with similar results; OsX just doesn't perform well for low-latency audio.

I understand but your issues with graphical refreshes causing audible glitching made me immediately think you were using a hack, as OSX's drivers are not very well optimized for modern gpu's to begin with (and they tend to lag months behind Windows more often than not). This goes for both Nvidia & ATI unfortunately. Technically quartzGL shouldn't do this at all as it runs at a lower priority than your audio drivers (lower ring) but even with full QE/CI support enabled it's not uncommong. Fwiw I have great luck with GTX285's in both audio & video tasks but the power consumption when you use a non EFI model (even in a real mac using the 'bootstrap' method off a lower end mac Nvidia card) leaves a lot to be desired.

Also fwiw my RME Multiface II on my 2008 era Xeon has no issues with 32sample buffer setting on tracks in 'live' input monitoring mode. I know that the 2009/2010 models had some pretty severe issues my Mac Pro doesn't have until Apple updated their EFI to account for things noted in a thread here in regards to the Nehelem (and sandy bridge etc) era power saving/turbo features. Per core performance is about half of the current Mac Pro models though, so while it's smooth to its limits it hits the limits much sooner...

[TAFKAT]

Originally Posted by Kane

Yes, this is the way to see performance at different buffer sizes (at least on a single core).

One thing you can do to compare across systems is to limit the system to a single core only (utilities to do this; although some DAWs have the option to disable multi-processing, you still get benefits from the extra cores). That way you can compare Nuendo/Live/Logic etc across a single core.

Things aren't pretty though, performance is WAY below the levels of Win.

Hey Kane,

Not really interested in single core performance as that isn't really reflective of Real World working environments.

I am still a little confused with how the live mode is interacting in regards to this particular benchmark tho.

I haven't had a chance to fire up the session as yet , but if for example the instances of Space Designer were initialised horizontally across the respective tracks i.e Insert 1 - Track 1, Insert 1 - Track 2 , Insert 1 - Track 3, Insert 1 - Track 4 , etc , then they will be load balancing across respective cores/threads

If they are initialised vertically however i.e Track 1, Insert 1-14 as they are configured , then all of those instances will load up on a single core.

We need some clarification on how the instances are being instigated.

If simply placing one track into Live Mode changes the playback buffer dynamic for all tracks , then its a prefect way to do comparative shootouts against the other DAW's using the full multicore/multithreading capabilities of the respective DAW's , it doesn't have to be reserved to a single core.

Or am I getting lost in the mix somewhere ?

[The Beatsmith]

Originally Posted by TAFKAT

Not really interested in single core performance as that isn't really reflective of Real World working environments.

[Kane]

If simply placing one track into Live Mode changes the playback buffer dynamic for all tracks , then its a prefect way to do comparative shootouts against the other DAW's using the full multicore/multithreading capabilities of the respective DAW's , it doesn't have to be reserved to a single core.

Hi again

Unfortunately it doesn't. It only applies to the plugins that you monitor live that this happens. So in the example of track 1 slot fx 1, track 2 slot fx 1, track 3 slot fx 1, these three tracks would be assigned to one core for low-latency purposes.


As far as I am aware, there's no way to run tracks across multiple cores and have them use the buffer setting of the audio interface. You can of course change the process buffer range.

Regardless, some DAWbench figures even at a single value would be useful I think.

[TAFKAT]

Originally Posted by Kane

Hi again

Unfortunately it doesn't. It only applies to the plugins that you monitor live that this happens. So in the example of track 1 slot fx 1, track 2 slot fx 1, track 3 slot fx 1, these three tracks would be assigned to one core for low-latency purposes.


As far as I am aware, there's no way to run tracks across multiple cores and have them use the buffer setting of the audio interface. You can of course change the process buffer range.

Regardless, some DAWbench figures even at a single value would be useful I think.

Hey Kane,

Thanks for the clarification.

That does make things more difficult for comparative purposes, also the behaviour of processing multiple tracks when placed in live input monitoring on a single core is just another one of those odd idiosyncrasies reserved for Logic.

I know not all will agree , as noted by the response already from the previous poster, but realistically single core performance is really only of value to compare scaling of core execution of the newer chip arcitectures , it doesn't really give us a lot of scope in multi-processor / multi-threaded DAW environments where the greater factors are governed by task scheduling and optimisation at the application/OS level.

Looks like Logic will still remain an enigma for me to try and get some comparative low latency head to head data using my benching methodology :-(

I Digress,

Sorry for the diversion..

[speakerfood]

Originally Posted by TAFKAT

Not really interested in single core performance as that isn't really reflective of Real World working environments.

I am still a little confused with how the live mode is interacting in regards to this particular benchmark tho.

I haven't had a chance to fire up the session as yet

Sorry that I didn't describe the session. What I did was route the live track through bus 1. Then bus 1 to bus 2, bus 2 to bus 3, etc. This creates one big signal chain that Logic has to process on one single core.

I created this benchmark because I often play a live instrument (in the studio that is..) in sessions that only get to 25% load per core, but when the signal chain from the live instrument goes active, a single core can get maxed out resulting in plops/cracks etc.

So, still interested in this performance on other macs. BTW, my Mac Pro is from 2008, but upgraded with quad-core processors, with a RME Fireface 800 as audio interface.

[DonCho]

OK...

MBP i7 4x2.0, 4 gig ram 2011 logic 9.1.3 osx 10.7.2 = 77 SD!

MP nehalem 8x2,26 2009 24 gig ram logic 9.1.5 osx 10.6.8 = 48 SD

Will double check the MP tonight.

Double checked and 48 it is on the MacPro

[The Beatsmith]

OP: There is no 2008 model 2.66Ghz mac pro. Can you please clarify which machine you have?

There is a 2.66 Quad from 2006 and you can get a 2009 2.66 Octo too.

The 2008 models come in quad 2.8, or dual quad (Octo) 2.8, 3.0, and 3.2Ghz models.

I'm guessing you have a 2.8ghz Octo from 2008

[speakerfood]

My mistake

it's indeed a Mac Pro 2,66 2006 (not 2008), but upgraded by replacing the processors with two 2,66 Intel Xeon X5355 Quad-Core processors

[The Beatsmith]

My results on an 8 x 2.8Ghz 2008 Mac Pro, 12GB Ram, Logic 9.1.5 64-Bit, OS 10.6.8, SSL MadiXtreme PCI-e:

- 42 instances (3 channel strips) at 512 buffer. Adding the 4th channel strip/43rd instance added a couple of crackles
- 14 instances at 64 buffer. Adding the 2nd channel strip/15th instance added a couple of crackles

I think it would be good to get a chart at 64 buffer too - more relevant for me at least

Arming or un-arming the software instrument made no difference. The last CPU was maxed out for a while and i could add more plugins before there were actually any crackles. OSX Activity monitor didn't show one core maxed out at all, in fact it showed only a small amount of activity spread across cores. I assume that meter is simply wrong.

I run Logic in my studio everyday at 64-buffer and am 100% native (no hardware monitoring). The time i run into issues is when record arming a heavy duty software instrument @ 64 sample buffer, even though when it's not record armed, the plugin might only use a couple of percent of CPU. Kinda crazy. I would certainly like more power per core. I think this machine will last me until mid to end 2012. It's still very capable most of the time.

FWIW i think this test is really important, if not more important than other more popular Logic test. It's obvious that all modern machines are capable DAWS to run hundreds of low to moderate plugins.

It's running at 64 buffer with plugins at the master and record arming a software instrument track like Kontakt/Abbey Road Drums... that separates the men from the boys!

Cheers!

Eddie

[The Beatsmith]

Originally Posted by gdoubleyou

62 Space Designers. interesting.

2GHz quad i7 MBP 4GB RAM OSX10.6.8

Amazing machines!

Here are my results for my 2010 15" MacBook Pro 2.53GHz i5 running Lion 10.7.2 and Logic 9.1.5 64-Bit:

- 43-44 instances at 512 buffer. It was weird, sometimes it was 43, sometimes it was OK at 45. So i guess it's 43.
- 10 or maybe 11 instances at 64 buffer. I'll give it the benefit of the doubt and say that it was 11

Surprisingly, while the 512 buffer performance just beat my Mac Pro, the 64 samples buffer was noticeably worse. Phewph, i would have had to sell everything and save up for a 6/12 core to try to over-compensate

I messed around with the core settings - 2, 4, and automatic. None of them make any difference - somewhat obvious/expected in this test, but to be honest it doesn't change how many can be run across cores either. It's not 'real' hyper threading, i guess. I think these 2010 MBP's kinda suck, overall.

Cheers

[The Beatsmith]

I would also be interested to know if any other DAWS handle single core processing better than Logic?

[speakerfood]

Thanks for the results so far, I will be updating the graph soon. And will include the results from the buffer at 64 samples test.

Also interested in handling of live instruments by other DAW's, especially other DAW's that run under OSX.

[Kane]

Haven't done detailed follow-up testing (TAKAT you might want to put something together?) but though it useful to post this:

I have argued before (usually against dissenting opinion) of the effect that memory bandwidth has on DAW use. This test (while synthetic and not representative of real-world use) would appear (although I haven't done detailed testing) to be a good example of this.

Here's are my figures with RAM @ 1866mhz (i7 2600k @ stock speed), having clean-booted each time and ran Logic with no other apps running:

32 Buffer: 23 instances
64 Buffer: 37 instances

Now with my RAM set to 800mhz:

32 Buffer: 11 instances
64 Buffer: 18 instances

In other words, the results appear 100% bandwidth limited. On the same CPU, doubling bandwidth exactly doubles performance. I couldn't believe how closely this correlates but it does (and I've repeated the test several times).

This would appear to be an excellent project to compare the effects of bandwidth and RAM latency although more testing is needed.

[speakerfood]

Didn't know that speed of RAM is so important. I think doing this test with the buffer set to 64 samples is indeed a better real-world test.

Here are the results on a Mac Mini Server 2Ghz i7 with 1333 Mhz DDR3 RAM:

buffer @ 64 (4,3 ms roundtrip latency): 13 instances

Can the others in this thread who did the test do it again with the buffer at 64? Then I can update the graph.

[gdoubleyou]

Originally Posted by gdoubleyou

62 Space Designers. interesting.

2GHz quad i7 MBP 4GB RAM OSX10.6.8

Just downloaded 9.1.6 and ran the test, now able to get 70 Space designers!

Will have to try the multicore benchmark after work.

[The Beatsmith]

Really interesting. I can't try it out until monday - will post some results then. But that's a promising improvement!

[luke p]

I don't have Logic 9 yet, so can't post reliable numbers. What really interests me personally here is the original's poster benchmarks results for his upgraded 2006 2.66 mac pro (upgraded to 2 x 2.66 intel x5355). I have a 2006 mac pro, dual 3.0 5160 and am seriously considering upgrading to x5355 to benefit from 8 cores. On the other hand, if single core action is just as bad on the 5355, I'll scrap my upgrade plans, since the single core spikes are the very reason I need to upgrade. So... speakerfood, could you please post your results?
Thanks.

[TNM]

Originally Posted by gdoubleyou

62 Space Designers. interesting.

2GHz quad i7 MBP 4GB RAM OSX10.6.8

Interesting ok....

you can run 62 at 64 buffer on a 2gh mac book pro with no crackles, and i can run between 22-24 and i have a 2.2 ghz quad mac pro.

i call complete bs and that you did not change your buffer to 64 samples as per instructions. No way your machine cna do almost 3 times mine when mine is a more powerful machine!

I can do 70 at 256 buffer