So... Which one do you use?

I tend to use "Linear phase" for drum mics and multiple bass signals and many times on also parallel stuff (and on the bus) but I noticed just lately that if it is NOT set on max quality, the default settings make the entire signal sound different, smeared stereo image, less detail, not in a good way. Not by a huge margin, but the degradation is noticeble to my ears... Then if you raise it to max settings, the quality is 1:1 but the cpu hit is large and my computer starts lagging and being a pain after multiple instances...

I tend to use "natural phase" in general, and "low latency" on more peripheral elements of the mix... I could probably load 3000 instances like that... But not on Linear phase/max.

I went a bit paranoid about it especially as it has been my main work-horse for the last couple of years, but i am now worried it might have been
a Trojan-horse instead eating up the fidelity of my stuff while lurking in the dark...

I need to do a proper extended listening comparisons now of every mode, once I have time to spare, but as a reference, I was interested, how do most people set it? Do you even hear a big difference like I do? And what setting do you prefer for what application (especially when using linear phase)?

Thanks in advance!