Have you ever done a true A/B comparison of an mp3 file and a wave file? I just got through doing one and it is pretty dramatic how much better wave files sound. The two files that I was comparing where from the cd by The Cowboy Junkies, Trinity Sessions. The mp3 was at 320 kbps and the wave was 16 bit/44.1
The main difference I noticed between the two files was in the high frequency information. The wave file had so much more air and presence to it. Even in the mids and upper mids, it just felt more full. When I switched back to the mp3 it was quite noticeable that the song just became sort of hollow sounding. It really was pretty dramatic. I suppose it would depend on the material and your monitors or headphones but on my system it is really obvious.
I guess my advice is, if your iTunes library is full of mp3's go out and by cd's and a 500 gig, (Or more : ) hard drive...