I agree with Kats that playing to a click SHOULD be about the music and not about poor musicianship. But often it IS in fact about poor musicianship.
I certainly don't think "perfect" time (btw, Bob O.'s comments about "perfect" time are spot on! But again, I'm not talking about great players in this post, unfortunately) is more important than the natural vibe of a songs progress, slight tempo changes and all.
But MANY times when I'm recording a young band and there's not much time at all in the budget, a click track will in fact make the drummer play more consistently and often times simpler as well (which means more consistency at the skill level these guys are at). At that point, the constrictive use of a click in the drummers cans actually frees up the band to sound more assured in their parts as they're no longer trying to ride a bucking, runaway horse, if you'll pardon the cowboy mentality (we just got the box set of the Sergio Leone spaghetti westerns on DVD at the studio... now THERE's some essential gear!).
Anyways it certainly helps things in a hurry. I've had many of these less-than-brilliant drummers say "wow, i didn't know i could play that consistently" after a few gos with the click. But even with this situation, there will be some of the songs that work best off.
personally, i'm not the biggest fan of clicks in most cases if i'm playing with a good drummer. btw/ i recorded an album for bernard purdie and man, his tempo varied like crazy! but i've never heard anyone complain of this about his drumming because i'm sure that if you're playing with Bernard, you just play to the MAN and all will be right.
