The excellent Whitlock chapter will give you a detailed engineering perspective on transformer characteristics.
The simple answer is that transformers can be used for many impedance matching and single-ended/balanced circuit conversions, or even to "split" a signal to allow driving multiple loads. Depending on design, some transformers can change the sound of a audio signal by adding both even and odd-order harmonic distortion, alteration of frequency response, signal "overshoot" and resonance ("ringing") and saturation (distortion that changes with the signal level). Some transformers are designed to be as audibly "transparent" as possible so as not
to change the sound, while some have clearly audible effects
on the signal.
Including a transformer or transformers may audibly alter the signal, and with modern electronics the other functions of impedance matching and balancing/unbalancing can usually be achieved with other components or circuit design. Therefore the use of a transformer can often be considered "optional" as is the case in some mics and mic preamplifiers.
The alteration to the signal (and therefore the sound) may or may not be considered "pleasing" to the listener. In many cases it is a subjective judgement. In a recording situation where it's important to achieve the most accurate treatment of the signal, such as recording acoustic, classical instruments, it's less common for transformers to be used. For recording a pop singer or an amplified guitar, the harmonic distortion and low frequency saturation effects created by some transformers in the signal path may be considered an advantage.
"Is it better to have them?" That depends on what kind of a sound one is after, and how you are trying to "shape" the sound, as well as personal "taste".
It's like asking if it's "better" to add cilantro seasoning to a meal. Some people love it. some hate it, some don't care, and some can't tell the difference.