Ah, I've actually borrowed one of those Olympus recorders from my office on occasion to do trial recordings when scoping out new halls. They actually don't do a bad job, but you'll get *much* better results from any handheld that's actually designed for music recording.
The better handhelds are designed to handle anything from a solo guitar to a full-on rock and roll band, although you'll need to get the hang of setting record levels. In other words, you generally can't just set the thing down and hit record and have the machine do all the work - you'll need to manually set the sensitivity of the recorder at an appropriate level for the room you're in, the piece you're singing, the distance from the mic, etc.
Fortunately, since you're doing a lot of your work in the same space, you should be able to find a level setting that works well and just use that as your default. When you go out to record a recital or something, you'll need to do a little testing and watch the meters in order to set the level, but if you have any interest at all in the recording process (and it sounds like you do), you'll get a feel for this. With the quality of today's digital recording, level-setting is a LOT more forgiving than it used to be.
All that said, you might look at the Olympus LS-10 or Tascam DR-1, which would both be around $300....there's also the Zoom H4n. You can look for opinions here, and there's also a lot of useful info at this dealer site, including audio samples of all the different recorders.
Portable Recording Solutions for People Who Care About Sound. WingfieldAudio.com.
This is a little sacreligious here, but don't be afraid to shop on the basis of ease of use as well as sound quality. Anything in this price range will be passable sound-wise for what you're doing, and a user interface that you find easy to work with will help you get the most out of the recorder every time you use it and avoid dumb errors (though you will probably, like all of us, make a few of those along the way as you're getting the hang of your equipment and the process!).
Good luck, with the purchase and your studies!