Loud buzz sound when listening to some music

Okay, I just downloaded cave story and was about to play it, but I got a problem with the sounds. The musics in the game tend cause a loud buzz sound in my speakers. I don't really understand why. The problem is not random, i.e. same songs will cause the buzz sound at exact same place (for example, when listening the background sounds with the listening app), however I doubt it's the cave story that's corrupted, and it's hardware/software problem.

Actually, similiar problems started when I installed windows xp on my comp a while ago. I thought it was just broken integrated sound card, since it was pretty crappy, and it couldn't even start when using official drivers (MS's drivers worked, though). So, I replaced the sound card (and speakers at the same time) and problems were gone from all games that previously had them. I thought I had gotten rid of the problem, but apparently I hadn't, since cave story is now making those loud buzz noises. (I now have Audigy SE soundcard).

Even any ideas what it could be would be nice, since I'm completely clueless. On interesting note, if I stop the music in cave story music listener when the speakers make the buzz noise, they will continue make the buzz noise until I make some other noise with the speakers...

On an interesting note, I was able to record the buzz by recording line out just with software on my computer. Here's a sample (so.. software/codec problem?):

http://koti.mbnet.fi/rithiur/temp/buzzeh.mp3

Sounds like a problem with whatever in Windows is used to play old school synth MIDI files. Download some .mid files from Limewire or somewhere and see if they buzz.