Has anyone tried Mumble for this? It's very low latency but I can't find exactly how low the latency is. It ofcourse depends also on your internet connection and other settings but the base latency that comes from buffering the sound before sending. Mumble also has lots of settings for sound quality and different sound formats so might work for music if you try all the settings.
Mumble has a setting for the audio buffer size and in fact they make you set it during initial configuration. It works great, has low latency and doesn't use much bandwidth (I hosted a server on a 1Mbps DSL connection for several people back in the days).
