I just tested a "test" application which outputs raw 16-bit unsigned data to SigmaDelta (stored in SPI flash). This data was the left-channel of Helloween's "Handful of Pain", clipped to a few seconds.
Then I connected a simple low-pass and DC filter to output, and connected one headphone.
And I heard the music very well :) Next time I'll try with a SD card, flash is too small for raw music.
Output frequency was 44100Hz, like CD. SigmaDelta outputs at 50MHz. That means an oversampling of 1133. Should be enough for any filter to settle down.