You can buy track by track on Bandcamp (FLAC) and Google Play Music (320 MP3), and old Amazon as well (not sure about Amazon HD). A few artists/labels disable it, but that’s rare.
That would be my case, where streaming is definitely not cheaper in the long run: I rarely find new artists that are worth my time, so paying constantly for access to a huge database only to ignore 95% of it would be absolutely more expensive for me, not to mention it limits me to listening only when I have working Internet. It’s DRM-free files all the way for me, TYVM.