Sure, the conversion rate for one client connection would probably outstrip download times. If you have only one client at once, and do nothing else on the server, then it's doable. But... do you really expect only a single client will ever connect at one time, and does the box do nothing else while the connection's in play?

Most boxes do other things, and potentially repeatedly converting the same track over and over again is a waste of time which could be used by other processes. And if you've got the potential for multiple connections simultaneously, it doesn't take that long before you'll find that either the conversion CPU requirements or the disk I/O for the (large) raw data is the bottleneck.