The only way I know of is to implement 'NTFS disk quotas', there are plenty of hits on Google. Your piping system should work OK - Windows implements anonymous pipes at the Win32 API level. If the app is writing relatively short text records to stdout with "\r\n" line terminators then I don't see an issue. From the MSDN: "If the anonymous read pipe handle has been closed and WriteFile attempts to write using the corresponding anonymous write pipe handle, the function returns FALSE and GetLastError returns ERROR_BROKEN_PIPE." It is bound to slow things up, but by how much I couldn't say.
Comment on Re: Windows-specific: Limiting output of created files