See the section "Streaming Output" in my node here, in particular IPC::Run might be useful to you.

for mysterious reasons _sometimes_ ... it waits forever even when $system_command finishes

It might be useful to debug that, since it's possible that it's the fault of the program you're calling (suffering from buffering maybe?), and/or that the other modules will suffer from exactly the same problem.