Difference: Your large program uses $SIG{ PIPE } = "IGNORE";.

Solution: Add local $SIG{ PIPE } = "DEFAULT"; in scope of your open. Alternatively, you could redirect zcat's STDERR to /dev/null.

---

Take a look at this:

seq 1000000000 | gzip | zcat | head
[download]

It should take a long time to generate and unzip that stream, but it doesn't.

$ time ( seq 1000000000 | gzip | zcat | head )
1
2
3
4
5
6
7
8
9
10

real    0m0.051s
user    0m0.056s
sys     0m0.005s
[download]

This is what happens:

1. Once head has printed its ten lines, it exits.
2. This breaks the pipe from zcat to head.
3. The next time zcat attempts to write to the pipe, SIGPIPE is sent to it.
4. Upon receiving SIGPIPE, zcat is killed.
5. gzip is similarly killed by SIGPIPE.
6. seq is similarly killed by SIGPIPE.

The above is what happens in your standalone script.

In your larger program, you appear to have used $SIG{ PIPE } = "IGNORE";. It's still just as quick, but the error message now appears.

$ seq 1000000000 | gzip | perl -e'$SIG{ PIPE } = "IGNORE"; exec("zcat"
+)' | head
1
2
3
4
5
6
7
8
9
10

gzip: stdout: Broken pipe
[download]

This is what happens now:

1. zcat inherits its parent's disposition of ignoring SIGPIPE.
2. Once head has printed its ten lines, it exits.
3. This breaks the pipe from zcat to head.
4. The next time zcat attempts to write to the pipe, SIGPIPE is sent to it.
5. zcat ignores the SIGPIPE.
6. The write call returns error EPIPE.
7. zcat exits with a message as a result of the error. (zcat is an alias for gzip, which is why the message says gzip.)
8. gzip is killed by SIGPIPE.
9. seq is killed by SIGPIPE.

Add local $SIG{ PIPE } = "DEFAULT"; in scope of your open.