in reply to Re: Filtering passwords from the output of a script
in thread Filtering passwords from the output of a script

Your first alternative would make the password string available to all logged-in users via the "ps" command.

Your second alternative will still be stuck with line buffering -- even with $| set to non-zero everywhere on every file handle, the standard print function and diamond read operators will operate only in line-buffered mode ($| merely makes sure that consecutive lines are not buffered into chunks of 4KB or 8KB or whatever the pipeline buffer size happens to be).

  • Comment on Re^2: Filtering passwords from the output of a script

Replies are listed 'Best First'.
Re^3: Filtering passwords from the output of a script
by ikegami (Patriarch) on Nov 29, 2006 at 05:07 UTC

    Your first alternative would make the password string available to all logged-in users via the "ps" command.

    Yeah, that's why I put the second alternative. I should have mentioned it.

    Your second alternative will still be stuck with line buffering

    Oh! right! Here's a version that doesn't wait for a whole line to be read before

    #!/usr/bin/perl # censor_passwd use warnings; use strict; my $passwd = 'hide_me'; my $replacement = 'removed'; # Used to check if the start of the password # is at the end of the read data. The returned # regexp will match 0 characters if not. # For efficiency, it expects the data to be # "reverse"d. my $partial_re = do { my $r = reduce { "(?:$a$b)?" } '', reverse map quotemeta, map /(.)/g, $passwd; qr/$r/ }; binmode(STDIN); binmode(STDOUT); $| = 1; my $buf = ''; for (;;) { my $rv = sysread(STDIN, $buf, 4096, length($buf)); defined $rv or die("Unable to read from STDIN: $!\n"); $rv or last; for (;;) { my $pos = index($buf, $passwd); last if $pos < 0; substr($buf, $pos, length($passwd), $replacement); print(substr($buf, 0, $pos+length($replacement), '')); } reverse($buf) =~ /^$partial_re/; print(substr($buf, 0, length($buf)-$+[0], '')); } print($buf);

    Points of interest:

    • It performs minimal input buffering. It outputs everything as soon as it receives it, without waiting for lines or words to be completed. It only buffers when the data that was read ends with the start of the password.

    • It is very efficient. It reads a block of data at a time, if more than a byte of data is available. It avoids using /...\z/ and even slower /^.*?.../ by using reverse. It avoids using captures. A major loop was replaced with a precompiled regexp.

    ( Caveat: What if "hide" and "_me" are received seperatly? Code updated to address this issue. )

      Caveat: What if "hide" and "_me" are received seperatly?

      That, together with the requirement for the stream to be as unbuffered as possible, is why it would be wrong to use "4096" (or any value greater than "1") on the sysread call.

        Not so. Unlike read (which waits for all 4096 characters to be read in) and <FILE> (which waits for a whole line to be read in), sysread returns as soon as possible. For example,
        perl -e "$|=1; print 'test1'; sleep 1; print 'test2'" | perl -e "sysre +ad(STDIN, $buf='', 1000); print(qq{[$buf]})"

        outputs

        [test1]

        If what you claimed is true, it would output [test1test2].

        ( And I'm working on the caveat. And I've addressed the caveat in an update. )