in reply to Re: Replace a Hex Value in Column
in thread Replace a Hex Value in Column

Thanks Haukex ! But I am dealing with piped traffic so STDIN will be the input and lines should be written one by one, your approach will get results in offline work.

Replies are listed 'Best First'.
Re^3: Replace a Hex Value in Column
by haukex (Archbishop) on Feb 03, 2018 at 09:12 UTC
    But I am dealing with piped traffic so STDIN will be the input and lines should be written one by one

    STDIN and STDOUT are filehandles, like $ifh and $ofh in my code. Here, I'm using select to get STDOUT and ARGV to get Perl's magic command-line argument filehandle, which reads from STDIN when no files are specified on the command line. I've assumed you don't want some of the binary data from here, but note that this still doesn't strip all of it out.

    use warnings; use strict; use Text::CSV; # also install Text::CSV_XS for speed my $csv = Text::CSV->new({ binary=>1, auto_diag=>2, escape_char=>'\\', eol=>$/, quote_char=>'"', sep_char=>";", always_quote=>1 }); while ( my $row = $csv->getline(*ARGV) ) { $row->[5] = pack '(H2)*', split /:/, $row->[5]; $row->[5] =~ s/\A\x19\x68\x09\x24\x00+|\x00+\z//g; $csv->print(select, $row); } $csv->eof or $csv->error_diag;
      tr might be faster than split:
      #!/usr/bin/perl use warnings; use strict; use Syntax::Construct qw( /r ); my $s = '19:68:09:24:00:5b:00:7e:53:53:4c:2e:47:6f:6f:67:6c:65:00:00:0 +0:00:00:00'; sub ch { my $s = $_[0]; pack 'H*', $s =~ tr/://dr } sub hx { my $s = $_[0]; pack '(H2)*', split /:/, $s } use Test::More tests => 1; is ch($s), hx($s), 'same'; use Benchmark qw{ cmpthese }; cmpthese(-2, { ch => sub { ch($s) }, hx => sub { hx($s) }, });
      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
      Thanks Guys! tr introduced difference almost 500%, I managed to clean up the binaries, below is the final code:
      #!/usr/bin/env perl use warnings; use strict; use Text::CSV_XS; # also install Text::CSV_XS for speed my $csv = Text::CSV_XS->new({ binary=>1, auto_diag=>2, escape_char=>'\ +\', eol=>$/, quote_char=>'"', sep_char=>";", always_quote=>1 }); while ( my $row = $csv->getline(*ARGV) ) { $row->[5] = pack 'H*',$row->[5]=~ tr/://dr; $row->[5] =~ s/\A\x19\x68\x09\x24\x00+|\x00+\z//g; $row->[5] =~ s/[^\w:]|(?<!:):(?!:)//g; $csv->print(select, $row); } $csv->eof or $csv->error_diag;
      Yup that fixed it, those binaries are much less also! excuse my comment before its just when I saw you opening 2 files I just turned away immediately, I just a have strange error on the exist but output is quiet fine :
      #!/usr/bin/env perl use warnings; use strict; use Text::CSV_XS; # also install Text::CSV_XS for speed my $csv = Text::CSV_XS->new({ binary=>1, auto_diag=>2, escape_char=>'\ +\', eol=>$/, quote_char=>'"', sep_char=>";", always_quote=>1 }); while ( my $row = $csv->getline(*ARGV) ) { $row->[5] = pack '(H2)*', split /:/, $row->[5]; $row->[5] =~ s/\A\x19\x68\x09\x24\x00+|\x00+\z//g; $csv->print(select, $row); } $csv->eof or $csv->error_diag;

      OUTPUT

      "Feb 2, 2018 18:06:49.993152000 ";"172.16.13.158";"1169";"151.101.112 +.102";"443";"[\0vSSL.Slack" "Feb 2, 2018 18:06:49.993349000 ";"172.16.16.75";"26504";"31.13.86.8" +;"443";"[\0wSSL.Facebook" "Feb 2, 2018 18:06:49.993467000 ";"172.16.13.158";"28802";"151.101.11 +2.102";"443";"[\0vSSL.Slack" "Feb 2, 2018 18:06:49.996467000 ";"192.167.170.118";"48620";"216.58.2 +10.206";"443";"&#65533;\0|QUIC.YouTube" "Feb 2, 2018 18:06:49.997690000 ";"172.16.15.155";"33318";"172.217.18 +.46";"443";"[\0&#65533;SSL.PlayStore" "Feb 2, 2018 18:06:49.998318000 ";"172.16.23.85";"26617";"31.13.86.51 +";"443";"[\0SL.WhatsApp" "Feb 2, 2018 18:06:49.998889000 ";"172.16.16.75";"26497";"172.217.18. +42";"443";"[\0&#65533;SSL.GoogleServi" "Feb 2, 2018 18:06:49.999517000 ";"172.16.14.116";"27917";"216.58.209 +.36";"443";"[\0~SSL.Google" "Feb 2, 2018 18:06:50.001838000 ";"172.16.14.163";"26396";"172.217.18 +.42";"443";"[\0&#65533;SSL.GoogleServi" "Feb 2, 2018 18:06:50.002442000 ";"192.167.175.103";"1169";"107.154.5 +0.239";"21120";"Unknown" Use of uninitialized value in split at ./dpi.pl line 9, <> line 11. "";;;;;""
      ~
        I just a have strange error on the exist but output is quiet fine
        Use of uninitialized value in split at ./dpi.pl line 9, <> line 11. "";;;;;""

        That's not an error, just a warning, about the fact that $row->[5] is undef. From the output I suspect you've just got an empty line at the end of your input (line 11, as the message says).