Replace a Hex Value in Column

ktham33n has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Replace a Hex Value in Column by choroba (Cardinal) on Feb 02, 2018 at 12:19 UTC
Where do you get "BitTorrent" from? I tried to replace the hex values by characters of the same ascii number, but the result is different: `$ perl -MText::CSV_XS=csv -we 'csv( in => ARGV, sep_char => ";", always_quote => 1, after_parse => sub { $_[1][-1] = join "", map chr h +ex, split /:/, $_[1][-1] } )' file` [download] Output: `"Feb 2, 2018 10:50:49.996148000 ";"172.16.21.120";"54859";"196.58.210. +193";"9889";"h $[~SSL.Google"` [download] with char 25 instead of the . ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7*2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l] [select]
Re^2: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 07:58 UTC
Thanks choroba ! You hit the spot in the sense of performance and simplicity, The speed of XS is just wow never new about that, Sorry for the BitTorrent thing I paste a wrong packet yes indeed it was an SSL.Google and the First part of the Hex 19:68:09:24:00 is just a magic and full hex is always fixed 24 pair,but output should be better below is your code in action: `#!/bin/dash perl -MText::CSV_XS=csv -we 'csv( in => *ARGV, sep_char => ";", always_quote => 1, after_parse => sub { $_[1][-1] = join "", map chr h +ex, split /:/, $_[1][-1] } )'` [download] INPUT "Feb 2, 2018 18:06:49.993349000 ";"172.16.16.75";"26504";"31.13.86.8" +;"443";"19:68:09:24:00:5b:00:77:53:53:4c:2e:46:61:63:65:62:6f:6f:6b:0 +0:00:00:00" "Feb 2, 2018 18:06:49.993467000 ";"172.16.13.158";"28802";"151.101.11 +2.102";"443";"19:68:09:24:00:5b:00:76:53:53:4c:2e:53:6c:61:63:6b:00:0 +0:00:00:00:00:00" "Feb 2, 2018 18:06:49.996467000 ";"192.167.170.118";"48620";"216.58.2 +10.206";"443";"19:68:09:24:00:bc:00:7c:51:55:49:43:2e:59:6f:75:54:75: +62:65:00:00:00:00" "Feb 2, 2018 18:06:49.997690000 ";"172.16.15.155";"33318";"172.217.18 +.46";"443";"19:68:09:24:00:5b:00:e4:53:53:4c:2e:50:6c:61:79:53:74:6f: +72:65:00:00:00" "Feb 2, 2018 18:06:49.998318000 ";"172.16.23.85";"26617";"31.13.86.51 +";"443";"19:68:09:24:00:5b:00:8e:53:53:4c:2e:57:68:61:74:73:41:70:70: +00:00:00:00" "Feb 2, 2018 18:06:49.998889000 ";"172.16.16.75";"26497";"172.217.18. +42";"443";"19:68:09:24:00:5b:00:ef:53:53:4c:2e:47:6f:6f:67:6c:65:53:6 +5:72:76:69:00" "Feb 2, 2018 18:06:49.999517000 ";"172.16.14.116";"27917";"216.58.209 +.36";"443";"19:68:09:24:00:5b:00:7e:53:53:4c:2e:47:6f:6f:67:6c:65:00: +00:00:00:00:00" "Feb 2, 2018 18:06:50.001838000 ";"172.16.14.163";"26396";"172.217.18 +.42";"443";"19:68:09:24:00:5b:00:ef:53:53:4c:2e:47:6f:6f:67:6c:65:53: +65:72:76:69:00" "Feb 2, 2018 18:06:50.002442000 ";"192.167.175.103";"1169";"107.154.5 +0.239";"21120";"19:68:09:24:00:00:00:00:55:6e:6b:6e:6f:77:6e:00:00:00 +:00:00:00:00:00:00" [download] OUTPUT "Feb 2, 2018 18:06:49.993349000 ";"172.16.16.75";"26504";"31.13.86.8" +;"443";"h $"0["0wSSL.Facebook"0"0"0"0" "Feb 2, 2018 18:06:49.993467000 ";"172.16.13.158";"28802";"151.101.11 +2.102";"443";"h $"0["0vSSL.Slack"0"0"0"0"0"0"0" "Feb 2, 2018 18:06:49.996467000 ";"192.167.170.118";"48620";"216.58.2 +10.206";"443";"h $"0�"0\|QUIC.YouTube"0"0"0"0" "Feb 2, 2018 18:06:49.997690000 ";"172.16.15.155";"33318";"172.217.18 +.46";"443";"h $"0["0�SSL.PlayStore"0"0"0" "Feb 2, 2018 18:06:49.998318000 ";"172.16.23.85";"26617";"31.13.86.51 +";"443";"h $"0["0SL.WhatsApp"0"0"0"0" "Feb 2, 2018 18:06:49.998889000 ";"172.16.16.75";"26497";"172.217.18. +42";"443";"h $"0["0�SSL.GoogleServi"0" "Feb 2, 2018 18:06:49.999517000 ";"172.16.14.116";"27917";"216.58.209 +.36";"443";"h $"0["0~SSL.Google"0"0"0"0"0"0" "Feb 2, 2018 18:06:50.001838000 ";"172.16.14.163";"26396";"172.217.18 +.42";"443";"h $"0["0�SSL.GoogleServi"0" "Feb 2, 2018 18:06:50.002442000 ";"192.167.175.103";"1169";"107.154.5 +0.239";"21120";"h $"0"0"0"0Unknown"0"0"0"0"0"0"0"0"0" [download]	[reply] [d/l] [select]
Re: Replace a Hex Value in Column by haukex (Archbishop) on Feb 02, 2018 at 12:21 UTC
#!/usr/bin/env perl use warnings; use strict; use Text::CSV; # also install Text::CSV_XS for speed my $infilename = 'in.csv'; my $outfilename = 'out.csv'; my $csv = Text::CSV->new({ binary=>1, auto_diag=>2, eol=>$/, quote_char=>'"', sep_char=>";", always_quote=>1 }); open my $ifh, '<', $infilename or die "$infilename: $!"; open my $ofh, '>', $outfilename or die "$outfilename: $!"; while ( my $row = $csv->getline($ifh) ) { $row->[5] = "BitTorrent"; $csv->print($ofh, $row); } $csv->eof or $csv->error_diag; close $ifh; close $ofh; [download]	[reply] [d/l]
Re^2: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 08:08 UTC
Thanks Haukex ! But I am dealing with piped traffic so STDIN will be the input and lines should be written one by one, your approach will get results in offline work.	[reply]
Re^3: Replace a Hex Value in Column by haukex (Archbishop) on Feb 03, 2018 at 09:12 UTC
But I am dealing with piped traffic so STDIN will be the input and lines should be written one by one `STDIN` and `STDOUT` are filehandles, like `$ifh` and `$ofh` in my code. Here, I'm using select to get `STDOUT` and ARGV to get Perl's magic command-line argument filehandle, which reads from `STDIN` when no files are specified on the command line. I've assumed you don't want some of the binary data from here, but note that this still doesn't strip all of it out. `use warnings; use strict; use Text::CSV; # also install Text::CSV_XS for speed my $csv = Text::CSV->new({ binary=>1, auto_diag=>2, escape_char=>'\\', eol=>$/, quote_char=>'"', sep_char=>";", always_quote=>1 }); while ( my $row = $csv->getline(ARGV) ) { $row->[5] = pack '(H2)', split /:/, $row->[5]; $row->[5] =~ s/\A\x19\x68\x09\x24\x00+\|\x00+\z//g; $csv->print(select, $row); } $csv->eof or $csv->error_diag;` [download]	[reply] [d/l] [select]
Re^4: Replace a Hex Value in Column by choroba (Cardinal) on Feb 03, 2018 at 10:19 UTC
Re^4: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 10:54 UTC
Re^4: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 09:39 UTC
Re^5: Replace a Hex Value in Column by haukex (Archbishop) on Feb 03, 2018 at 09:51 UTC
Re: Replace a Hex Value in Column by hippo (Archbishop) on Feb 02, 2018 at 11:22 UTC
With which specific part of the task are you having problems? Please show your code as it stands. An SSCCE would be best.	[reply]
Re^2: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 08:02 UTC
I think we will take it from choroba approach mine was quite different I started from here by snatching out the hex and got stuck there: `perl -pe 's/([[:xdigit:]]{1,2}:){23}[[:xdigit:]]{1,2}/$HEX/g;' testing +.csv` [download]	[reply] [d/l]
Re: Replace a Hex Value in Column by NetWallah (Canon) on Feb 02, 2018 at 17:30 UTC
`C:>SET CSV="Feb 2, 2018 10:50:49.996148000 ";"172.16.21.120";"54859";" +196.58.210.193";"9889";"19:68:09:24:00:5b:00:7e:53:53:4c:2e:47:6f:6f: +67:6c:65:00:00:00:00:00:00" C:>echo %CSV% \| perl -F; -ane "$F[5]=qq\|\"BitTorrent\"\|;print join q\| +;\|,@F" "Feb 2, 2018 10:50:49.996148000 ";"172.16.21.120";"54859";"196.58.210. +193";"9889";"BitTorrent"` [download] For Linux, remove "SET", use $CSV for %CSV%, and use single quotes for the perl command. Is a computer language with goto's totally Wirth-less?	[reply] [d/l]
Re^2: Replace a Hex Value in Column by BillKSmith (Monsignor) on Feb 02, 2018 at 19:57 UTC
Great way to meet the specification. Do you really think that this is what's intended? Bill	[reply]
Re^3: Replace a Hex Value in Column by NetWallah (Canon) on Feb 04, 2018 at 03:39 UTC
Sorry about that - it was not intended to be cynical. I was genuinely confused about where the OP was getting "BitTorrent" from. Is a computer language with goto's totally Wirth-less?	[reply]
Re^2: Replace a Hex Value in Column by ktham33n (Novice) on Feb 03, 2018 at 08:04 UTC
Thanks Bill,but that was out of scope Hexs are dynamic.	[reply]