in reply to Re: Building a dynamic array or some other method?
in thread Building a dynamic array or some other method?

Sample CSV:
File Server,Access Path,Current Permissions,Logon Name,Inherited From +Folders,Flags,User/Group,Classification Results,Classification Result +s by Category (Including Nested),Total Hit Count 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,@FOO NOW Onsite Support,\Common,This folder only, +Pathway12.My.Corp.com\@FOO NOW Onsite Support,IRS Data (1/1),PII (1), +1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,Administrators,\Common,This folder only,10.15.106 +.71\Administrators,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,Creator Owner,\Common,This folder only,Abstract\C +reator Owner,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,FP NOW BMG FSE NTFS Admins,\Common,This folder on +ly,Pathway12.My.Corp.com\FP NOW BMG FSE NTFS Admins,IRS Data (1/1),PI +I (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,ClusterSvcDIR,\Common,This folder only,Pathway12. +My.Corp.com\ClusterSvcDIR,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,SYSTEM,\Common,This folder only,Abstract\SYSTEM,I +RS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,MiJim,<not inherited>,This folder only,"Pathway12 +.My.Corp.com\Michaels, Jim@My",IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,MRWX,@FP DIR BMG,\Common,This folder only,Pathway12.My. +Corp.com\@FP DIR BMG,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,&CDAdmin,\Common,This folder only,Pathway12.My.Corp. +com\&CDAdmin,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,@FOO DSMS Admins,\Common,This folder only,Pathway12. +My.Corp.com\@FOO DSMS Admins,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,FOO BMG FS Support,\Common,This folder only,Pathway1 +2.My.Corp.com\FOO BMG FS Support,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,DPeterso,\Common,This folder only,"Pathway12.My.Corp +.com\Peterson, Dan@My",IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,FP BMG IMG Read Access,\Common,This folder only,Path +way12.My.Corp.com\FP BMG IMG Read Access,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/My-B8245.pdf,FMRWX,@FOO NOW Onsite Support,\Com +mon,This folder only,Pathway12.My.Corp.com\@FOO NOW Onsite Support,IR +S Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/My-B8245.pdf,FMRWX,Administrators,\Common,This +folder only,10.15.106.71\Administrators,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/My-B8245.pdf,FMRWX,Creator Owner,\Common,This f +older only,Abstract\Creator Owner,IRS Data (1/1),PII (1),1<br><br>
Sample output (differences in permissions for each file, with users concatenated having the same permissions)
(when someone shows "<Not Inherited>", I'll put a "(*)" in as an identifier)
File Server,Access Path,Current Permissions,Logon Name,Inherited From +Folders,Flags,User/Group,Classification Results,Classification Result +s by Category (Including Nested),Total Hit Count 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,FMRWX,"@FOO NOW Onsite Support,Administrators,Creator O +wner,FP NOW BMG FSE NTFS Admins,ClusterSvcDIR,SYSTEM,MiJim(*)",\Commo +n,This folder only,Pathway12.My.Corp.com\@FOO NOW Onsite Support,IRS +Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,MRWX,@FP DIR BMG,\Common,This folder only,Pathway12.My. +Corp.com\@FP DIR BMG,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/Axidome Quote My Corp 2020211 KnowBe4 2-yr Rene +wal FINAL.pdf,RX,"&CDAdmin,@FOO DSMS Admins,FOO BMG FS Support,DPeter +so,FP BMG IMG Read Access",\Common,This folder only,Pathway12.My.Corp +.com\&CDAdmin,IRS Data (1/1),PII (1),1 10.15.106.71,/Common/Awareness and Training/KnowBe4/2020- KnowBe4 Subs +cription Renewal Docs/My-B8245.pdf,FMRWX,"@FOO NOW Onsite Support,Adm +inistrators,Creator Owner",\Common,This folder only,Pathway12.My.Corp +.com\@FOO NOW Onsite Support,IRS Data (1/1),PII (1),1
Code: it shows lots of iterative notes
#!/usr/bin/perl use NetAddr::IP; #used for CIDR use Getopt::Long; # handle command line arguments/flags use File::Basename; # use strict; # Okay, I'm gonna take some heat for this. # use warnings 'all'; # Yeah, this too. use Text::CSV qw( csv ); # see https://metacpan.org/pod/Text::sort { $ +a <=> $b } keys %hash;CSV for info use open ":std", ":encoding(UTF-8)"; use Data::Dumper; # use print Dumper($scalar_var); to see data structu +res or objects $command = basename($0); sub help { print "\n"; print "perl $command -r report# report_name Use perl $command -l t +o list supported report formats\n\n"; print "example: perl $command -r 4b May24.csv\n\n"; print "\nUSAGE:\n"; print " -h -H or -help Display this help\n"; print " -r report# Required argument - repor +t number\n"; print " -l -L or -list List supported report num +bers\n"; print " report Report name to process (c +sv format)\n"; exit; } # sub help sub list { print "Supported report formats: ". join(" ",@supported_reports) +. " \n"; print " 4b - User or Group Permissions for Directory - (compres +s report based on folders and permissions)\n\n"; print "\n"; help; exit; } # sub list # Default values for options # $list = ""; # $help = ""; # GetOptions complains if you use upper AND lower case options below. +It converts args into lowercase. GetOptions ( 'list|l' => sub { list }, 'report|r=s' => \$report, 'help|h' => sub { help } ); # Put supported report formats here - note the format! # @supported_reports = qw( 4b 5c 6d); # multiple report formats can be + listed like this - need subroutines for each @supported_reports = qw( 4b ); # list if $list; help if ($#ARGV >= 1); #needs minimum -r <reportnum> filename as argum +ents - should be nothing but a filename now # assign filename ($filename) = @ARGV; # (should only have one argument) # see if report is supported - %reports is a hash array unless ( grep {$_ eq $report } @supported_reports ) { print "Report format $report is NOT supported\n"; list(); exit(1); } # can't get map to work except with @ARGV - don't know why - keeping f +or later analysis #my %reports = map { $_ => 1 } @supported_reports; #if (exists($reports{$report})) { ... } # I started to work with manually parsing them but ran into embbeded c +ommas in fields, and found Text::CSV. It slurps in everything, but p +ulls it into a hash of hashes (never worked with that before, but tho +ught I'd try to slog through it as a learning exercise. # Open file, if it exists open(my $fh , '<', $filename) or die "Unable to open file, $filename"; # old method slurping in lines manually # Grab 1st line headers $first_line = <$fh>; $first_line =~ s/\x{FEFF}//; # strip out leading UTF8 Byte Order Mark +er (BOM) chomp $first_line; # strip off trailing carriage return @headers = split(",",$first_line); # Grab headers into array, split o +n commas $num_headers = scalar @headers; # get number of headers. # Here's the newer stuff # $aoa and $aoh are hash refs (in a scalar) - need to figure out how t +o parse/manipulate them. And build new ones $aoa = csv (in => $filename , encoding => "UTF-8"); # this slurps a fi +le into an array of arrays (need to find a way to sort this and also +remove all the UTF-8 encodings on each line.) $aoh = csv (in => $filename , headers => "auto", encoding => "UTF-8"); + # this slurps a file into an array of hashes (need to find a way to +sort this and also remove all the UTF-8 encodings on each line.) # pre-process keys to remove unicode embedded in key names - Found in +first few characters of 1st line, but was embedded in all the arrays. + (this fixed it!) for $key ( keys @{$aoh} ) { $data = $aoh->[$key]; for $temp_key (keys %$data) { if ($temp_key =~ /\x{FEFF}/) { $new_key = $temp_key; $new_key =~ s/\x{FEFF}//; $data->{$new_key} = delete $data->{$temp_key}; $temp_key = $new_key; print "------> Data key $data->{$temp_key} \n"; #print state +ments to look at data } print "Temp key $temp_key \n"; #print statements to look at data print "Data key $data->{$temp_key} \n"; #print statements to look at + data } } # These were earlier attempts to understand the data structure - I can + delete this for $key ( keys @{$aoh} ) { $data = $aoh->[$key]; print "key: $key, data $data \n"; print "key: $key, values:\n"; for $field ( keys %$data ) { $temp_field = $field; # $field =~ s/\x{FEFF}//; # (error in read only field) print " temp_field = $temp_field field = $field \n"; printf " %-18s => %s\n", $field, $data->{$field}; } } # for $key ( keys @{$aoh} ) { # $data = $aoh->[$key]; # print "key: $key, values:\n"; # for $field ( "Logon Name", "Current Permissions" ) { # printf " %-18s => %s\n", $field, $data->{$field}; # } # } # Generated # key: 0, values: # Logon Name => @FOO NOW Onsite Support # Current Permissions => FMRWX # key: 1, values: # Logon Name => Administrators # Current Permissions => FMRWX # key: 2, values: # Logon Name => Creator Owner # Current Permissions => FMRWX # route errors to the bit bucket (disappears) open STDERR, '>/dev/null'; # For each server and Access Path (file) basis, Collect the Current Pe +rmissions, and concatenate all the logon names with that same permiss +ion. Note when it's not inherited "<not inherited>" in the Inherited + From Folders field. # # {File Server}{Access Path}{Current Permissions}{Logon Name}{Inherite +d From Folders}{Flags}{User/Group}{Classification Results}..... # # 4b headers # 0 'File Server' # 1 'Access Path' # 2 'Current Permissions' # 3 'Logon Name' # 4 'Inherited From Folders' # 5 'Flags' # 6 'User/Group' # 7 'Classification Results' # 8 'Classification Results by Category (Including Nested)' # 9 'Total Hit Count' # Process 4b report # Need code about dynamic arrays here. # Early code to manually loop through the file, but found embedded com +mas in various fields, throwing the count of fields relative to the h +eaders. Gave it up after finding TEXT::CSV # Loop through file $line_count = 0; while (<$fh>) { $line_count++; chomp; # remove CR/LF @curr_line = split(",",$_); print "Line $line_count Field count != header count - Could be ext +ra commas embedded\n $_ \n\n" if ((scalar @curr_line) != (scalar @he +aders)); exit if ($line_count == 10); } # (<$fh>)