Concept99 has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks, I'm a bit of a newbie to Perl, so please bear with me. I have developed/pieced together a script that will calculate the total file size of a directory (including its subdirectories). What I really need to be able to do is have my script: (1) read a list of server names and their shares(ie \\server\share) FROM a csv file (2) scan those shares and report the size of each (3) output that information to a new csv file Here is my code so far: 
use File::Find;
use strict;

my ($dir);

open(OUT, "> output.csv") or die("Couldn't open output.csv\n"); 

#Displays total size of specified path (total includes subfolders)
foreach my $dir (@ARGV) {
   my $total;
   print "\n\nWalking $dir\n\n";
   find(sub { $total += -s }, $dir);
   $total = ($total / 1024) / 1024;
   $total = sprintf("%0.2f", $total);
   print OUT "$dir, $total,";
   print "\tOutput created.\n";
}
close(OUT);


#Displays program syntax on screen
if ($ARGV[0] eq ""){&syntax; exit 1;}
$dir = $ARGV[0];

sub syntax
 {
 print "
 SYNTAX
 \tperl dirsize.pl {dirname}\n\n
 Examples\n
 \tperl dirsize.pl D:\MYFILES\n\t 
 \tperl dirsize.pl \\\\SERVER1\\PROFILES\n\n";
 }

[download]
I understand that I probably need to use DBI to parse the CSV, but I am not quite sure how to go about it. I would appreciate any type of help on this. Thanks in advance!

Replies are listed 'Best First'.
Re: Reading and writing CSV
by Aristotle (Chancellor) on Jun 04, 2003 at 14:44 UTC
    DBI would be overkill here. Have a look at Text::xSV - that'll do the trick. There's also Text::CSV_XS if you prefer it. Tie::CSV might be worth a look as well.

    Makeshifts last the longest.

[reply]
Re: Reading and writing CSV
by DrHyde (Prior) on Jun 04, 2003 at 14:47 UTC
    Whilst you *can* use DBI to look at CSV files with the DBD::CSV driver, you're probably better off just using the Text::CSV (pure-perl) or Text::CSV_XS (perl and C) module. Try the C version first, cos it's faster, but if you're programming on Windows you might have to use the pure perl version.
[reply]

      Text::CSV_XS is available via PPM compiled for ActivePerl. Run 

      ppm install Text-CSV_XS

      [download]
      and off you go :-)

      Jenda
      Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
         -- Rick Osborne

      Edit by castaway: Closed small tag in signature

[reply]
[d/l]
        Ah, I thought PPM still relied on having a C compiler, but just put modules into Windows' database so you could easily uninstall them or sumfink. That's useful to know, thanks.
[reply]
Re: Reading and writing CSV
by hangmanto (Monk) on Jun 04, 2003 at 15:59 UTC
    I agree with the comments that DBI would be overkill. DBI is a great module, but get some more experience with more basic perl before trying to tackle it.

    The Perl Cookbook has a recipe for parsing comma-separated data. This recipe (1.15) is taken from Mastering Regular Expressions. (My apologies if posting this is inappropriate).

    BEGIN QUOTED MATERIAL:

    Use the procedure in Mastering Regular Expressions. 

    sub parse_csv {
    my $text = shift;      # record containing comma-separated values
    my @new  = ();
    push(@new, $+) while $text =~ m{
        # the first part groups the phrase inside the quotes.
        # see explanation of this pattern in MRE
        "([^\"\\]*(?:\\.[^\"\\]*)*)",?
           |  ([^,]+),?
           | ,
       }gx;
       push(@new, undef) if substr($text, -1,1) eq ',';
       return @new;      # list of values that were comma-separated
}

    [download]
    Or use the standard Text::ParseWords module. 
    use Text::ParseWords;

sub parse_csv {
    return quotewords(",",0, $_[0]);
}

    [download]
    END QUOTED MATERIAL

    The Perl Cookbook is a must-have book for any perl programmer. If you don't have a copy, please buy one. It has helped me immensely.

[reply]
[d/l]
[select]
[reply]
[d/l]
Re: Reading and writing CSV
by AcidHawk (Vicar) on Jun 04, 2003 at 15:30 UTC

    As you have been told in earlier posts you probably don't really need the DBI::CSV stuff. But if you are interested look at DBD::CSV Memory Leak which has some code in it. The memory issue was sorted in a newer version of the module.

    -----
    Of all the things I've lost in my life, its my mind I miss the most.
[reply]
Re: Reading and writing CSV
by wufnik (Friar) on Jun 05, 2003 at 11:43 UTC
    hola dudes, dudettes, others;

    i know this is late, but a wee scripty follows which actually does what Concept99 wants. and a little more; filecounts, directory counts for the paths to be examined. remove to taste.

    i use Adam Rich's Win32::DirSize because i gather from the Concept99's posting's paths that we are on a Win32 system and also, incidentally, it is > 50% faster, does the formatting etc and more.

    Concept99 need not read & reread mastering RegeXes, admirable tho the book is, to parse a csv. is anything simpler to parse than a csv of directory paths? well, yes, i hope that none of the paths have commas in them etc. but do i need a module to write a csv file? no sirree. perl is practical enough by itself. here goes:

    use Win32::DirSize; # >= 50% faster than file::find
chomp(my @lines = (<DATA>));
my @pathsets = map { [ split ",", $_ ] } @lines;
my @dstats;
push @dstats,join ",",("Directory", "Size", "FileCount", "DirCount");

foreach my $pathset (@pathsets){
    foreach my $path (@{$pathset}){
    if (dir_size($path, my $dstat) == DS_RESULT_OK){
        my $size = best_convert(my $unit, 
                    $dstat->{HighSize}, 
                    $dstat->{LowSize});
        my $fcnt = $dstat->{FileCount};
        my $dcnt = $dstat->{DirCount};
        push @dstats, join ",",($path, sprintf("%7.4f %s", $size, $uni
+t),$fcnt, $dcnt);
    }
    push @dstats, undef;    
    }
}
map { print } grep { defined } @dstats;

__DATA__
\\ooby\dooby$\stodge,\\ooby\ccc$\etc\lib
\\foo\bar$\wufnik,\\foo\bar$\id

    [download]
    which will spit out the results for each dir encountered. hope it helps,

    ...wufnik

    -- in the world of the mules there are no rules --
[reply]
[d/l]
      As I said earlier, I'm new to Perl, so I am not quite sure how to install "Win32-DirSize-1.00.tar.gz" file I downloaded. I am running ActivePerl 5.6 on a Win2k Box. When I use nmake and try to install this, I get the following response: 
      cp csvtest.pl blib\lib\Win32\csvtest.pl
cp dirsize.pl blib\lib\Win32\dirsize.pl
cp csvtest2.pl blib\lib\Win32\csvtest2.pl
cp test2.pl blib\lib\Win32\test2.pl
cp DirSize.pm blib\lib\Win32\DirSize.pm
AutoSplitting blib\lib\Win32\DirSize.pm (blib\lib\auto\Win32\DirSize)
cp try.pl blib\lib\Win32\try.pl
        C:\Perl\bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib C:\Perl\lib\E
+xtUtils/xs
ubpp  -typemap C:\Perl\lib\ExtUtils\typemap DirSize.xs > DirSize.xsc &
+& C:\Perl\
bin\perl.exe -IC:\Perl\lib -IC:\Perl\lib -MExtUtils::Command -e mv Dir
+Size.xsc D
irSize.c
Please specify prototyping behavior for DirSize.xs (see perlxs manual)
        cl -c  -nologo -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE -DNO_ST
+RICT -DHAV
E_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT
+_READFIX -
O1 -MD -Zi -DNDEBUG    -DVERSION=\"1.00\"  -DXS_VERSION=\"1.00\"  -IC:
+\Perl\lib\
CORE  DirSize.c
'cl' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: 'C:\WINNT\system32\cmd.exe' : return code '
+0x1'
Stop.

      [download]
      I'd appreciate any suggestions if you have em.
[reply]
[d/l]

Back to Seekers of Perl Wisdom