awohld has asked for the wisdom of the Perl Monks concerning the following question:

I have CSV files upwards of 20 MB. I need to prepend a variable to the beginning of each line.

What would be the best way to do this?

I thought about opening up the file and and reading each line in one by one, prepending it, then writing it to a second file. Once the second file is finished the first one will be deleted.

Or I was thinking of loading it into memory all at once into a variable then doing a regex to to prepend that.

How would you guys recommend going about this?

Replies are listed 'Best First'.
Re: Prepend Evey Line in a File
by tirwhan (Abbot) on Feb 03, 2006 at 09:34 UTC

    This will modify your file and leave a copy of the original in filename.csv.bak

    perl -ni.bak -e 'print "PREFIX $_"' filename.csv

    Dogma is stupid.
      perl -pi.bak -e '$_="PREFIX $_"' filename.csv perl -pi.bak -e 's/^/PREFIX /' filename.csv
Re: Prepend Evey Line in a File
by GrandFather (Saint) on Feb 03, 2006 at 09:50 UTC

    The "code" (rather than "one liner") equivelent of tirwhan's example:

    use strict; use warnings; local @ARGV = ('file.csv'); local $^I = '.bak'; while (<>) { print "prepended stuff - $_"; }

    DWIM is Perl's answer to Gödel
Re: Prepend Evey Line in a File
by Enlil (Parson) on Feb 03, 2006 at 10:03 UTC
    one more way:
    perl -pi.bak -e 's/^/PREFIX/' filename.csv
Re: Prepend Evey Line in a File
by glasswalk3r (Friar) on Feb 03, 2006 at 13:46 UTC

    Loading 20MB in the memory will speed up the things, but there are some problems related with that:

    1. Do you have that much of memory available to your application?
    2. Do you need this speed improvement?
    3. Do your application runs in batch mode? Or it's used thru user interaction?

    Anyway, do not use REGEX to prepend the variable there. REGEX is a cool thing, but is a mistake to use REGEX as the solution for everything. In your case, a very simple print will do, so I using REGEX?

    As a hint, try to use print like this:

    print $prepend_value, $line_from_file, "\n";

    Instead of:

    print "$prepend_value$line_from_file\n";

    Usually using print with multiple arguments is faster them concatenating strings and print after that. Use Benchmark to test the speed of different methods.

    Alceu Rodrigues de Freitas Junior
    ---------------------------------
    "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill