bentrim has asked for the wisdom of the Perl Monks concerning the following question:

I have a csv file of 28,000 lines. I need to append to each line this string: ,UF, A, Y, 9, U I have an example 8 line file "test2.csv":

10004,Able,Baker,able.baker@notrealemail.com 14634,Charley,Delta,charley.delta@notrealemail.com 12886,Echo,Foxtrot,echo.foxtrot@notrealemail.com 14366,Golf,Hotel,golf.hotel@notrealemail.com 10178,India,Juliet,india.juliet@notrealemail.com 10164,Kilo,Lima,kilo.lima@notrealemail.com 10124,Mike,November,mike.november@notrealemail.com
This is the result that I need:
10004,Able,Baker,able.baker@notrealemail.com,UF, A, Y, 9, U 14634,Charley,Delta,charley.delta@notrealemail.com,UF, A, Y, 9, U 12886,Echo,Foxtrot,echo.foxtrot@notrealemail.com,UF, A, Y, 9, U 14366,Golf,Hotel,golf.hotel@notrealemail.com,UF, A, Y, 9, U 10178,India,Juliet,india.juliet@notrealemail.com,UF, A, Y, 9, U 10164,Kilo,Lima,kilo.lima@notrealemail.com,UF, A, Y, 9, U 10124,Mike,November,mike.november@notrealemail.com,UF, A, Y, 9, U My goal is to perform this appending function within a perl script. I found a site suggesting this type of command formating: <code> perl -ne 'chomp; printf "%sUF,A,Y,9,U\n", $_' test2.csv > final.csv;
It works perfectly. So I tried it within a single line script with strict and warnings. It generated this error:
Bareword found where operator expected at ./test_2.pl line 8, near "'c +homp; printf "%sUF,A,Y,9,U\n", $_' test2" (Missing operator before test2?) syntax error at ./test_2.pl line 8, near "-ne" Execution of ./test_2.pl aborted due to compilation errors
I have no clue how to address this error. I can run this awk ommand at command line and it also works perfectly:
awk '{print $0,"UF,A,Y,9,U"}' OFS="," < test2.csv > final.csv
So, I tried to run the same thing inside a perl script:
print "awk '{print \$0,\"UF,A,Y,9,U\"}' OFS\=\",\" < test2\.csv > fina +l\.csv\n";
It displays to screen without errors but generates zero output and does not create the 'final.csv' file. How do I accomplish appending this string correctly?

Replies are listed 'Best First'.
Re: Appending string to all lines
by AnomalousMonk (Archbishop) on Nov 11, 2016 at 05:42 UTC
    I found a site suggesting this type of command formating ... I tried it within a single line script with strict and warnings. It generated this error ...

    One way to get an idea of what a command line "one-liner" would have to look like as a script is to deparse the command line source together with the interpreter switches:

    c:\@Work\Perl>perl -MO=Deparse -n -e "chomp; printf qq{%sUF,A,Y,9,U\n}, $_ " LINE: while (defined($_ = <ARGV>)) { chomp $_; printf "%sUF,A,Y,9,U\n", $_; } -e syntax OK
    Please see O and B::Deparse. (Update: Please also see perlrun for a discussion of the wrappers the interpreter puts around code when it sees switches like -n and -p.)


    Give a man a fish:  <%-{-{-{-<

Re: Appending string to all lines
by robby_dobby (Hermit) on Nov 11, 2016 at 04:51 UTC
    Hello bentrim,

    Ideally, you should be using a well-tested module like Text::CSV for these kind of activities. Since this is an one-time activity, I'm going to show you the simplest way. Why not do something like:

    $ cat > somestuff.csv 10004,Able,Baker,able.baker@notrealemail.com 14634,Charley,Delta,charley.delta@notrealemail.com 12886,Echo,Foxtrot,echo.foxtrot@notrealemail.com 14366,Golf,Hotel,golf.hotel@notrealemail.com 10178,India,Juliet,india.juliet@notrealemail.com 10164,Kilo,Lima,kilo.lima@notrealemail.com 10124,Mike,November,mike.november@notrealemail.com $ perl -pi -e 's/\Z/,UF,A,Y,9,U/' somestuff.csv $ cat somestuff.csv 10004,Able,Baker,able.baker@notrealemail.com,UF,A,Y,9,U 14634,Charley,Delta,charley.delta@notrealemail.com,UF,A,Y,9,U 12886,Echo,Foxtrot,echo.foxtrot@notrealemail.com,UF,A,Y,9,U 14366,Golf,Hotel,golf.hotel@notrealemail.com,UF,A,Y,9,U 10178,India,Juliet,india.juliet@notrealemail.com,UF,A,Y,9,U 10164,Kilo,Lima,kilo.lima@notrealemail.com,UF,A,Y,9,U 10124,Mike,November,mike.november@notrealemail.com,UF,A,Y,9,U $

    Doing things like calling perl to do AWK is just overkill, since perl was meant to take in all the goodness of sed, awk and shell scripting, wiping away the pain of all other scripting stuff!

    Do read more of perl "pie-ness" in perlrun and CSV handling in Text::CSV. Have fun!

      Reading about Bareword errors helped me to figure out that I need to pursue your suggestion in using Text::CSV module. I will move in that direction.

      Thank you for the assistance.

        Frankly, IMHO, using Text::CSV for such a simple problem is overkill, since it can be done without any regard to the fact that your file contains CSV data. only need to add a trailer to each line.

        Update: see my post below for a simple solution.

Re: Appending string to all lines
by Laurent_R (Canon) on Nov 11, 2016 at 08:03 UTC
    So I tried it within a single line script with strict and warnings. It generated this error:
    How are we supposed to guess your error if you don't show your script? Please show the code.

      It is simply a single line attempt.

      #!/usr/bin/perl use strict; use warnings; perl -ne 'chomp; printf "%sUF,A,Y,9,U\n", $_' test2.csv;
      It generates the Bareword error. I have been reading on that topic and can see that the earlier comment pointing me to Text::CVS module is the correct approach. So that is where I'm going.

      Thank you.

        That should indeed generate syntax errors because perl is not a valid function name in Perl. You can't just type random shell commands into perl and expect it to process them like your shell would - they are completely different systems.

        #!/usr/bin/perl -n use strict; use warnings; chomp; printf "%sUF,A,Y,9,U\n", $_;

        See perlrun for how to use flags like -n and perlfunc for a list of all the functions available. Note that I've made no attempt to optimise your algorithm here.

Re: Appending string to all lines
by Laurent_R (Canon) on Nov 11, 2016 at 16:30 UTC
    Assuming you pass the input file as an argument and redirect the output to your output file, i.e. that you use a syntax like this for launching the program under your shell:
    perl script.pl test2.csv > final.csv
    then the script itself can be something like this:
    use strict; use warnings; while (my $line = <>) { chomp $line; print "$line", " ,UF, A, Y, 9, U\n"; }
Re: Appending string to all lines
by bentrim (Initiate) on Nov 11, 2016 at 04:19 UTC

    I just realized that my example of desired output includes spaces between the appended values. There should not be spaces. Sorry.

      So, why not edit your post and mark it as an EDIT? :-)

        Sorry, I missed the strike component.

        Thanks.