Akatsuki has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks, I need your guidance with an issue. I have a script that I want to use on multiple files from the same folder with the script and write the output for each file with a different extension. For example: file1.txt file2.txt file3.txt after being processed by my script become file1.csv file2.csv file3.csv My script is

#!/usr/bin/env perl 
use strict;
use warnings;

print "Enter the name of the original file(including termination): ";
my $original = <STDIN>;
chomp ($original);
print "Enter the name of the final file(including termination): ";
my $final = <STDIN>;
chomp ($final);

open( my $original_fh, "<", $original ) or die $!;
open( my $final_fh,    ">", $final )    or die $!;

my $first = <$original_fh>;
print $final_fh $first;
my @replace = $first =~ /"([^"]*)"/g;

while (my $line = <$original_fh>) {
    print "Enter the desired number of columns: ";
        my $nr = <STDIN>;
        chomp($nr);
    $line =~ s/((\w\w\t){$nr})/$1\n/g;
    $line =~ s/\t/,/g;
    $line =~ s/(\d{2})/$replace[$1]/g;
    print $final_fh $line;
}

close $original_fh;
close $final_fh;

[download]

I want this script to read all files with a specified extension from it's folder so I don't have to enter the files manually when there are to many files. I have tried a few methods unsuccessfully, but I am new to perl and still learning the basics...thank you for any help I can get!

  • Comment on Read from multiple files change the data with a script and the write the respective output files with a different extension
  • Download Code

Replies are listed 'Best First'.
Re: Read from multiple files change the data with a script and the write the respective output files with a different extension
by Discipulus (Canon) on Oct 13, 2016 at 09:19 UTC
    Hello Akatsuki

    you can use glob to collect all files you need. Then foreach file you are processing you can make up the name of the copy changing it's extension from .txt to .csv

    Please note that Perl has many options to do such kind of operations with a single line of program: see perlrun and serch for -i -l -n switches. See Inplace Editing for a bunch of examples.

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
[reply]
[d/l]
[select]
Re: Read from multiple files change the data with a script and the write the respective output files with a different extension
by Eily (Monsignor) on Oct 13, 2016 at 09:25 UTC

    I have tried a few methods unsuccessfully
    You should have shown us those attempts, so we could help you understand why they failed, and how to improve them.

    Anyway, a rather simple and straightforward solution to your problem, is to turn your code into a function: 

    sub process_file
{
  my ($original, $final) = @_;

  open( my $original_fh, "<", $original ) or die $!;
  open( my $final_fh,    ">", $final )    or die $!;

  my $first = <$original_fh>;
  print $final_fh $first;
  my @replace = $first =~ /"([^"]*)"/g;

  while (my $line = <$original_fh>) {
    print "Enter the desired number of columns: ";
          my $nr = <STDIN>;
          chomp($nr);
    $line =~ s/((\w\w\t){$nr})/$1\n/g;
    $line =~ s/\t/,/g;
    $line =~ s/(\d{2})/$replace[$1]/g;
    print $final_fh $line;
  }

  close $original_fh;
  close $final_fh;
}

    [download]
    And call that function on all your files (process_file("A.txt", "B.csv");).

    You can get the list of files using glob ; this will return the list of files that match a given pattern: my @files = glob 'path/*.txt'; If you don't provide a path, glob will look into the current directory, which you can change with chdir.

    And to change the extension, you can just use a substitution: my $new_file = $name =~ s/txt$/csv/r;

    Edit: changed the last sentence, Akatsuki obviously already knows about substitutions.

[reply]
[d/l]
[select]

      Thank you very much for your help, but after I turn my code into a function, I still can't figure out how to read all the files and replace the extension...I wrote something like this: 

      #!/usr/bin/perl
use warnings;
use strict;

sub process_file
{
  my ($original, $final) = @_;

  open( my $original_fh, "<", $original ) or die $!;
  open( my $final_fh,    ">", $final )    or die $!;

  my $first = <$original_fh>;
  print $final_fh $first;
  my @replace = $first =~ /"([^"]*)"/g;

  while (my $line = <$original_fh>) {
    print "Enter the desired number of columns: ";
          my $nr = <STDIN>;
          chomp($nr);
    $line =~ s/((\w\w\t){$nr})/$1\n/g;
    $line =~ s/\t/,/g;
    $line =~ s/(\d{2})/$replace[$1]/g;
    print $final_fh $line;
  }

  close $original_fh;
  close $final_fh;
}

my @files = glob '*.txt';
my $new_file = $name =~ s/txt$/csv/r;
process_file($files, $new_file);

      [download]

      but I know it can't run...I didn't quite understand how can I use the substitution that you wrote because I know that $name is not declared. I appreciate any advice you can give me!

[reply]
[d/l]

        You have many files and want to process one at a time, so use a loop:

        my @files = glob '*.txt';
for my $name (@files) {
  my $new_file = $name =~ s/txt$/csv/r;
  process_file($name, $new_file);
}

        [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom