in reply to Re: Cannot work on second file after reading first file.
in thread Cannot work on second file after reading first file.


Thank you McA. Sorry for the error. I was writing the program with variables in Spanish and decided to change to English before posting here. I ran a search & replace on that variable and replaced without $. All edi_file should be $edi_file.

#!/usr/bin/perl
use strict;
use warnings;
use File::HomeDir;
use List::Util 'first';
use List::MoreUtils 'first_index';
use Text::CSV;

my @po = &open_po;

#my @po_no_headings = &erase_headings(@po);

#my @po_separated = &make_product_store_quantity(@po_no_headings);
#print "@po_separated\n";

my ($li_stores_ref, $ff_stores_ref) = &separate_stores;

my @liv_stores = @$li_stores_ref;
my @ffr_stores = @$ff_stores_ref;


print "Li stores: @liv_stores\n";
print "FF Stores: @ffr_stores\n";


# ****************************************
# 1) open .EDI file
# 2) split into lines through "~" line separator
# 3) Place .EDI file contents into @po_array

sub open_po 
{
  my ($edi_file, $po_data, @po_array);
  $edi_file = File::HomeDir->my_home . "/example/po//example.EDI" or d
+ie ".EDI file not found\n";
  open ($po_data, '<', $edi_file) or die "Could not open the file 'edi
+_file' $!\n";
  undef $/;
  @po_array = split (/\~/, <$po_data>);
  close $po_data;
  return @po_array;
}

# ****************************************
# Erase all information from purchase order before first PO1 line

sub erase_headings 
{
  my ($delete_point);
  $delete_point = (first_index {/PO1/} @_) - 1; # Find first PO1 line
# Shift array up to first PO1 line
  for (0..$delete_point) {
    shift @_;
  }
  @_;
}

# ****************************************
# Return array with product, store, quantity 
# (i.e. THSB01 00021 3 00043 1 THSB02...etc.)

sub make_product_store_quantity 
{
  my (@line, @product_stores, $product, $product_index);
  while(scalar (@_) !=0) {
    @line = split (/\*/, $_[0]); # Split line into fields

# Find product code (i.e. THSB01) and shift array twice
    if ($line[0] eq "PO1") {
      $product_index = (first_index {/ST/} @line) + 1;
      $product = $line[$product_index];
      shift;
      shift;
    }

# Find stores & quantity for each store. 
# Write product code followed by stores & quantity to array @product_s
+tores
    if ($line[0] eq "SDQ") {
      shift (@line);
      shift (@line);
      shift (@line);
      unshift (@line, $product);
      push (@product_stores, @line);
      shift;

# Exit when PO line starts with CTT, meaning end of PO1 details
    } elsif ($line[0] eq "CTT") {
      undef (@_);
    }
  }
  @product_stores;
}

# ****************************************
# 1) Open and read stores.csv 
# 2) Place all "li" type store numbers in @li_store array
# 3) Place all "ff" type store numbers in @ff_store array
# 4) Return array references

sub separate_stores
{
  my (@li_stores, @ff_stores);
  my $stores_file = File::HomeDir->my_home . "/example/data/example.cs
+v";
  open my $fh, "<", $stores_file or die "$stores_file: $!";

  my $csv = Text::CSV->new ({
    binary    => 1,
    auto_diag => 1,
  });

  my $count_li = 0;
  my $count_ff = 0;

  $csv ->getline ($fh);
  while (my $row = $csv->getline ($fh)) {
   if ($row->[1] eq "li") {
      $li_stores[$count_li] = $row->[0];
      $count_li ++;
    } else {
      $ff_stores[$count_ff] = $row->[0];
      $count_ff ++;
    }
  }
close $fh;
return (\@li_stores, \@ff_stores);
}

[download]

Replies are listed 'Best First'.
Re^3: Cannot work on second file after reading first file.
by davido (Cardinal) on Feb 28, 2014 at 16:59 UTC

    I've added some comments to your separate_stores function to try to better explain the potential issues I see.

    sub separate_stores
{
  my (@li_stores, @ff_stores);
  my $stores_file = File::HomeDir->my_home . "/example/data/example.cs
+v";
  open my $fh, "<", $stores_file or die "$stores_file: $!";

# So at this point we know that you were able to successfully
# open example.csv, because you tested your open.

  my $csv = Text::CSV->new ({
    binary    => 1,
    auto_diag => 1,
  });

# Also know that if your parsing of CSV generates an error,
# that error will be spat to the screen for us.  Good.

  my $count_li = 0;
  my $count_ff = 0;

  $csv ->getline ($fh);

# We don't know if the preceding line returned 'undef',
# indicating an empty CSV file.  We also don't know if 
# it really stripped away a header row, or something useful
# instead.  We also don't know if there is any more CSV remaining
# after processing that first line.

#################
# Change the preceding line "$csv ->getline ($fh);" to the following,
# for better diagnostics:
#
  my $head = $csv->getline($fh) or die "Empty CSV file $!\n";
  warn "CSV header line contained (@{$head})\n" if $ENV{MYTEST_DEBUG};
  die "Nothing found after CSV header line.\n" if $csv->eof;
#
# Now set an environment variable "MYTEST_DEBUG" true, and re-run your
+ test.
#################

  while (my $row = $csv->getline ($fh)) {

# Your 'while' loop will never be entered if you're already at
# the end of file.  We don't know if that's an issue, because you
# didn't explicitly test what happened when you stripped the first
# CSV line.

    if ($row->[1] eq "li") {
      $li_stores[$count_li] = $row->[0];
      $count_li ++;
    } else {
      $ff_stores[$count_ff] = $row->[0];
      $count_ff ++;
    }
  }
  close $fh;
  return (\@li_stores, \@ff_stores);
}

    [download]

    It's almost certain that if this subroutine is failing in the big script, one of those assertions will also fail in the big script. And that will give you better information on why your subroutine is failing to do what you want.


    Dave

[reply]
[d/l]

      Thank you for the explanation. I set MYSCRIPT_DEBUG to true. When I ran the script it printed:
      Field names detected: (# de tienda  tipo  ubicacion

      [download]

      with the exception of "(", these are the names of the fields
[reply]
[d/l]

        Now put a "warn "I'm inside the While loop\n";" statement inside of your while loop.

        Remember, you said: "&separate_stores seems to never go into the while loop and the two store number arrays return empty.". Those are also assertions that you need to prove to yourself. Add "warn ......" statements all over the place to track your progress, and to report the values held in variables.


        Dave

[reply]
[d/l]

      Sorry for the ignorance, how do I set the environment variable to true?
[reply]

        The environment is the shell in which your program runs. That's an OS-dependent question. I didn't mean to force you to learn how to use your operating system too. Just remove the "if $ENV{....}" stuff.

        Here's the point. There is nothing wrong with the code in your subroutine. Add to that the fact that your subroutine is very much a self-contained black box. There are no parameters being passed in, and no global values being absorbed into it. You are testing your "open" correctly, so we know the CSV file opened. You have "auto_diag" set, so we know your CSV object instantiated correctly, and that calls to $csv->getline($fh) aren't throwing errors. There are no points of possible failure except that you haven't proven that your CSV file contains any lines of CSV. You also haven't proven that after stripping away the first line, there are still more CSV lines to follow.

        You have told us that your while() loop is never being entered. That can only happen if your CSV file has no lines of CSV after you strip the header.

        Come to think of it, you're also not telling us how you know that your while() loop is never being entered. Put a "warn "I'm in the while loop!\n"; statement inside of your while loop, and prove to yourself that you're not entering it, as you've told us. If it turns out that you are entering it, then the problem exists within code that you haven't shown us, which means this has been a waste of an hour or so of time.


        Dave

[reply]
[d/l]
[select]

      Hi Dave,

      I apologize, I don't mean to waste your time.

      The code I posted is all the code, there's nothing else.

      I added the warn inside the while loop and it does not print. I had done something similar, using print instead of warn, which is how I (thought I) knew it was never entering the loop

      As I said, it works perfectly when run independently of the large script. It stops working when I add it to it

      Thanks
[reply]

        Ok, so you know the following:

        • The file opens.
        • The first line of the CSV file contains headers, as you expected.
        • The file has not reached "eof" after reading the first line (because we've now tested that).
        • The while loop never gets entered (because your "I'm inside the while loop" never prints.
        • The only way for the loop to not be entered is for my $row = $csv->getline($fh) to return a false value.
        • The documentation for Text::CSV says that 'getline' returns undef (which is false) if it fails to get and parse the line. This can happen at end of file, or in an error condition.
        • You set auto_diag in the constructor, so errors are being reported. I'm assuming you would tell us if an error was being reported, so it's probably safe to say one isn't. That only leaves "end of file" as a possible reason for you to not be entering the loop.

        Some of those things we know are contradictory. They can't both be true. We can't both not be at the end of the file, and be at the end of the file. We can't both have "auto_diag" set, and have an unreported error occurring.

        Is your "large script" running as the same user as your test script?


        Dave

[reply]
[d/l]

        "It stops working" is why you want to have those prints in it. They will tell you *where* it stopped working, and if you put useful variables and comments in the prints, it will also tell you *what* things are not as you expect. From where and what, you can run the nearby steps of the code in your head and thus figure out *why* it is not working.

[reply]

      Yes, both scripts are running as the same user

      I have updated my initial question with a sample CSV and a sample EDI file
[reply]

In Section Seekers of Perl Wisdom