in reply to Re^2: Cannot work on second file after reading first file.
in thread Cannot work on second file after reading first file.

I've added some comments to your separate_stores function to try to better explain the potential issues I see.

sub separate_stores
{
  my (@li_stores, @ff_stores);
  my $stores_file = File::HomeDir->my_home . "/example/data/example.cs
+v";
  open my $fh, "<", $stores_file or die "$stores_file: $!";

# So at this point we know that you were able to successfully
# open example.csv, because you tested your open.

  my $csv = Text::CSV->new ({
    binary    => 1,
    auto_diag => 1,
  });

# Also know that if your parsing of CSV generates an error,
# that error will be spat to the screen for us.  Good.

  my $count_li = 0;
  my $count_ff = 0;

  $csv ->getline ($fh);

# We don't know if the preceding line returned 'undef',
# indicating an empty CSV file.  We also don't know if 
# it really stripped away a header row, or something useful
# instead.  We also don't know if there is any more CSV remaining
# after processing that first line.

#################
# Change the preceding line "$csv ->getline ($fh);" to the following,
# for better diagnostics:
#
  my $head = $csv->getline($fh) or die "Empty CSV file $!\n";
  warn "CSV header line contained (@{$head})\n" if $ENV{MYTEST_DEBUG};
  die "Nothing found after CSV header line.\n" if $csv->eof;
#
# Now set an environment variable "MYTEST_DEBUG" true, and re-run your
+ test.
#################

  while (my $row = $csv->getline ($fh)) {

# Your 'while' loop will never be entered if you're already at
# the end of file.  We don't know if that's an issue, because you
# didn't explicitly test what happened when you stripped the first
# CSV line.

    if ($row->[1] eq "li") {
      $li_stores[$count_li] = $row->[0];
      $count_li ++;
    } else {
      $ff_stores[$count_ff] = $row->[0];
      $count_ff ++;
    }
  }
  close $fh;
  return (\@li_stores, \@ff_stores);
}

[download]

It's almost certain that if this subroutine is failing in the big script, one of those assertions will also fail in the big script. And that will give you better information on why your subroutine is failing to do what you want.


Dave

Replies are listed 'Best First'.
Re^4: Cannot work on second file after reading first file.
by 1straw (Novice) on Feb 28, 2014 at 17:23 UTC

    Thank you for the explanation. I set MYSCRIPT_DEBUG to true. When I ran the script it printed:
    Field names detected: (# de tienda  tipo  ubicacion

    [download]

    with the exception of "(", these are the names of the fields
[reply]
[d/l]

      Now put a "warn "I'm inside the While loop\n";" statement inside of your while loop.

      Remember, you said: "&separate_stores seems to never go into the while loop and the two store number arrays return empty.". Those are also assertions that you need to prove to yourself. Add "warn ......" statements all over the place to track your progress, and to report the values held in variables.


      Dave

[reply]
[d/l]
Re^4: Cannot work on second file after reading first file.
by 1straw (Novice) on Feb 28, 2014 at 17:11 UTC

    Sorry for the ignorance, how do I set the environment variable to true?
[reply]

      The environment is the shell in which your program runs. That's an OS-dependent question. I didn't mean to force you to learn how to use your operating system too. Just remove the "if $ENV{....}" stuff.

      Here's the point. There is nothing wrong with the code in your subroutine. Add to that the fact that your subroutine is very much a self-contained black box. There are no parameters being passed in, and no global values being absorbed into it. You are testing your "open" correctly, so we know the CSV file opened. You have "auto_diag" set, so we know your CSV object instantiated correctly, and that calls to $csv->getline($fh) aren't throwing errors. There are no points of possible failure except that you haven't proven that your CSV file contains any lines of CSV. You also haven't proven that after stripping away the first line, there are still more CSV lines to follow.

      You have told us that your while() loop is never being entered. That can only happen if your CSV file has no lines of CSV after you strip the header.

      Come to think of it, you're also not telling us how you know that your while() loop is never being entered. Put a "warn "I'm in the while loop!\n"; statement inside of your while loop, and prove to yourself that you're not entering it, as you've told us. If it turns out that you are entering it, then the problem exists within code that you haven't shown us, which means this has been a waste of an hour or so of time.


      Dave

[reply]
[d/l]
[select]
Re^4: Cannot work on second file after reading first file.
by 1straw (Novice) on Feb 28, 2014 at 17:32 UTC

    Hi Dave,

    I apologize, I don't mean to waste your time.

    The code I posted is all the code, there's nothing else.

    I added the warn inside the while loop and it does not print. I had done something similar, using print instead of warn, which is how I (thought I) knew it was never entering the loop

    As I said, it works perfectly when run independently of the large script. It stops working when I add it to it

    Thanks
[reply]

      Ok, so you know the following:

      • The file opens.
      • The first line of the CSV file contains headers, as you expected.
      • The file has not reached "eof" after reading the first line (because we've now tested that).
      • The while loop never gets entered (because your "I'm inside the while loop" never prints.
      • The only way for the loop to not be entered is for my $row = $csv->getline($fh) to return a false value.
      • The documentation for Text::CSV says that 'getline' returns undef (which is false) if it fails to get and parse the line. This can happen at end of file, or in an error condition.
      • You set auto_diag in the constructor, so errors are being reported. I'm assuming you would tell us if an error was being reported, so it's probably safe to say one isn't. That only leaves "end of file" as a possible reason for you to not be entering the loop.

      Some of those things we know are contradictory. They can't both be true. We can't both not be at the end of the file, and be at the end of the file. We can't both have "auto_diag" set, and have an unreported error occurring.

      Is your "large script" running as the same user as your test script?


      Dave

[reply]
[d/l]

        In my testing, $csv->eof did not return true even after trying a second get_line (According to the documentation is does not return true until after an attempt to read past the EOF.)

        In any event the cause was a change in global state in open_po: $/ was being set to undef. See Re: Cannot work on second file after reading first file.

[reply]
[d/l]
[select]

      "It stops working" is why you want to have those prints in it. They will tell you *where* it stopped working, and if you put useful variables and comments in the prints, it will also tell you *what* things are not as you expect. From where and what, you can run the nearby steps of the code in your head and thus figure out *why* it is not working.

[reply]
Re^4: Cannot work on second file after reading first file.
by 1straw (Novice) on Feb 28, 2014 at 17:48 UTC

    Yes, both scripts are running as the same user

    I have updated my initial question with a sample CSV and a sample EDI file
[reply]

In Section Seekers of Perl Wisdom