monty77 has asked for the wisdom of the Perl Monks concerning the following question:

#!/usr/bin/perl use strict; use warnings; #Open data file and put it into variable open FILE, "<data.txt"; my $string = do {local $/; <FILE> }; #Set offset if there is a header record and add it to initial position my $offset = 0; my $currentpos = 0 + $offset; #Set lastpos = length of data my $lastpos = length($string); #Read until end of data while ($currentpos < $lastpos) { #Grab record length within data my $recordlength = (substr $string, $currentpos, 4); #Grab record type which is 5th char my $recordtype = substr $string, $currentpos+4, 1; #Using record length grab the record and put it into variable my $fragment = substr $string, $currentpos, $recordlength; #Calculate starting position for next record .. not used at present my $nextstartpos = $currentpos + $recordlength + 1; #Use if clause to identify only $ record types and print the record da +ta if ($recordtype eq '$') { print " record type: $recordtype\n"; print " data: $fragment\n"; #Inrement starting position for the next loop $currentpos = $currentpos + $recordlength; } }

Any ideas?

I've commented to try and explain what it does, it's working through a file identifying records of different types within it and printing certain record types to screen however it just hangs after printing the 1st record for me

Thanks!

Replies are listed 'Best First'.
Re: Can someone tell me why this hangs??
by dasgar (Priest) on Aug 28, 2010 at 22:11 UTC

    It looks like you might have an infinite loop in your code. Based on the condition for your while loop, you need $currentpos to be incremented each time through the loop. However, you're line of code to increment that variable is inside an if statement. Should you ever hit a condition when that if statement evaluates to be false, your while loop will continue forever.

    I'd recommend modifying you code to move the incrementing line out of the if statement like the following:

    if ($recordtype eq '$') { print " record type: $recordtype\n"; print " data: $fragment\n"; } #Inrement starting position for the next loop $currentpos += $recordlength;
Re: Can someone tell me why this hangs??
by AnomalousMonk (Archbishop) on Aug 28, 2010 at 23:09 UTC

    dasgar points out what seems likely to be your most immediate problem.

    In addition, you don't define the structure of your data records, but if the 'record length' field is anything other than four decimal digits (in particular, if it's a raw, four-byte integer of some kind), you may have a problem down the road with record lengths that work out to be something unexpected, especially zero. (However, you do have warnings turned on, which could alert you if some of these problems arise.)

    Here are some examples of the handling of various four-digit strings. The '0123' string is intended to represent something that some humans and computers might be tempted to interpret as an octal number. The  qq{\x00\x00\x00\x08} represents a 32-bit, raw binary, big-endian 8.

    >perl -wMstrict -le "print qq{'$_' -> }, 0 + $_ for '1234', '12AB', 'AB12', '0123', qq{\x00\x00\x00\x08}; " '1234' -> 1234 Argument "12AB" isn't numeric in addition (+) at -e line 1. '12AB' -> 12 Argument "AB12" isn't numeric in addition (+) at -e line 1. 'AB12' -> 0 '0123' -> 123 Argument "\0\0\0^H" isn't numeric in addition (+) at -e line 1. ' ' -> 0
Re: Can someone tell me why this hangs??
by Old_Gray_Bear (Bishop) on Aug 29, 2010 at 00:01 UTC
    There are a couple of ways to debug this kind of problem. You can alter the code by littering the landscape with print statements. This has the advantage of making you look at the code to determine where you should be dropping your "DEBUG Reached line NNN, variable xxx=$xxx<<<<\n" statements. Quite often when I use this style of debugging I would run across my logic error in the process. Then I had to go back and remove my previous debugging statements. When I initially was introduced to Perl, some mumble years ago, that is how I did debugging.

    After a few months I discovered the Perl Debugger --  perl -d; I have not gone back to print statements, except as a last resort (and usually in CGI coding).

    I step through the code until the code takes off in a different direction from what I expected. I look at the variables involved in the control construct that surprised me. I track back to where the erroneous value appeared, fix my bug and go on.

    Whether one method or the other is 'better' is not something that I can really say. I get a better feel for the logic by watching the way the Program flows as I step from point to point in the execution. I know other programmers who can parse the logic-flow as they read the code, and decide on where they want to insert debugging statements based on their mental map of the logic. It all depends on what you are comfortable with.

    ----
    I Go Back to Sleep, Now.

    OGB

Re: Can someone tell me why this hangs??
by ww (Archbishop) on Aug 29, 2010 at 00:18 UTC
    This fragment compiles... BUT it makes no sense because you have many vars with no indication of how they're populated.

    Neither do you give us any sample data which shows typical headers (post at least a representative fragment of data.txt, please) nor any hint of how you're determining where a header ends and its data begins... nor is there any indication of how you determine the value of $offset.

    In short, my charitable assessment is that this is a badly-pared-down fragment of your code; my less charitable views aren't worth writing about.

    But I am pretty sure you should read perldoc -f substr.

    In any case, please, clarify.

Re: Can someone tell me why this hangs??
by TomDLux (Vicar) on Aug 29, 2010 at 01:46 UTC

    I would use unpack or regexen or substr to extract the data ... the key is to move the values into variables on a reliable basis.

    #!/usr/bin/perl use strict; use warnings; #Open data file and put it into variable open FILE, "<data.txt"; my $string = do {local $/; <FILE> }; my ($idx, $max) = (0, length $string ); while ( $idx < $max - 4 ) { my $fragment= substr( $string, $idx, 4 ); $idx+= length $fragment; my $recordtype = substr( $string, $idx, 1 ); $idx++; if ($recordtype eq '$') { print " record type: $recordtype\n"; print " data: $fragment\n"; } else { print "Non-dollar fragment: '$fragment', '$recordtype'\n"; } } print "Unused portion of data file: ", substr( $string, $idx );

    You will deduce I had a line break in my data.

    record type: $ data: 1234 Non-dollar fragment: 'qwer', 't' Non-dollar fragment: 'asdf', 'g' Non-dollar fragment: 'zxcv', 'b' record type: $ data: !@#$ record type: $ data: ?:"

    Notice that if i come within 4 characters of the end, there are characters which can't be used duetot he lack of a recordtype.

    As Occam said: Entia non sunt multiplicanda praeter necessitatem.

      Thanks everybody .. moving the $currentpos update out of the if fixed it beautifully.

      To answer one question, the record length is always 4 chars in length and as for the other suggestions they're very much appreciated but I've only been writing perl for a couple of days so my scripts are going to be a little amateurish at this stage - does the job though :)

      Cheers!

Re: Can someone tell me why this hangs??
by toolic (Bishop) on Aug 28, 2010 at 23:20 UTC
    To debug your problem, just start adding print statements throughout your code. Then you can verify if each variable contains what you expect it to contain. If you end up in an infinite while loop, and get too much info scrolling quickly by your screen, just exit out of it.

    You should have also posted a very short sample (10 lines) of your input file.

      I like to declare a variable at the top of the script, like $DEBUG or something catchy, then use if statements to print variables throughout the script. This way you can easily turn debugging off or on.

      Example:

      my $DEBUG = 1; #Set to 1 (print debugging info) or 0 (no debugging) print "Script executing...\n" if $DEBUG; do_stuff(); sub do_stuff{ print "sub->do_stuff()\n" if $DEBUG; #do things here... }