greatshots has asked for the wisdom of the Perl Monks concerning the following question:

monks,

whenever I do file read operation from my perl program, it is reading the whole file in a single stretch. I understand that the records separator is <EOF> for my OS. ( not sure ).
So, In all my programs I am explicitly setting the $/ = "\n". how can I make a global change. should I need to change something from OS level or any changes in perl configuration ?

  • Comment on changing record separator to all my perl programs

Replies are listed 'Best First'.
Re: changing record separator to all my perl programs
by davido (Cardinal) on Oct 13, 2006 at 08:18 UTC

    What OS are you using? This sounds fishy.

    My guess is that you're looking at a text file that was transferred in binary mode between two different operating systems. Macs, Unix/Linux, and Windows machines each have their own "end of line" sequence. If you transfer a text file blindly between these OS's, you end up with a minor problem. Unix has utilities to deal with this after the fact, in case a file's line-endings got messed up somehow.: dos2unix and unix2dos.

    So, if your Perl script is having difficulty recognizing line endings, you've probably simply got your line endings mismatched.

    One solution is to simply check the line endings of your text files to ensure they're correct, and if not, fix them. It's trivial to write a Perl script to do this for you. Another solution would be to build line-end fixing logic into your scripts, but that seems like too much fiddling for such a common and simple problem.

    And of course, from now on, be sure to transfer your text files in text mode via software smart enough to fix the line endings for you in the first place.


    Dave

      I am using hp unix. I am running the program in ksh. both the perl program and all input files residing in same OS.

        all input files residing in same OS

        But what generates these input files? Are you sure that they have the correct Unix line endings in them? Have you looked at them using a utility like od?

        --
        <http://dave.org.uk>

        "The first rule of Perl club is you do not talk about Perl club."
        -- Chip Salzenberg

Re: changing record separator to all my perl programs
by cdarke (Prior) on Oct 13, 2006 at 08:25 UTC
    Show us yer code! $/ is end-of-record, not end-of-file. Maybe you are reading the file in list context? Then you will read the whole file. So:
    @lines = <$handle>; # Reads the whole file $line = <$handle>; # Reads the next record
Re: changing record separator to all my perl programs
by davorg (Chancellor) on Oct 13, 2006 at 08:23 UTC

    What operating system are you using? And where does your data come from?

    When Perl is built it should set up the correct value for $/ by default. If that hasn't happened then there is something very strange about your Perl installation.

    One way that I can think of for this effect to happen, is if you are running your programs on Windows (which uses \r\n as the line end marker) but that your data is generated on some version of Unix (which uses \n). In that case, your Windows programs won't ever recognise the Unix line end characters. If this is the case then really the conversion of the line end characters should be dealt with by the file transfer mechanism (for example, using FTP in ASCII mode.

    --
    <http://dave.org.uk>

    "The first rule of Perl club is you do not talk about Perl club."
    -- Chip Salzenberg

      One way that I can think of for this effect to happen, is if you are running your programs on Windows (which uses \r\n as the line end marker) but that your data is generated on some version of Unix (which uses \n). In that case, your Windows programs won't ever recognise the Unix line end characters. If this is the case then really the conversion of the line end characters should be dealt with by the file transfer mechanism (for example, using FTP in ASCII mode.

      This does not seem to be the case:

      C:\temp>cat -A foo.txt foo$ bar$ baz$ C:\temp>perl -le "chomp(@a=<>);print for map qq|<$_>|, @a" foo.txt | c +at -A <foo>^M$ <bar>^M$ <baz>^M$

      Oh, and for completeneess, the other way round:

      tilde:~ [11:19:27]$ cat -A foo.txt foo^M$ bar^M$ baz^M$ tilde:~ [11:19:47]$ perl -le 'chomp(@a=<>);print for map qq|<$_>|, @a' + foo.txt | cat -A <foo^M>$ <bar^M>$ <baz^M>$
      Windows may use CRLF for the end of line marker, but $/ is set to LF on Windows. When reading, Perl converts CRLF to LF (except in binary mode). This happens before searching for the end of the line, so setting $/ to "\r\n" would not work. It also means that Perl can read UNIX files no problem. (On the other hand, unix doesn't read Windows files correctly.)
Re: changing record separator to all my perl programs
by blazar (Canon) on Oct 13, 2006 at 09:30 UTC
    whenever I do file read operation from my perl program, it is reading the whole file in a single stretch. I understand that the records separator is <EOF> for my OS. ( not sure ).

    The input record separator, as of perldoc perlvar is a newline by default. Said this, perl does may apply some translations to \n in input and output layers, but to the effect of doing the right thing for your OS, and anyway that's a whole another story.

    So, In all my programs I am explicitly setting the $/ = "\n". how can I make a global change. should I need to change something from OS level or any changes in perl configuration ?

    How strange! And does the explicit setting change things? What does

    perl -le 'print $/ eq "\n" ? "OK" : "NO"'

    print for you?

Re: changing record separator to all my perl programs
by swampyankee (Parson) on Oct 13, 2006 at 15:46 UTC

    Show us some code!

    Also, if your record separator is set to "EOF", could you tell us what the value of EOF is for your OS? My experience is that explicit EOF markers aren't used in most media (iirc, they're still used for tapes, cards, and JCL) in most current O/S (even Windows pretty much ignores its ^Z EOF marker of the distant past).

    So, I see a few possibilities. One is that your file has no record separators (unlikely, but possible). A second is that your version of Perl was built improperly, so it's not recognizing your O/S's record separators. Since you've said you're on a *ix, it's possible that you're processing files from a Mac, which once used \r as a record separator. The final possibility is that there is a bug in your code. The first place I look is in what I've done, so I would suggest you look to your code first, then the other possibilities.

    emc

    At that time [1909] the chief engineer was almost always the chief test pilot as well. That had the fortunate result of eliminating poor engineering early in aviation.

    —Igor Sikorsky, reported in AOPA Pilot magazine February 2003.
Re: changing record separator to all my perl programs
by tinita (Parson) on Oct 13, 2006 at 16:47 UTC
    please do a
    perl -wle'use Data::Dumper; $Data::Dumper::Useqq = 1; print Dumper $/'
    and show the otuput.
    then read in a file and dump the content like above and post this output here, too.