In the spirit of TIMTOWTDI and the fact that I have been having a blast using Tie::File lately here is another solution.

Preconditions

First off I set up a very small data file to test with: 
Tom
data
data
data
data
data
$
Dick
data
data
data
data
data
$
Harry
data
data
data
data
data
$

[download]
This is a very small test set I realize but wasn't really ambitious enough to create a file by hand as large as you were talking about.

The code

Here is another suggestion for how to solve the problem: 
#!/usr/bin/perl -w
#############################
use strict;
use warnings;
use Tie::File;
 
my @ry=();    # This will be tied
my $OLDIFS=$/; # save the IFS
$/="\$\n";     # now change it to suit our needs

#
# Rope tie and brand 'em! YEEE-HAW! 
tie @ry,"Tie::File","datFile.txt" or die $!;
 
foreach my $rec(@ry){ #iterate through the records
    chomp $rec;  # Eliminate IFS
    next unless $rec;  # Eliminate spew about blank records
                       # if they happen
    my $fname=(split(/\n/,$rec))[0]; # Get the name
    next unless $fname; # ho hum. Sometimes you're the 
                        # windshield, sometimes you're
                        # the bug.
    open FOUT,"> $fname" or die "$fname:$!"; # Open file
    print FOUT $rec; #store record
    close FOUT;  # done with this
}
untie @ry; # cut them thar ropes!

[download]

The potential issue with this solution is I've been told (and I can't verify this one way or another) is that Tie::File is memory greedy and will slurp the whole file into memory in one gulp. If you have a memory starved box that you are running this script on and the assertion of memory use is true then you might have an issue. You mention having memory issues in your OP and I don't know how starved for memory the box you are running this on is..

You didn't specifiy if you wanted to preserve the dollar sign record separator or not but if you eliminate the chomp you will accomplish that as well.

HTH!

In reply to Re: Splitting long file by blue_cowdawg
in thread Splitting long file by Micz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.