Keep in mind that you are reading LDIF file. This means that the splitting of the line:

1. Will not always happen on the 80s character
2. The splitting can happen to any LDAP attribute, not just the DN

The following two ideas may help you:
1. Use Perl API's for reading LDIF (perldap-1.4.1 can do this). You will have to play around with API calls to prevent the API functions from reading all of the 2G file at once.

Pros: It should work, and you can even write back, to the server, the created LDAP Entry objects.

Cons: Mozilla LDAP API is based on C compiled libraries, however I have no idea how fast the resulting code will be.

2. As far as I'm aware when line in LDIF is split, the next line will begin with blank space. It should be a very easy for you to write an filter program will 'unsplit' big LDIF file. (btw: personally, i think that changing the line break separator is a totally wrong way to go here)


P.S.: Oh.. and you can try looking for that command line switch, that will prevent your whatever2ldif utility from doing the splitting ;-)

In reply to Reading LDIF (was: Re: Match a string that can contain a carriage return in a random position.) by fuzzycow
in thread Match a string that can contain a carriage return in a random position. by robartes

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.