Re: Split very big string in half

needs to be split into two smaller strings

Your code creates more than two smaller strings/files. What do you want? Two equal-sized strings/files or strings/files with a maximum length?

strings [...] to be passed to a PDF converter

Blindly splitting HTML at arbitary offsets will most likely create invalid HTML. Depending on how it is rendered, you will lose some information. In the worst case, the converter may simply refuse to render it at all.

Back to your posted problem, assuming strings/files with a size limit, maybe more than two parts: Have you tried substr? Something like this (untested):

if (length($html[$i]) > 1000000) {
    my $z=0;
    while ($html[$i] ne '') {
        open (my $file, '>', "giant_${i}_$z.htm") or die $!;
        print $file substr($html[$i],0,500000,'');
        close $file;
        push(@lists, "giant_${i}_$z.htm");
        $z++;
    }
} else {
    # ...
[download]

Note that the above code is destructive due to using a four-argument substr, i.e. $html[$i] will be empty after the code has run. Non-desstructive code can use the $z counter, like this:

if (length($html[$i]) > 1000000) {
    my $z=0;
    while ($html[$i] ne '') {
        open (my $file, '>', "giant_${i}_$z.htm") or die $!;
        print $file substr($html[$i],$z*500000,($z+1)*500000);
        close $file;
        push(@lists, "giant_${i}_$z.htm");
        $z++;
    }
} else {
    # ...
[download]

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Comment on Re: Split very big string in half Select or Download Code

Replies are listed 'Best First'.
Re^2: Split very big string in half by fpscolin (Initiate) on Apr 15, 2015 at 19:21 UTC
You are correct, my apologies. I should have clarified; this massive HTML string is actually a concatenation of hundreds of smaller HTML files. I realize this sounds stupid, but I am combining hundreds of files into few (about 10) and then checking if any of these is too big. This is necessary for reasons I won't go into too much (table of contents generation, internal pdf linkage...). Ideally I would split the string at the first closing </html> tag past 500,000 characters. This is why I was looking to use a regex `/(.{500000,}?<\/html>/gs`	[reply] [d/l]
Re^3: Split very big string in half by karlgoethebier (Abbot) on Apr 15, 2015 at 21:03 UTC
"...I am combining hundreds of files into few (about 10) and then checking if any of these is too big..." Perhaps you should rethink your concept? I'm unsure if my idea fits to your needs but couldn't you convert each file to pdf first and then concatenate them, perhaps step by step? As far as i remember CAM::PDF provides this feature. Best regards, Karl ŤThe Crux of the Biscuit is the Apostropheť	[reply]
Re^3: Split very big string in half by afoken (Chancellor) on Apr 16, 2015 at 11:45 UTC
karlgoethebier is right. Your concept is broken. You can't concat several HTML documents to make one big document. Of course your computer let's you do exactly that, but while the result may be rendered by some generous browser, it is really junk. Split the big file into the original documents, and pass each document separately to the PDF converter. Splitting should not be that hard, assuming the original documents are reasonably clean: Open the big file for reading open an output file read a line from the big file if the line contains something that looks like the start of a HTML document ("`<!DOCTYPE`", "`<HTML`", "`<?xml`"), write everything up to the match to the current output file, then close it, create a new file, write the match and everything following it to the new file. else, write the line to the current output file repeat from step 3 until eof close input and output files You may need to add some special cases: The first output file will usually be empty if there is no junk at the start of the big file. All of the signatures for the document start may be found in a single document, following each other (a valid XHTML document starts with `<?xml version="1.0" ?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml">`, whereas HTML5 starts with `<!DOCTYPE html><html>`). You usually want only the first match. A simple trick is to assume that the `<?xml` and `<!DOCTYPE` declarations are relatively short, but a complete HTML document needs much more data, at least 500 characters (or something like that). So if `tell OUTPUT` returns a non-negative number less than 500 when matching a signature, don't create a new output file, but continue to write to the old output file. This also avoids an empty first file. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l] [select]