Re^2: split large CSV file >9.1MB into files of equal size that can be opened in excel

Have you taken a look at the split csv files?

You're splitting in bytes, which means that unless your csv files are *very* reliably structured, you will likely break the file up into chunks that break full lines.

Say you have a file like this:

1,2,3,4,5
[download]

and a read statement like this:

read $fh, $chunk, 5;
[download]

You'll end up with split file one with:

1,2,3
[download]

and the second split file with:

,4,5
[download]

Which might not be what you wanted.

Comment on Re^2: split large CSV file >9.1MB into files of equal size that can be opened in excel Select or Download Code

Replies are listed 'Best First'.
Re^3: split large CSV file >9.1MB into files of equal size that can be opened in excel by pryrt (Abbot) on Sep 28, 2016 at 23:17 UTC
I think that's why the OP ran `$chunk = <FH>;` after the `read(FH, $chunk, 10000);` block: it reads whatever is left on the current line (or, if the `read` happened to end at the end of a line, it will read one extra line) and includes that extra bit in the same file. (This of course still assumes that there are no newlines supposedly protected by quotes in the source CSV, but with a name like "all_tags2", I am guessing that there aren't newlines in "tags".)	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: split large CSV file >9.1MB into files of equal size that can be opened in excel
by pryrt (Abbot) on Sep 28, 2016 at 23:17 UTC

I think that's why the OP ran $chunk = <FH>; after the read(FH, $chunk, 10000); block: it reads whatever is left on the current line (or, if the read happened to end at the end of a line, it will read one extra line) and includes that extra bit in the same file. (This of course still assumes that there are no newlines supposedly protected by quotes in the source CSV, but with a name like "all_tags2", I am guessing that there aren't newlines in "tags".)

[reply]
[d/l]
[select]