in reply to Re^2: JSON::XS Wide Character Problem
in thread JSON::XS Wide Character Problem
If the actual size of your "huge file with like a million of those objects" really is known for certain to be problematic for the memory capacity on the machine you're using, read the section of the JSON::XS manual that talks about "INCREMENTAL PARSING". Also learn about the $json->shrink() function.
UPDATE: / In particular, look at the section of the manual that contains this sentence: "Assume that you have a gigantic JSON array-of-objects, many gigabytes in size, and you want to parse it, but you cannot load it into memory fully (this has actually happened in the real world :)." /
Some json files do not have line-breaks at all (and those that do may vary as to "CRLF" vs. "LF" style). Even if you think you're very confident about knowing the format/layout of the json data, I'd say it's virtually never a good idea to treat json data as line-oriented input. Don't do that.
|
---|