Is there a way to reduce Data::Dumper's output size?

tkguifan has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Is there a way to reduce Data::Dumper's output size? by cursion (Pilgrim) on Feb 10, 2015 at 21:08 UTC
`perldoc Data::Dumper` [download] reveals a couple of things, Indent and Freeze. Indent can reduce the spaces and keep it human readable, while Freeze can do anything you want. Or just gzip the output files.	[reply] [d/l]
Re: Is there a way to reduce Data::Dumper's output size? by GotToBTru (Prior) on Feb 10, 2015 at 21:02 UTC
Have you looked at the documentation? Dum Spiro Spero	[reply]
Re: Is there a way to reduce Data::Dumper's output size? by Laurent_R (Canon) on Feb 10, 2015 at 21:23 UTC
You could reprocess the dumper output before storing it by replacing any sequence of one (or two) or more spaces by a single space: `$dumper_output =~ s/ +/ /g;` [download] The result will not be pretty, but you'll save quite a bit of space. Or you could use a module more dedicated to what you want to do than Data::Dumper, such as, for example Storable, which produces a more compact output. Update: crossed out my first suggestion because, as pointed out by Anonymous Monk below, it might mess things up is either keys or values of the hash have multiple spaces. Je suis Charlie.	[reply] [d/l]
Re^2: Is there a way to reduce Data::Dumper's output size? by Anonymous Monk on Feb 10, 2015 at 22:25 UTC
`$dumper_output =~ s/ +/ /g;` Probably not a good idea... `use Data::Dumper; my $dumper_output = Dumper({"foo bar"=>"quz\n baz"}); print $dumper_output; $dumper_output =~ s/ +/ /g; print $dumper_output; __END__ $VAR1 = { 'foo bar' => 'quz baz' }; $VAR1 = { 'foo bar' => 'quz baz' };` [download]	[reply] [d/l] [select]
Re^3: Is there a way to reduce Data::Dumper's output size? by Laurent_R (Canon) on Feb 10, 2015 at 22:49 UTC
Ooops, yes, you are quite right, I did not think of the cases where keys or values might have multiple spaces. This would usually not happen where I work (because we usually normalize keys and values), but, yes, there is a danger of messing things up. Thank you for for the comment. Je suis Charlie.	[reply]
Re^2: Is there a way to reduce Data::Dumper's output size? by tkguifan (Scribe) on Feb 11, 2015 at 05:15 UTC
I'm trying Storable. Especially because the doc says: 'The heart of Storable is written in C for decent speed. Extra low-level optimization have been made when manipulating perl internals, to sacrifice encapsulation for the benefit of a greater speed.' My data gets so big that speed matters.	[reply]
Re^3: Is there a way to reduce Data::Dumper's output size? by Anonymous Monk on Nov 01, 2015 at 10:23 UTC
You really should try Sereal.	[reply]
Re: Is there a way to reduce Data::Dumper's output size? by ikegami (Patriarch) on Feb 11, 2015 at 20:05 UTC
Looks like you want JSON. Or anything else. I wouldn't use DD for serializing. But you could use `local $Data::Dumper::Indent = 0; local $Data::Dumper::Terse = 1;` [download] Don't use Terse you can you don't have two references to the same variable.	[reply] [d/l]
Re: Is there a way to reduce Data::Dumper's output size? by locked_user sundialsvc4 (Abbot) on Feb 11, 2015 at 00:31 UTC
May I cordially suggest that you, instead, use one of several alternatives which are expressly designed to store data structures: `freeze/thaw` ... not my favorite anymore. XML JSON YAML You will find, in CPAN, multiple robust implementations, ready to go.
Re^2: Is there a way to reduce Data::Dumper's output size? by tkguifan (Scribe) on Feb 11, 2015 at 06:14 UTC
I also tried the JSON module. Its output is almost the same size that of freeze ( only marginally bigger ). Its advantage that it is pure text, still somewhat human readable. I will see how fast it is when my structure gets really big.	[reply]
Re^3: Is there a way to reduce Data::Dumper's output size? (Sereal) by Discipulus (Canon) on Feb 11, 2015 at 08:00 UTC
Give a try to Sereal. L* There are no rules, there are no thumbs.. Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.	[reply] [d/l]
Re^4: Is there a way to reduce Data::Dumper's output size? (Sereal) by perlancar (Hermit) on Feb 11, 2015 at 16:58 UTC
Re^5: Is there a way to reduce Data::Dumper's output size? (Sereal) by BrowserUk (Patriarch) on Feb 11, 2015 at 17:38 UTC
Some notes below your chosen depth have not been shown here
Re^5: Is there a way to reduce Data::Dumper's output size? (Sereal) by Anonymous Monk on Nov 01, 2015 at 10:02 UTC
Re^4: Is there a way to reduce Data::Dumper's output size? (Sereal) by tkguifan (Scribe) on Feb 11, 2015 at 08:36 UTC
Re^5: Is there a way to reduce Data::Dumper's output size? (Sereal) by Anonymous Monk on Nov 01, 2015 at 10:27 UTC
Re^2: Is there a way to reduce Data::Dumper's output size? by tkguifan (Scribe) on Feb 11, 2015 at 05:18 UTC
Currently I'm trying freeze/thaw. Why did you abandon using it? Is it unsafe?	[reply]