I think the workflow would then become:

1. During the build procedure, run the PM's through pack to create the binary version of the file.
2. During runtime, read the binary file via unpack

Matt.

Most of the comments I've seen have assumed this is data. But your phrasing of "run the PM's thru pack..." implies that this isn't data, per se, but an actual perl module (.pm that you're accessing via use Some::Module). If this is true, I am not sure that rolling your own is the best choice. You might want to clarify on the point. Is the file you're trying to read, which you called "the PM's", pure data, data in perl format, or data plus other perl code (such as functions, for loops, etc), or something else?

I don't know specifically of a CPAN module that allows loading of a compressed module, but it would surprise me if there wasn't one (a quick search for "perl compress module" finds perl modules that compress something else, not perl modules that allow you to compress your source code). Or something like the Acme::Buffy, which will modify the source code. I just don't know of what that module would be... but maybe my phrasing will spark something in a more experienced monk

I hesitate to recommend Module::Crypt: I hesitate, because Module::Crypt doesn't really do what the name implies: never rely on Module::Crypt to protect your source code from prying eyes; it will not keep it secret! But I mention it nonetheless because I think that maybe the XS output from Module::Crypt would be smaller than your 10MB++ perl module. I don't know if it would be, but it might be something to try.

You can use Filter::gunzip to use a compressed module.

Just compress the module using gzip and then prepend the header telling perl to use the filter:

gzip -9 Module.pm
echo "use Filter::gunzip;" >Module.pm
cat Module.pm.gz >>Module.pm
[download]

> I don't know specifically of a CPAN module that allows loading of a compressed module,

well Par::Packer seems to have a compress option.

Cheers Rolf
_{(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery}

Yes pryrt, you are correct; the 'data' is all contained in a *massive* PM that is computer-generated.

The PM only contains many stanzas of 'data' in the format I described in the OP. That is, the module contains no code that operates on it. The 'operational' code is contained in other modules that pull in this *massive* PM.

Regarding compression, that is another option. I was hoping to avoid it, because of the overhead associated with decompressing, storing the result, and then operating on it.

I will certainly be keeping your suggestions in mind as I move forward with this. They are much appreciated.

Matt.

> Is that what a Perl expert would do?

A Perl expert would ask for more details.

You can certainly do what you described...

...just probably there is an even better solution.

Cheers Rolf
_{(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery}