How about IO::All?

It seems to cover most things:

IO::All
IO::All of it to Graham and Damian! 
0.33
IO::All::DBM
DBM Support for IO::All 
 
IO::All::Dir
Directory Support for IO::All 
 
IO::All::File
File Support for IO::All 
 
IO::All::Filesys
File System Methods Mixin for IO::All 
 
IO::All::Link
Symbolic Link Support for IO::All 
 
IO::All::MLDBM
MLDBM Support for IO::All 
 
IO::All::Pipe
Pipe Support for IO::All 
 
IO::All::STDIO
STDIO Support for IO::All 
 
IO::All::Socket
Socket Support for IO::All 
 
IO::All::String
String IO Support for IO::All 
 
IO::All::Temp
Temporary File Support for IO::All
[download]

The closest I can think of is Filesys::Virtual. Or maybe you want Cache::Cache (with its many backends)?

Makeshifts last the longest.

You could possibly use the DBI module / system for a level of abstraction (and for even more abstraction, subclass Class::DBI. This may be helpful to you because of the myriad DBD drivers designed to work with DBI, which include:

• DBD::Anydata: DBI access to XML, CSV and other formats.
• DBD::SQLite: A single-file, self-contained RDBMS in a DBI driver.
• DBD::MySQL (and other more heavy-weight databases)

My point is that if you start with the framework for using a database, you can keep it as generic as possible at the higher levels of abstraction, and with minimal changes, substitute any of the wide range of DBD database drivers.

If you wish to layer DBI over flat files, you can start with DBD::Anydata. When it comes time to graduate to a real database, you can use DBD::SQLite, and when that doesn't keep you afloat you can switch to MySQL, PostgreSQL, or some other real database.

Interesting. I can just see it now: DBD::FTP, DBD::HTTP, ... ;-)

I'm not actually interested in parsing or dealing with the internals of these files. The files would simply be BLOBs in a database. Think of this more as systems management - the task at hand is simply to store one or more files together so that they can be retrieved together. These files could be RPMs or the like (more like tarballs in my case), but the code I'm looking at is not actually interested in the contents of those tarballs, just an easy way to copy them into multiple locations - think, for a second, of the CPAN archive as an example. We want to add a bunch of tarballs to the repository, and then later pull them out - you want 10, I want 15, and between us a few are the same. No point in both of us going to the originator of those few tarballs to get them, we'll all go to the repository to get them. This gets even more important the more people there are, imagine being the owner of File::Spec or FindBin or other such modules and being bugged by every perl developer to get your modules. The repository takes care of all that work.

Our current process goes and recreates the tarball each time it's needed. I want to nix that because we can be creating the same tarball 20+ times per day. A repository would, of course, help here. We'd create each tarball once, put it in the repository, and then extract it from there each time it's needed.

Interesting. I can just see it now: DBD::FTP, DBD::HTTP, ... ;-)

That's actually not a bad idea. The trick is how to make FTP and HTTP interactions into a SQL like dialect. For inspiration, take a look at DBD::Google and the more recently released DBD::iPod, or the older, and kinda funky DBD::Chart. And actually DBD's are not really as hard to write as you might think. There is plenty of good documentation, and solid implementations out there which you can work from. Personally I always look at DBD::mysqlPP whenever I need a reference, the code is really clean and very easy to read.

-stvn

I've been using Subversion, so that is what I'll explain it with, but CVS is very similar.

Basically, you can put a directory or files under version control, at which point they are uploaded into the repository (and yes, they do call it a respository, which is why your question made me think of it). You can allow access to the repository through SSH or through HTTP, and you can use the normal SSH or HTTP restrictions on read/write permissions, authenticaction, and domain restrictions. The people or programs that you want to be able to get these files would request the appropriate files. This could be done in preset groups by using directories (as in, they could request a whole directory to get the files in it), or they could request any subset of them that they want.

Anyway, that's just a quick suggestion. It sounds like it may be what you're looking for, or it may not be. I hope it is helpful, though.

~Fatty

Just in case anyone finds this node via Super Search (hahahaha... sorry.), I'll let everyone know that my manager *did* allow me to put Cache::Repository on CPAN. And it's working marvelously.

What I've seen so far is that it has: cut down on the time to create all our packages (since 90% of our tarballs and associated files are common and would otherwise be rebuilt each time), and cut down on development time when our requirements changed on nearly zero-notice (the associated files just up and changed - we only had to change the "insert to repository" code, the "retreive from repository" code was unchanged).

Currently, I only have a filesystem driver - but anyone who thinks this may be useful with other backends are more than welcome to develop other backends for it ;-)

Just a reminder/clarification: this is designed to work with files. Not file contents. That's the significant difference between this and, say, the rest of the caching modules mentioned in this thread. Some of my files are 100's of MB in size - I don't really want to be copying that into memory to copy it back out somewhere else. I use enough memory as it is ;-)