::VERSION - what is it, why do we need it, is there now something we should differently.

Updating to new releases of a dist is probably the most important purpose of version. However the old perl motto is there is more than one way to do it.

Presently nice programmers are to place a VERSION in each module. There is no mandate to do so, nor is there a mandate that the version change for each distribution.

This generates all kinds of problems. Some dists, the version numbers all reflect the dist version. Some versions are simply an rcs or similar version number of the specific file. Many modules have no version.

Brad Fitzpatrick recognized what a pain in the `donkey` it is to update all the version numbers and has an auto-update in his Shipit dist. Nice.

Can we do better? Should we do better? Is there something better? Is it not a problem?

Can it also play into improved security in today's hacker-infested environment?

What if _ONLY_ the dist and it's related top-level module contained a <pkg>::VERSION number. Suppose all other modules part of the dist referred to the top-level module for a version number?

Is not the top-level version what most people want/need to know to be able to update to the latest 'version'?

Git is nice, it seems to be a rage, and has a nice concept and utilization of hashes.

What if we introduced a <pkg>::HASH::SHA1 in each package? Or something similar.

What if this became a part of perl core? Does calculating the hash cost much? It could certainly be calculated at perl's parse time. With 'require' you first have to traverse the filesystem to find the file, then open it, then read - all very costly - now you have something to parse, so why not do a hash calc on the side while parsing.

A hack is to place code in each module to calc its own hash when a version sub is called.

A better hack it to place it in an appropriate base class within the Moose/Class::MOP subsystem. Better yet would be in perl core.

This could be use to verify the file on disk is same as what the admin installed (in case of break-in) via some other data-repository of known hashes.

It also provides a unique version on the file - same hash - same file-version. For the dist version, it would auto-query the top-level module.

No more touching each file for each dist just to update the version - ugly. Diff lists show real changes for a given dist. Not a complete listing of files just because the version number got bumped.

Not the same as git, but conceptually analogous.

Just something to think about...

Best regards,

..Otto

Update 2009-04-27

Great example of VERSION/CPAN pain...

DBM-Deep-1.0014 v. DBM-Deep-1.0013

The ...13 version has DBM::Deep::File version 1.0013, but the ...14 version has dropped the version, so using cpan, it will load 1.0014, great, but then the next time, it thinks that the unversioned files of 14 are less than that of the versioned files of 13, so it wants to install 13, but you have 14 installed.

One word for this: INSANE

Update 2009-05-01

More pondering... So how do the various install pkgs (e.g. Module::Install) determine if your installed version of a module (which is what is specified in the prereq list) is at?

Fundamentally it is either effectively *grep'ing the module code file or evaluating and then querying the <pkg>::VERSION value. (Just looked.) Many rely upon ExtUtils::MakeMaker, which has a MM, which contains the parse_version which effectively is *grep'ing the code!

So CPAN.pm (to fetch new versions) uses MM as do the installers once they are downloaded and need to determine if all prereqs have been met.... Furthermore I believe that the PAUSE/CPAN indexing / searching mechanism is likewise *grep'ing.

Who actually evaluates the <pkg>::VERSION value???

This all does not seem to be hygienically clean or nice.

So why play the games of problematically *grep'ing when we can absolutely calculate a hash? This provides exactness of identity, not wishful thinking the author updates their concept of version. This hash can be mapped to a dist.

Some may argue that it then requires some kind of data repository, and that is problematic.

Right, don't we already depend upon a data repository to use CPAN? - The 02packages.details.txt.gz and 03modlist.data.gz are a requirement are they not? Why not add to or change the functionality...

  • Comment on <pkg>::VERSION, git, hashes, shipit, Class::MOP, Moose, perl core support - what NOW makes sense.
  • Select or Download Code

Replies are listed 'Best First'.
Re: <pkg>::VERSION, git, hashes, shipit, Class::MOP, Moose, perl core support - what NOW makes sense.
by moritz (Cardinal) on Apr 20, 2009 at 21:20 UTC
    I see your point with SHA1 hashes, and it does seem to work well for git.

    But remember that git keeps a database that lets you map that SHA1 hash to a date, author, log message and diff - and the other way round. If you don't have such a database, all you'll get on a SHA1 mismatch is just that - a message "hash verification failed", no more informations.

    It also implies that you can't require a minimum version of a module, just one specific version.

    (I do agree on your point of having just version number per distribution - I do that already, and it works fine for me; other solutions are svn $Id$ tags that automatically get expanded into revision numbers).

    So in summary I think that it might be worth pondering such an idea if you redo the complete module loading and distribution system, but I don't think it's feasible to plug onto the existing system.

    Note that Rakudo, a Perl 6 implementation, is now sufficiently advanced that people write modules for it, and want to distribute them. If you're into actually building something, they'll happily accept anything that actually works, and that tracks versions and dependencies.

    (The emphasis is on building stuff - we have enough people that are happy to discuss it, but discussions don't deploy code. Programs do.)

      Thanks for the feedback

      My ref's to git was simply as a jumping ground, not specific applications.

      ...git keeps a database that lets you map that SHA1 hash to a date, author, log message and diff

      Again analogy. Remember that 02packages.details.txt.gz as dist'd by cpan contains the following: module, version, dist. So each module has a version or undef. So there really is a "current" concept of mapping a hash->module->dist as "current". Furthermore a concept of top-level module can provide a concept of version, which applies to the whole dist.

      It also implies that you can't require a minimum version of a module, just one specific version.

      Not true. Again the top-level module is the "version" for the whole dist, and hence all modules part of the dist. So you could spec a min or max version for a "module" which would imply the top-level module of the dist, and you get the min version required result.

      ...but I don't think it's feasible to plug onto the existing system

      Actually I'm working on a module in which I hope to incorporate some of these ideas

      ...discussions don't deploy code. Programs do.

      It is the automation of this that I'm suggesting. Shipit is facilitating a better solution, but that solution pushes a version in each file. I question whether this is needed by posing the question of what ::VERSION is really used for.

        Again analogy. Remember that 02packages.details.txt.gz as dist'd by cpan contains the following: module, version, dist. So each module has a version or undef. So there really is a "current" concept of mapping a hash->module->dist as "current".

        Sure, but it's CPAN.pm that reads 02packages.details.txt.gz, not perl. Maybe I didn't fully understood what you want to do, but if you want be able to write things like

        use My::Module 'sha1:DEADBEEF... or above';

        and have perl verify that checksum, then perl needs to have access to that database, not only cpan. And it would need to store all hashes of all previous versions of all installed modules - which seems a bit like overkill.

        Actually I'm working on a module in which I hope to incorporate some of these ideas

        Ah, maybe I jumped to conclusions while reading the "perl core support" from the subject line.

Re: <pkg>::VERSION, git, hashes, shipit, Class::MOP, Moose, perl core support - what NOW makes sense.
by kyle (Abbot) on Apr 21, 2009 at 03:20 UTC

    You could make a module part of your distribution whose job is only to hold the distribution version number.

    package Your::Dist::Version; # note no strict $VERSION = 1.23; sub import { my $pkg = caller; ${ "$pkg\::VERSION" } = $VERSION; } 1;

    Then use Your::Dist::Version from every module in the distribution. Everything that does has the One True Version inserted into its namespace. You can change it in one place and keep other info in that one file as convenient.

      Another way is to make the top-level module more akin to a base of the whole dist, which in some manners it is. Then others inherit the version. Class::MOP::Object provides dump for everything ...

      At issue is to examine and dissect what really is the use of ::VERSION for each of the concepts of file, module, package, and distribution. These are all very different, but often are used interchangeably due to some of the syntactic sugar applied to them. Dealing with versioning means you really need to carefully examine the distinctions

      Then use Your::Dist::Version from every module in the distribution. Everything that does has the One True Version inserted into its namespace.
      But it won't work.

      The CPAN indexer searches for lines in modules that seems to set the VERSION number, extracts that line, and executes it. Just that line. Nothing more. Any clever tricks you'd like to use will work at run time (for instance, if someone does use Module VERSION;), but the important part, the CPAN indexer, won't be able to find the version number.

        Sorry if that might seem ignorant, but why is that a problem? Can't the CPAN indexer simply take the version from the META.yaml and take that as a default version?

        Then all you have to do is to make sure that the meta data and Your::Dest::Version agree.

Re: <pkg>::VERSION, git, hashes, shipit, Class::MOP, Moose, perl core support - what NOW makes sense.
by Bloodnok (Vicar) on Apr 21, 2009 at 11:05 UTC
    I would suggest that there is scope for both of your suggestions - I particularly like the SHA1 related suggestion since it helps assure the integrity of each installed module.

    From a $Pkg::VERSION POV, IMO (& I speak as a CM practitioner of many years standing), there is no one standard approach since the degree to which version control is required will, in almost every case, be required to be flexible enough to handle situations varying from top-level module only, down to every module.

    Furthermore, I'm interested in how your proposal would handle the situation whereby 2 otherwise separate modules are installed over the same directory space e.g. Some::Class::Frob & Some::Class::Nicate, each being a top-level module in it's own right, but neither is top-level when installed maybe an argument for version id on an individual basis.

    A user level that continues to overstate my experience :-))

      Multiple cases:

      Case Simple:

      Distribution containing:

      Some/Class/Frob.pm (which contains package Some::Class::Frob)

      and nothing else.
      and
      Distribution containing:

      Some/Class/Nicate.pm (which contains package Some::Class::Nicate)
      and nothing else.

      Well this is the simplist, each Frob.pm and Nicate.pm contain a VERSION for each of their dists and a HASH::SHA1 value for each. No issues.

      Case Typical:

      Distribution containing:

      Some/Class/Frob.pm (which contains package Some::Class::Frob)

      and nothing else.
      and

      Distribution containing:

      Some/Class/Nicate.pm (which contains package Some::Class::Nicate)
      Some/Class/Nicate/nsp1.pm (which contains package Some::Class::Nicate::nsp1)
      Some/Class/Nicate/nsp2.pm (which contains package Some::Class::Nicate::nsp2)
      Some/Class/Nicate/nsp2/nsp21.pm (which contains package Some::Class::Nicate::nsp2::nsp21)
      Some/Class/Nicate/nsp2/nsp22.pm (which contains package Some::Class::Nicate::nsp2::nsp21)

      Frob.pm contains VERSION for its distribution and a HASH::SHA1 value for itself.

      Nicate.pm contains a VERSION for its distribution and a HASH::SHA1 value for itself.

      nsp1.pm contains a VERSION which is a sub fetching the version of Nicate.pm and contains a HASH::SHA1 value for itself.

      similar for nsp2.pm, nsp21.pm, nsp22.pm.

      Case Harder - not simple module to file to package mapping.

      Distribution containing:

      Some/Class/Frob.pm (which contains package Some::Class::Frob)
      Some/Class/Frob/nsp1.pm (which contains package Some::Class::Frob::nsp1)
      Some/Class/Frob/nsp2.pm (which contains package Some::Class::Frob::nsp2)

      Distribution containing:

      Some/Class/Nicate.pm (which contains package Some::Class::Nicate)
      Some/Class/Nicate/nsp1.pm (which contains package Some::Class::Nicate::nsp1)
      Some/Class/Nicate/nsp2.pm (which contains package Some::Class::Nicate::nsp2 and contains package Some::Class::Frob::nsp1)

      Ugh, so dist Some-Class-Nicate has a file that contains multiple packages, one of which populates its module package space (i.e. module name mapped to file on file system contains package same as module name) and it contains package Some::Class::Frob::nsp1.

      So what do you want for versioning? ::VERSION is what is defined within a package or forced into a package name space.

      I contend that you would NOT place a ::VERSION in the package namespace for Some::Class::Frob::nsp1 as defined in Some::Class::Nicate::nsp2. You would have a ::VERSION in Some::Class::Nicate::nsp2 which is a sub fetching the version from the top-level module of dist for Some::Class::Nicate and you would have a HASH::SHA1 for the Some/Class/Nicate/nsp2.pm. Module requirement would be for Some::Class::Nicate::nsp2::VERSION (a sub). That would be the case even if the only change was in Some::Class::Frob::nsp1 as found in Some::Class::Nicate::nsp2 because the only way to get the change is to require Some::Class::Nicate::nsp2. You can not partially require a module. (Well I lie, you can with BEGIN & co).

      Hopefully that addresses what you were asking.

Re: <pkg>::VERSION, git, hashes, shipit, Class::MOP, Moose, perl core support - what NOW makes sense.
by JavaFan (Canon) on Apr 21, 2009 at 22:08 UTC
    Suppose modules used git SHA1 numbers as versions. Now I want to install Module X. Module X needs module Y, and at least version 81AC89FE38.... I have version DA21348FF0... installed. Quick, do I need to update module Y to be able to use module X or not?

    What if this became a part of perl core?
    Replacing a working system that is independent of Perl core support (and hence will work with quite old versions of Perl) with a system that needs perl core support (and hence cannot become active for another 5 years or so, until 5.12 comes around) seems like 77 giant leaps backwards.

    For all the suggestions I've heard the past years to change the CPAN VERSION handling, yours is the most insane. By a long shot.

      Suppose modules used git SHA1 numbers as versions. Now I want to install Module X. Module X needs module Y, and at least version 81AC89FE38.... I have version DA21348FF0... installed. Quick, do I need to update module Y to be able to use module X or not?

      Exactly the point. The concept of a version number and a file identification should be separate concepts.

      As analogy git has separated the concept of a file name and the file contents. The file contents is a blob which has an id. The id is then related to a name in the tree object. They are two separate concepts. The concept of version number, more like a git tag, identifies a group of items, a hash uniquely identifies the contents of something, in the case we are discussing, files as related to modules and packages in perl.

      ...yours is the most insane. By a long shot.

      Thank you - either i'm insane or cutting new trails :)