make-like dependency tools in perl

bsb has asked for the wisdom of the Perl Monks concerning the following question:

Dependency driven execution is a problem that keeps on cropping up. My lastest encounter is in progating changes through a data warehouse. Before that is was running one section of a test script but ensuring the dependencies are fulfilled (eg. login first). Build systems and component publishing systems deal with dependencies too.

What I need is something like "make" which has a graph of dependencies, rules for updating them and a means to check the age/change state of items. You can then "make endproduct" and it will update the parts that need to be updated in the right order. The problem with make is that it's tied to files (is there another way to use it?)

I've looked around for either a module or software with the right hooks, but can't find anything. Cons and Makepp are written in Perl but, as with make, the seem too file-system tied.

AAP by Bram "vim" Moolenaar looks more flexible but uses Python as it's scripting language. (I recommend the linked "design decisions" article as an information source for the problem domain).

On CPAN there's Algorithm::Dependency, but it's doesn't address this problem, leaving Graph as the best module to try an adapt.

Is there anything better out there that I've missed?
Or another way of thinking about the problem?

Brad

Comment on make-like dependency tools in perl

Replies are listed 'Best First'.
Re: make-like dependency tools in perl by cbrandtbuffalo (Deacon) on Jun 24, 2005 at 11:53 UTC
You may be able to get some ideas from Module::Dependency. There are three parts; I think the important bits for you would be in the Indexer. I haven't delved into the code, but I've used it and it does a good job.	[reply]
Re: make-like dependency tools in perl by adrianh (Chancellor) on Jun 24, 2005 at 13:26 UTC
Is there anything better out there that I've missed? Express your dependency graph as a state machine and use something like FSA::Rules ?	[reply]
Re: make-like dependency tools in perl by toma (Vicar) on Jun 25, 2005 at 04:59 UTC
I've used Algorithm::Dependency::Ordered for this sort of thing. If you can show a specific example of what it doesn't do, perhaps someone will suggest a solution. Rebuild the dependency graph when something changes instead of using rules for updating the graph. Incremental graph-update functions are certainly possible to write, but many graph modules don't support it. "Age/change state" can be checked with ordinary date modules or functions. Do you need something else? How many dependencies do you have? If you have more than about 50,000 you may have a difficult job. If you have more than about 3,000,000 it can be very tough. It should work perfectly the first time! - toma	[reply]
Re^2: make-like dependency tools in perl by bsb (Priest) on Jun 25, 2005 at 05:52 UTC
From my cursory inspection, Algorithm::Dependency didn't seem to buy me much over Graph. I'd still be handling the execution and the graph building (or using the format of A::D::Source::File). Maybe I should have a second look, since you think it may be useful. "Age/change state" can be checked with ordinary date modules or functions. Do you need something else? The only unusual requirement here is that I should be able to source the date from a database instead of simply from a file's age. Another option in build systems is comparing md5 hashes for changes, but I haven't needed that myself. My dependency graphs have all been quite small an simple. Fewer than 20 nodes, just managing task order and execution.	[reply]
Re: make-like dependency tools in perl by schwern (Scribe) on Jun 25, 2005 at 01:16 UTC
AI::Prolog?	[reply]
Re^2: make-like dependency tools in perl by bsb (Priest) on Jun 25, 2005 at 06:29 UTC
Maybe if sufficiently encapsulated.	[reply]
Re: make-like dependency tools in perl by nothingmuch (Priest) on Jun 25, 2005 at 21:43 UTC
For general dependency solving, stvn and I wrote Algorithm::Dependency::Objects because it's more convenient, at least in our opinion, than Algorithm::Dependency. For dependency driven action composition, somewhat like Java's and in a perlish set of metaphors, we also wrote Verby. Have fun! -nuffin zz zZ Z Z #!perl	[reply]