Re: designing a program - your wisdom needed

How can I declare the database connection parameters just once?

In general, you can share such data between different scripts by:

Storing them in a configuration file and loading that in each script
Storing them in an environment variable (or several) and reading from that in each script
Storing them as data in a Perl module and using that in each script
(Not applicable here because catch-22) Storing them in a database/LDAP directory/other storage facility and reading from that in each script

Note that here the module option is perhaps best because the module can handle not just shared data but also shared operations such as creating the database connection, handling errors/exceptions, serialising/encoding data, etc. Or you could combine two of them - have a module to handle all this and then that module could read the config file or environment variable, etc.

how can I serialize the execution? such that subsequent scripts execute if and only if the first one executes successfully.

You can do this in any number of ways, but I wouldn't. It is almost never a smart move to call a Perl script directly from another Perl script. Call it an antipattern if you like. Far better to take the second script and put all of its operations in a module instead. The second script then just becomes a wrapper around that module should you ever want to run it independently. The code in the module can be run from any script (or module) which uses it.

Any general wisdom here?

General rule: don't shell out from Perl to run Perl.

🦛

Comment on Re: designing a program - your wisdom needed

Replies are listed 'Best First'.
Re^2: designing a program - your wisdom needed by Marshall (Canon) on Jan 20, 2022 at 12:39 UTC
I think the OP and us may be way overthinking the OP's problem. first.pl and second.pl will create temp table schema and each will run a huge query that loads data into respective temp table schema. third.pl will join the two temp tables from first and second and load it into third. It could very well be that the OP just needs to log onto a network SQL DB and run just 3 SQL commands!? I am not sure at all there needs to be 4 Perl programs for this! It appears that a single Perl program of a couple of pages long will do what the OP is requesting. If not, then I want to know why not? (that's a question for the OP).	[reply]
Re^3: designing a program - your wisdom needed by SpaceCowboy (Acolyte) on Jan 20, 2022 at 21:40 UTC
To have some readability, as the queries are very long but you are right, it can be done with one program. Serial execution will remain a concern though - executing a program only after the previous step is completed successfully	[reply]
Re^4: designing a program - your wisdom needed by Marshall (Canon) on Jan 21, 2022 at 20:16 UTC
As I mentioned before, all programs provide the shell with an exit status. No errors is indicated by 0. The simple thing is to make a batch file and stop executing Perl programs when you get a non zero status code (no matter if negative or positive). You don't mention your OS. For Windows command line: test.bat `ECHO test.bat file starting.... perltest1.pl IF %ERRORLEVEL% NEQ 0 EXIT /B %ERRORLEVEL% perltest2.pl` [download] perltest1.pl `use strict; use warnings; open my $fh, "<", "bogus file" or die "unable to open file $!";` [download] perltest2.pl `use strict; use warnings; print "this is test2";` [download] running test.bat yields: `C:....PerlProjects>perltest1.pl unable to open file No such file or directory at C:......\PerlProject +s\perltest1.pl line 4. C:....PerlProjects>IF 2 NEQ 0 EXIT /B 2` [download] So testperl1.pl returns code of 2. testperl2.pl never runs. There are all sorts of fancy things that can be done with shell scripts. My advice is to keep things simple. Just automate the running of separate commands for each Perl program. Don't use Perl to run Perl programs. And avoid other tricky module oriented syntax. If you have a bash shell on Unix, then there is an equivalent set of statements which will do the same thing. Update: Now if you want to cause a non-zero exit, then "exit(2345);" in your Perl program will return code 2345. Since you are using an Oracle DB, I would just use one of the many INI modules on CPAN to read an INI file with account and password. You will have a small amount of "boiler plate" code to add to each of your programs, but I don't see a problem with that. And I guess you could pass the account and password to each of the programs from the command line. But overall, I think using an INI module is the least complicated way to do what you want.	[reply] [d/l] [select]


good chemistry is complicated, and a little bit messy -LW
	PerlMonks