powerhouse has asked for the wisdom of the Perl Monks concerning the following question:

We are using MySQL and Perl to program our website, we have it all done, but now we are working on the payroll, when we send out checks using our Automatic Data Processing company, we get an excel list of all the checks (the account identifier, the check number and the amount of the check), we then have to go update the database to change the record from preissued to issued, the check number and timestamp.

The problem is that if the employee had 3 or more checks (they are combined into one check by the payroll processor), I cannot find the record that shows preissued because I have to do a search this way:
my $_checkNo = 0; my $_checkAmt = 1; my $_emp_id = 2; my $_paydate = 3; my @fileparts = split(/\,/, $_line); # Get potid from pot(Pay Out Tracking) table my $_potid = $dbh->selectrow_array(qq{SELECT `potid` FROM `pot` WHERE +`status` = "preissue" AND `amount` = ? AND `adpid` = ?}, undef, $file +parts[$_checkAmt], $fileparts[$_emp_id]); if(!$_potid) { # ok, the amount does not match any, so I'm sure they had more tha +n one transaction.... Check for two: ($_potid,$_chkamt) = $dbh->selectrow_array(qq{SELECT `potid`,`amou +nt` FROM `pot` WHERE `status` = "preissue" AND `adpid` = ?}, undef, $ +fileparts[$_emp_id]); if($_potid) { $_newChkAmt = sprintf('%.2f', $fileparts[$_checkAmt] - $_chkam +t); $_potid2 = $dbh->selectrow_array(qq{SELECT `potid` FROM `pot` +WHERE `status` = "preissue" AND `amount` = ? AND `adpid` = ?}, undef, + $_newChkAmt, $fileparts[$_emp_id]); } if($_potid && $_potid2) { $inserted = $dbh->do(qq{UPDATE `pot` SET `status` = "issued", +`checkno` = ?, `statuschanged` = ? WHERE (`potid` = ? OR `potid` = ?) +}, undef, $fileparts[$_checkNo], $fileparts[$_paydate], $_potid, $_po +tid2); } if($inserted) { $_success++; } else { $_failed++; } }
That will pick up only one or two... I just wrote that code to show you what I'm doing, but there is a lot more code, that is not the complete code or the clean version of it so forgive my errors that I put... the code works now, but it excludes paychecks that have 3 or more checks combined into one because the amounts won't match.

So my question is this, is there a way with Perl/MySQL to do something like this and make it work:
$dbh->do(qq{UPDATE `pot` SET `status` = "issued", `checkno` = ?, `stat +uschanged` = ? WHERE `status` = "preissue" AND `apdid` = ? AND $filep +arts[$_checkAmt] IN SUM(amount)}, undef, @placeholder); # @placeholder will contain all the values for the ?....
I see the logic in my head but cannot figure out a way to get it to work... what I want it to do is get MySQL to update all the records that have that employee identifier and are matched and the amounts all equal the amount I am giving it.

Please let me know if you don't understand what I'm trying to do, I've been up for about 22 hours programming and am totally exhausted so not thinking 100% clearly.

Thanks for any advice you can offer.
thx,
Richard

Replies are listed 'Best First'.
Re: payroll, mysql and perl question
by graff (Chancellor) on Jan 02, 2008 at 06:47 UTC
    I'm not sure I understand, but I suspect this snippet conveys the core of the matter:
    ... we then have to go update the database to change the record from preissued to issued, the check number and timestamp... if the employee had 3 or more checks (they are combined into one check by the payroll processor)...

    So, if you select "amount" from all records for a given "_emp_id" (the actual table column name for this is "apdid"), where "status='preissued'", the sum of the returned amounts should add up to the amount of the single check, right?

    Given that the "_emp_id/apdid" and "status" fields identify the set of rows to be updated, you don't need to include the amounts as part of the query, do you? I presume it would be a good idea to test and report whether the sum of amounts in the database add up to the amount of a given check, but it would be easier to do this part in perl rather than in sql -- something like this maybe (untested):

    my $get_amt = $dbh->prepare( qq{SELECT SUM(amount) FROM `pot` WHERE `apdid`=? AND `status`='preissue'} ); my $upd_pay = $dbh->prepare( qq{UPDATE `pot` SET `status`='issued', `checkno`=?,statuschanged=? WHERE `apdid`=? AND `status`='preissue' +} ); my @file_fields = qw/checkNo checkAmt empID payDate/; while ( my $_line = <IN> ) { # assuming lines are coming from a file my %lineData = (); @lineData{@file_fields} = split /,/, $_line; $get_amt->execute( $lineData{empID} ); my $rows = $get_amt->fetchall_arrayref; my $sum = 0; for my $row ( @$rows ) { my ( $amt ) = @$row; $sum += $amt; } if ( $sum != $lineData{checkAmt} ) { warn sprintf( "empID %s: mismatch: sum of preissue rows= %s, c +heckAmt= %s\n", $lineData{empID}, $sum, $lineData{checkAmt} ); next; } $upd_pay->execute( $lineData{checkNo}, $lineData{payDate}, $lineDa +ta{empID} ); }
    (update; removed a presumably unnecessary "GROUP BY apdid" clause in the SELECT query)

    You might want more error checks on the DBI calls (unless you have RaiseError set to die on any failure). I would also recommend more error checks on reading the input data (correct number of fields, fields are non-empty, etc).

    Note that the update statement will assign the current checkNo and payDate values (as well as resetting the status to "issued") for all the "preissue" rows for a given empID. This is a normal way of doing multi-row updates.

    Also note that the statements are prepared once, outside the processing loop. This will save some overhead on the DB server.

    You'll need to work out what really needs to be done when the amount of the check does not match the sum of "preissue" rows for a given employee, but I think the rest of it is basically what you want. (Or it could be that I don't get what you're doing at all.)

      You understood me correctly, however the amount does matter, because it could be that they have other checks 'preissued' by the time we get the file from adp, that has the check numbers.

      So the amounts are relevant, however, it will be the first ones I come to, as I don't upload a later file before an earlier one, so it could be that the amount will equal the FIRST 1 or more but not likely it would be the second and third and fourth but not the first...

      So, what I need to do is something like this, I guess:
      my $_startingAmount = sprintf('%.2f', $fileparts [$_checkAmt]); my @potids = (); my $sth = $dbh->prepare(qq{SELECT * FROM `pot` WHERE `adpid` = ? AND ` +status` = "preissue" ORDER BYE `potid`}); $sth->execute($fileparts[$_emp_id]); while(my $_pot = $sth->fetchrow_hashref()) { if(sprintf('%.2f', $_pot->{amount}) <= $_startingAmount) { push(@potids, $_pot->{potid}); $_startingAmount -= $_pot->{amount}; last if $_startingAmount == 0.00; } next; } $sth->finish(); ## Now go update all the potid records in a foreach statement in @poti +ds
      I think I need to think that through, but it SHOULD work... but what if the payroll company messes up and he amounts do not match up... guess I could keep track of that in the while statement and if I never reach 0.00 set a flag for that record and don't update those records. Hmm, what do you think?

      Thanks again for your input, BTW, the employee identifier is 'adpid' that is the record of the employee file primary key, unique to each employee (Currently we have just shy of 2,000 in that table).

      thx,
      Richard
        ... the amount does matter, because it could be that they have other checks 'preissued' by the time we get the file from adp, that has the check numbers.

        In other words, between the time that a set of rows goes out for check processing, and the time when the check-processing results come back to you for updating the table, there might be additional "preissue" rows that have been inserted for a given "empID". Is that it?

        In that case, you are missing a piece of information, which might be simple to add in the current table structure: the date when a given set of rows were sent out for check processing (i.e. when the status field was set to "preissue"). You appear to have a "statuschanged" field, which apparently holds a date value.

        If that field is set when the status field becomes "preissue", and if a set of check-processing results can be associated with that date, then you can select "preissue" rows with "statuschanged" less-than-or-equal-to the particular date, and you should get the right set of rows (excluding the rows that were added after the current list of payments were sent out for processing).

        Your alternative of looping over records until you reach the payment amount would most likely solve it as well, but seems like more work. (And you would have to fix the spelling: "ORDER BY" -- no final "E")

Re: payroll, mysql and perl question
by igelkott (Priest) on Jan 02, 2008 at 06:58 UTC
    Perhaps just a matter of taste but I'd use separate tables for keeping track of your preissued and issued checks. It's basically like the common practice of separate accounts for debits and credits.

    The advantage is that it's notably easier to group the preissues to match the combined checks. For example, it should make it easier to deal with special situations where the amount paid might (temporarily) not equal the amount owed.

      Actually, that is a very good idea, I just was trying to keep the databases to a minimal, we already have over 350 tables in it, a little much... thx though
      thx,
      Richard
Re: payroll, mysql and perl question
by perrin (Chancellor) on Jan 02, 2008 at 06:41 UTC
    I can't quite tell what you're trying to do with that SQL. When you say you want to "update all the records that have that employee identifier and are matched and the amounts all equal the amount I am giving it", what does "matched" refer to? You can certainly group by employee ID to match a sum you have, but you don't want all of the records matching that ID, right? I suspect you need a subquery here so that you can group records and also update, but I'm not clear enough on what you want to suggest the SQL.