Actually, I think that using a database would not be the best approach to solve this problem. The problem is where the OP says:

In terms of relational databases it would mean one has to keep track of rows that have already been processed or 'next' rows or 'last occurrence' rows. Of course it can be done, but as far as I know it is going to be an ugly mess using row counters that each database technology handles differently.

Besides that using relational database technologies the rule that the OP states: Table A.Column 5 - COUNTER == 1 does not give one row as result but five (marked with *):

1,02 29031990 feb 21 5,         4,Unit4877 29031990 02 1
2,02 29031990 feb 18 5,         3,Unit4876 29031990 02 2
3,02 29031990 feb 20 5,         2,Unit4875 29031990 02 3
4,02 29031990 feb 21 5,         1,Unit4874 29031990 02 4
4,02 29031990 feb 21 5,         1,Unit4874 29031990 02 4 *
4,02 29031990 feb 21 5,         2,Unit4875 29031990 02 3 *
4,02 29031990 feb 21 5,         3,Unit4876 29031990 02 2 *
4,02 29031990 feb 21 5,         4,Unit4877 29031990 02 1 *
4,02 29031990 feb 21 5,         5,Unit4878 29031990 02 0 *
[download]

The only way to solve that would be something like WHERE (TABLE A.COL5 - COUNTER_A = 1) AND (TABLE A.COL5 = COUNTER_B). Again that means using row counters and on top of that now I am making assumptions about the data and I actually don't want to go there.

I have seen a couple of examples that EBK has posted and the only thing that I see is that he is being provided with very low quality data. I understand that he is trying to handle it the best way he can but I think that the solution should be coming from the organisation who are providing him this data.

Here is some code:

use strict ;
use warnings ;

# Test script to put all data next to each other from two tables and a
+pply some filters (Similar to how a join would do that)

# data from perlmonks: 1216134

my @d1 = (
['02',29031990,'feb',21,5],
['02',29031990,'feb',18,5],
['02',29031990,'feb',20,5],
['02',29031990,'feb',21,5]
) ;
my @d2 = (
['Unit4874',29031990,'02',4],
['Unit4875',29031990,'02',3],
['Unit4876',29031990,'02',2],
['Unit4877',29031990,'02',1],
['Unit4878',29031990,'02',0]
) ;
for ( my $i = 0 ; $i < 4 ; ++$i ) {
    my @d1d = @{$d1[$i]} ;
    for ( my $j = 0 ; $j < 5 ; ++$j ) {
        my @d2d = @{$d2[$j]} ;
        if ( ($i + 1) == $d2d[3] ) {
            print( ($i + 1) . ",@d1d,\t\t" . ($j + 1) . ",@d2d\n" ) ;
        }
        if ( $d1d[4] - ( $i + 1 ) == 1 ) {
        #  WHERE (TABLE A.COL5 - COUNTER_A = 1) AND (TABLE A.COL5 = CO
+UNTER_B):
        # if ( ( $d1d[4] - ( $i + 1 ) == 1 ) && ( $d1d[4] == ( $j + 1 
+) ) )  {
            print( ($i + 1) . ",@d1d,\t\t" . ($j + 1) . ",@d2d (x)\n" 
+) ;
        }
    }
}
__END__
1,02 29031990 feb 21 5,         4,Unit4877 29031990 02 1
2,02 29031990 feb 18 5,         3,Unit4876 29031990 02 2
3,02 29031990 feb 20 5,         2,Unit4875 29031990 02 3
4,02 29031990 feb 21 5,         1,Unit4874 29031990 02 4
4,02 29031990 feb 21 5,         1,Unit4874 29031990 02 4 (x)
4,02 29031990 feb 21 5,         2,Unit4875 29031990 02 3 (x)
4,02 29031990 feb 21 5,         3,Unit4876 29031990 02 2 (x)
4,02 29031990 feb 21 5,         4,Unit4877 29031990 02 1 (x)
4,02 29031990 feb 21 5,         5,Unit4878 29031990 02 0 (x)
[download]