in reply to Re: Getting data from second file, based on first files contents;
in thread UPDATED - Getting data from second file, based on first files contents;

here is some sample data:

file1.txt
123 456 789
file2.txt
123 string 1 111 string 1 script should skip this line 222 string 1 333 string 1 456 string 2 444 string 2 it should skip this line as well 555 string 2 666 string 2 789 string 3 777 string 3 also skipping this line too 888 string 3 999 string 3
and the stuff that gets extracted from file 2 are based off of file1's contents. iIt takes the data from file 1 and gets the first match it finds in file 2, then gets the right side column and compasres that against the entire file.

Output:
123 string 1 111 string 1 222 string 1 333 string 1 456 string 2 444 string 2 555 string 2 666 string 2 789 string 3 777 string 3 888 string 3 999 string 3
Thanks for the tips btw :)
Will updated OP

Replies are listed 'Best First'.
Re^3: Getting data from second file, based on first files contents;
by kcott (Archbishop) on Oct 29, 2015 at 17:43 UTC

    The following achieves what you want with just one pass over file1.txt and two passes over file2.txt.

    #!/usr/bin/env perl use strict; use warnings; use autodie; my ($ref_file, $data_file) = qw{pm_1146340_file1.txt pm_1146340_file2. +txt}; my (%ref_left, %ref_right, @output); open my $ref_fh, '<', $ref_file; while (<$ref_fh>) { chomp; undef $ref_left{$_}; } close $ref_fh; open my $data_fh, '<', $data_file; while (<$data_fh>) { my ($left, $right) = split ' ', $_, 2; next unless exists $ref_left{$left} and not defined $ref_left{$lef +t}; ++$ref_left{$left}; ++$ref_right{$right}; } seek $data_fh, 0, 0; while (<$data_fh>) { my ($left, $right) = split ' ', $_, 2; next unless $ref_right{$right}; push @output, $_; } close $data_fh; print for @output;

    Output:

    123 string 1 111 string 1 222 string 1 333 string 1 456 string 2 444 string 2 555 string 2 666 string 2 789 string 3 777 string 3 888 string 3 999 string 3

    If the data in file2.txt is always ordered as shown, i.e. references to file1.txt data always appear first, such as

    123 string 1 111 string 1

    and never as

    111 string 1 123 string 1

    you'll only need one pass over file2.txt.

    To more fully test your code, I'd completely jumble up file2.txt and then add additional records, such as

    123 string 4 111 string 4

    The output should be the same with no instances of "string 4" appearing at all.

    Update: I took my own advice (re "To more fully test your code, ...") and found a problem. I have fixed this by making changes to the first and second while loops. The original code is in the spoiler below.

    — Ken

      yeah it SHOULD match 'string 4' (on all occurences in file2) IF it contains any lines from file1.txt. if you put '123 string 4' inside of file 2, then it should take '123' from file one, and match the same '123' in file2. then you get the value directly to the right of the match (in file2) and compare it with the whole of file2, if the value to the right of '123' is 'string 4' then it will most def need to match if 123 is in file1, which obviously is. in essence you are trying to filter the file2.txt and you could say it could be like a database or something. anyway thanks for post replying :)

        I didn't go into details in my Update:, but what I considered to be the problem was that it DID match 'string 4'.

        I posted the original code with my update. That's probably what you want.

        — Ken