Re: Lower-casing Substrings and Iterating Two Files together

If you bitwise or (|) an uppercase letter with a space, (assuming latin-1/ASCII files), it will lowercase it:

print 'ACGT' | '    ';;
acgt
[download]

So, if you translate all the 'N's in your mask to spaces and then bitwise or the sequence and the mask, it will achieve your goal very efficiently:

$s = 'GGTACACAGAAGCCAAAGCAGGCTCCAGGCTCTGAGCTGTCAGCACAGAGACCGAT';;
$m = 'GGTNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNT';;

( $mm = $m ) =~ tr[N][\x20];;
print $mm;;
GGT                                                    T

print $s | $mm;;
GGTacacagaagccaaagcaggctccaggctctgagctgtcagcacagagaccgaT
[download]

Which makes your entire program (excluding the unmentioned fact that your files may be in FASTA format):

#! perl -slw
use strict;

open SEQ,  '<', 'data1.dat' or die $!;
open MASK, '<', 'data2.dat' or die $!;

while( my $seq = <SEQ> ) {    ## Read a sequence
    my $mask = <MASK>;        ## And the corresponding mask
    $mask =~ tr[N][ ];        ## Ns => spaces
    print $seq | $mask;       ## bitwise-OR them and print the result 
}
close SEQ;
close MASK;
[download]

Redirect the output to a third file and you're done.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re: Lower-casing Substrings and Iterating Two Files together Select or Download Code

Replies are listed 'Best First'.
Re^2: Lower-casing Substrings and Iterating Two Files together by tilly (Archbishop) on Dec 27, 2008 at 15:01 UTC
Were I maintaining such code I would definitely want a comment along the lines of: `# ord(" ") = 32 and flipping that bit lower cases ASCII.` [download] Otherwise++.	[reply] [d/l]
Re^3: Lower-casing Substrings and Iterating Two Files together by BrowserUk (Patriarch) on Dec 27, 2008 at 16:14 UTC
ord(" ") = 32 If it were my code, I think `ord(" ") = 0x20` or even `ord(" ") = 0b0010_0000` might be clearer, but that's essentially why I rarely comment the code I post. We tend to write comments that mean something to us, rather than others. My antisipation (hope!) is that what I post will rarely be used verbatim in other peoples programs. Indeed, I quite often leave the code deliberately shy of being a complete solution to the OPs question so that they will have to adapt it to their use. With the intent that anyone basing their code upon my postings, will first play with it a little to ensure that they understand it, and then adapt it to their needs. At that point they can adjust the coding style to their preferences or local requirements, and add whatever comments they think most appropriate. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l] [select]
Re^4: Lower-casing Substrings and Iterating Two Files together by tilly (Archbishop) on Dec 27, 2008 at 18:02 UTC
Any of those comments would be better than none. Because right now there is no way from the code to understand that you meant to upper case letters. Without which knowledge it would be hard to tell whether you intended G and T to give you W or whether that would be an accident.	[reply]
Re^5: Lower-casing Substrings and Iterating Two Files together by BrowserUk (Patriarch) on Dec 27, 2008 at 19:01 UTC
Re^2: Lower-casing Substrings and Iterating Two Files together by neversaint (Deacon) on Dec 29, 2008 at 05:32 UTC
Dear BrowserUK, Your magical approach is at least 6980x faster than mine! My approach: `191.52user 153.47system 5:49.44elapsed 98%CPU` [download] BrowserUk Perl's approach (modified to handle FASTA): `open SEQ, '<', $ARGV[0] or die $!; #plain open MASK, '<', $ARGV[1] or die $!; #hardmask while ( my $seq = <SEQ> ) { ## Read a sequence my $mask = <MASK>; ## And the corresponding mask if ( $mask =~ /^>/ ) { print "$seq"; } else { $mask =~ tr[N][ ]; ## Ns => spaces print $seq \| $mask; ## bitwise-OR them and print the re +sult; } } close SEQ; close MASK;` [download] Takes only this much time: `0.45user 0.07system 0:05.59elapsed 9%CPU` [download] This is tested on 12MB dataset, and the breakdown of sequence length is this: `#Seq_name #Seq_len 2-micron 6318 MT 85779 I 230208 VI 270148 III 316617 IX 439885 VIII 562643 V 576869 XI 666454 X 745745 XIV 784333 II 813178 XIII 924429 XVI 948062 XII 1078175 VII 1090946 XV 1091289 IV 1531918` [download] Update: These test datasets (full and hardmasked) can be downloaded here. --- neversaint and everlastingly indebted.......	[reply] [d/l] [select]


P is for Practical
	PerlMonks