in reply to Convert numbers from a binary file to texts

I expected a text file that I can understand.

You've explicitly asked perl convert each byte to a number, so you're expectations were naive.

A quick look on the web turns up this page which identifies:

  1. The first four bytes are the signature: '.sff';

    Which you have.

  2. The next four byte form an unsigned 32-nit integer that is the version: Apparently always 1.

    Which you have.

  3. The "index offset" and "index length", which it doesn't state the size, but from the example, the offset must be at least 4 bytes and the index at least two. Either or both could also easily be 8 bytes. Especially the offset.

    You have 14 zeros; which perhaps means your file doesn't contain an index.

  4. Lots more fields.

    You haven't shown us the data.

The point of that was to make it clear that you cannot "convert binary to text" unless you understand what the binary is.

That means you need to know:

And that is true regardless of whether you read the file using Perl, C, or any other language.

As an aside, I'd bet that -- given the information required -- a Perl script can decode one of these files just as quickly (within a few percent) as a program written in C.

And it would be a whole heap faster to write the Perl version. (Admittedly, this is the type of thing I've been doing week in, week out for a decade or more. :)

(Also, if you post a link to a file in this format, I might have a go at decoding it. I've been sitting here waiting for a simulation to complete for 3 days now, and I'm a little bored.)

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: Convert numbers from a binary file to texts
by FreeBeerReekingMonk (Deacon) on Mar 27, 2016 at 10:31 UTC
    @BrowserUK: Just updated my post with the structure, if you want to have a go at it...
[reply]

      Totally untested Partly tested; ignoring any index if it is present as the spec doesn't mention it; and making no attempt to present the contain data in any useful way since I don't know what it represents; but it should be pretty close:

      #! perl -slw
use strict;

my $fname = $ARGV[ 0 ];

open IN, '<:raw', $fname or die $!;

read( IN, my $header, 31 );
my( $sig, $ver, $iOff, $iLen, $reads, $hLen, $kLen, $flows, $fmt ) = u
+npack 'a4 N Q N N n n n C', $header;

print "sig: $sig ver:$ver iOff:$iOff iLen:$iLen reads:$reads hLen:$hLe
+n kLen:$kLen flows:$flows fmt:$fmt";

read( IN, my $varHead, $hLen - 31 );

my( $flowchars, $keySeq ) = unpack "a$flows a$kLen", $varHead;

print "flowchars: $flowchars\nkeySeq:$keySeq\n";

for my $read ( 1 .. $reads ) {
    my $buffer;
    read( IN, $buffer, 16 );
    my( $hLen, $nLen, $bases, $cqLeft, $cqRight, $caLeft, $caRight ) =
+ unpack 'n n N n n n n', $buffer;

    read( IN, $buffer, 8*int( ( $nLen + 7 ) / 8 ) );
    my $name = unpack "a$nLen", $buffer;

    read( IN, $buffer, 2*$flows );
    my @signal = unpack "n$flows", $buffer;

    read( IN, $buffer, $bases );
    my @posns = unpack "C$bases", $buffer;

    read( IN, $buffer, $bases );
    my @scores = unpack "C$bases", $buffer;

    read( IN, $buffer, 8*int( ( 4*$bases + 7 ) /8 ) ); ## discard padd
+ing

    ## do something useful with data here.
    print "Name: $name\nSignal: [ @signal ]\nPosns: [ @posns ]\nScores
+: [ @scores ]\n";
    last if eof( IN );
}

      [download]

      Output from the partial file in the OP:

      Read more... (61 kB)
      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]

      That's good. All I need now is an actual file to test it on?

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom