Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^2: Best way to a parse a big binary file

by Dirk80 (Pilgrim)
on Nov 30, 2019 at 19:33 UTC ( [id://11109488]=note: print w/replies, xml ) Need Help??


in reply to Re: Best way to a parse a big binary file
in thread Best way to a parse a big binary file

Thank you very much for your example. I was thinking a lot about this task. The headers and sections are big. Each part contains a lot of fields of different things, e.g. single bytes, integers, floats, doubles, some strings, ... . Because of this I would prefer to have the parse logic in each class.

If I understand it right, I would open the file in the main package, then passing the lexical file handle to the constructor of the first class. Parsing the things for e.g. the MainHeader. Then passing the file handle to the ExtHeader class, ... ? Or is it a bad style to pass a file handle to a constructor?

I have no checksum issue. But would be interesting what would you recommend if I had to compute a checksum over all sections although I would like to have the parse logic in each class instead of one central place

Thanks again in advance for your suggestions. I (nearly) always find a way to do my things with perl. But I want to learn how to solve my tasks with a better design and in a better way. That's why I'm asking.

  • Comment on Re^2: Best way to a parse a big binary file

Replies are listed 'Best First'.
Re^3: Best way to a parse a big binary file
by haukex (Archbishop) on Nov 30, 2019 at 20:47 UTC
    Or is it a bad style to pass a file handle to a constructor?

    No, it's fine, as long as you're using lexical filehandles (open my $fh ...). It only gets difficult if any code that is reading from the filehandle either needs to look back at something that was already read from the file, or needs to look ahead further into the file into a section that is supposed to be parsed by another piece of code - in cases like that, it's usually more appropriate to use an approach similar to what I showed above.

    Because of this I would prefer to have the parse logic in each class.

    Sure, that's fine too. Here's one quick example*:

    But would be interesting what would you recommend if I had to compute a checksum over all sections although I would like to have the parse logic in each class instead of one central place

    Well, if by that you mean you want to checksum the entire file, then probably the above sub parsefile is a good place, perhaps devising a way to keep track of the bytes already read or computing the checksum while reading - like for example, an object that wraps the filehandle and exposes the binread method I showed to read from the file could calculate the checksum as the file is read piece by piece. But in my experience it's more common to see checksums on a per-packet basis, in which case, in the above code, Packet::parse could take over the checksum reading and checking.

    But I want to learn how to solve my tasks with a better design and in a better way.

    * There are a whole bunch of possible variations on the above code. For example, I could've used Moo's features like BUILDARGS to have the constructor do the parsing, instead of a separate sub parse (although the former solution makes it a little more tricky to create packets in code that haven't been parsed from a file). Or, I could have structured the classes differently: If this was like a network protocol and each "packet" has a header, then it would make sense for the class Packet to have a header field that is populated with a corresponding class, instead of having the *Header classes be subclasses of Packet. Or I could've defined a role that requires each class to have a parse method. And so on.

    So in general, the usual software design principles apply: reduce repetition, design your OO "isa" and "has" relationships in a sensible manner, make judicious use of factory methods, and so on. If you feel like something is getting too difficult, then it's best to step back and see if there might be some architecture changes that would help the situation, instead of plowing on, because the more code you write, the more reluctant you'll be to make larger architecture changes.

      Thank you so much for your great reply. It helped me a lot. I could solve my problem in a good way. Like you recommended I built a class around the file handle. In this helper class I also put the endianness and checksum computation. It works fine and is good encapsulated.

      Also before beginning the coding I took a big piece of paper and was drawing all classes and interactions (inheritance/aggregation) between them. And then I could implement my task without the need to change something afterwards. A good design at the beginning helps a lot :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11109488]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2024-04-19 09:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found