in reply to Design/Style question about writing to a file from different functions

"... main_func calls sub_func1 and after that sub_func2, which are in other modules"

Why?

If you consider the parameters of "better" to include 'how many times' the solution opens and closes your data file and 'how many modules' you have to load, I submit that refactoring all the scripts using main_funct, sub_func1, sub_func2 and whatever the "add something" may be, so you end up with a single module... at which point your other alternatives become moot.

Replies are listed 'Best First'.
Re^2: Design/Style question about writing to a file from different functions
by Dirk80 (Pilgrim) on Jul 20, 2010 at 19:31 UTC

    Thank you very much for your answers.Now I'm sure that solution 1 is the better solution.

    As the author of a perl script my goal is it that it runs on a computer with much and less memory. So I do not know how much memory I have. But to give you numbers. sub_func1 writes about 50MB to the output file and sub_func2 writes about 150MB to the output file.

    Do you think that holding about 200MB in memory is ok and then writing everything to the output file in one swoop? Or the other alternative would be to write first 50MB to the file and then 150MB. How many MB do you usually collect in memory if you want to use it on different computers?

    Now to the reason why I want to move a line directly after building a part of the file. I'm computing a checksum which I know when sub_func1 finished. But the line with the checksum has to be at the beginning of the file and NOT at the end. My problem was that I thought that it is better to write everything directly to the file and that it is not ok to collect everything in memory. That's why I had the problem to move a line afterwards. If the memory solution is ok, then everything gets easier. Only main_func will then write to the file and the helper functions collect everything in memory.

[reply]
[d/l]
[select]
      I'm computing a checksum which I know when sub_func1 finished. But the line with the checksum has to be at the beginning of the file and NOT at the end.

      Most checksums are (or can be) a fixed length string.

      So, why not write a dummy checksum at the beginning. Write the rest of the file as you generate it. When you've finished writing, rewind seek the file to the beginning and overwrite the checksum. Close and done.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      I would probably use temp files then append the rest to the file with the checksum once everything is done. That way it doesn't have to be stored in the memory while processing and let the OS do it which would most likely be the most efficient. But without specifics it is hard to tell, just poking in the dark here.
[reply]

In Section Seekers of Perl Wisdom