in reply to Re: Re: Re: Re: File I/O
in thread File I/O

...often mandated by inferior programmers....

Ouch! I can feel the wrath of a zillion mainframe programmers aiming lightening bolts in your general direction...or was that private parts? :)

Funny thing is, if your filesystem supports transparent compression (as many mainframe file subsystems, and even my PC do), the "wasted space" per record amounts to 2 or 3 physical bytes per record in many cases. A filler byte plus a repetition count. Remove the need for a record separator and your almost at a status quo. Removing the need for field separators and your often into into positive ground.

Add the fact that you don't have to parse to find the fields and the performance can be a vast improvement over CSV and the like, never mind XML

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.

Replies are listed 'Best First'.
Fixed-length records (Re: File I/O)
by jonadab (Parson) on Sep 23, 2003 at 23:59 UTC

    The big problem with fixed-length records IMO is that real-world data will always find ways to be longer than the fixed length. 

    $;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}}
split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/

    [download]
[reply]
[d/l]

      Exactly what I mean when I say they dont scale well.


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi
[reply]
[d/l]
Re: Re: Re: Re: Re: Re: File I/O
by demerphq (Chancellor) on Sep 24, 2003 at 00:12 UTC

    Actually I'm not real sure if I agree with you here. The arguments you have made don't gibe with my understanding of fixed width, and non fixed width. If the filesystem is doing transparent compression, then the file system is in fact a stream and you are adding a layer of work (no doubt handled by subsidary controllers or whatnot) to then represent the data at a logical fixed width format. Presumably if someone thought the cycles weren't worth wasting doing CSV handling (or XML for that matter) they could also offload the task to subsidiary hardware, and we are back at point one, which is that fixed width records dont scale well. :-) The only argument left for fixed width records is they dont care what value is within them, but this also goes for at least one form of delimited files (length delimited fields) which also scale well.

    BTW, im not saying that fixed width doesnt suit some circumstances. Just that I dont think those circumstances are really that common. (These days. :-)


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi
[reply]
[d/l]

      You might also say that "a stream" is a purely logical view that is an artificial superimposition atop an inherently block structured device but that would be ... erm ... correct!

      I've no particular axe to grind, sometimes fixed width fits the data or application best, sometimes variable width does. I don't dismiss the use of a quick sort algorithm because it doesn't scale well above memory size.

      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
      If I understand your problem, I can solve it! Of course, the same can be said for you.
[reply]

In Section Seekers of Perl Wisdom