in reply to Re^2: Working with large amount of data
in thread Working with large amount of data

(My NTFS is limited to 2GB file size).

Huh? NTFS has been able to handle individual files up to a gnat's todger under 16TB forever. (At least since NT4 days; and I believe since 3.51). Even FAT16 & FAT32 can handle 4GB.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP PCW It is as I've been saying!(Audio until 20090817)
  • Comment on Re^3: Working with large amount of data

Replies are listed 'Best First'.
Re^4: Working with large amount of data
by Marshall (Canon) on Sep 20, 2009 at 23:58 UTC
    At the "end of the day", if the file size (pointers) in the OS's file system are limited to 32 bits, then you can only do what 32 bits can do. 2**32-1=4,294,967,296.

    Anyway 2GB vs 4 GB is WAY off from 1 TB! Update: Windows NTFS file pointers are NOT limited to 32 bits, even on a 32 bit systems. So the above is "right", but still "wrong".

[reply]

      Update: This is (irrelevant) the article that was originally cited in the post above!

      Sorry, but I'm not sure how an article describing the 4GB limit of files on FAT32 partitions is relevant to a claim that NTFS can only handle 2GB files?

      For the real skinny, go to the source. Besides which I used files far greater than 2GB on my old thinkpad running NT4/NTFS. From memory the largest I ever created was 32GB, but when I first got my current machine I used contig to create a 500GB contiguous file just for the fun of it. I don't have enough diskspace to repeat the process, but I did just create one 16GB.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
        I have updated previous posts. Windows NTFS can have a file >4GB.

        However interesting these details are, the main point remains:
        -why would you process a TB file on a wimp "laptop" machine?
        -where does this file come from?
        -why wouldn't this be a DB report?

[reply]

In Section Seekers of Perl Wisdom