in reply to Displaying/buffering huge text files

I haven't tried it myself yet (at least, not in Perl), but this sounds like a good case for using Mmap. The idea would be that you set yourself a comfortable buffer size (a few meg, or whatever makes sense for the GUI), and you map this amount of the big file into the scalar that gets put into the Text widget.

When the user scrolls the widget to a point outside the current mapping window, remap the scalar to the appropriate offset in the file. Assuming that it actually works, it ought to be pretty fast, and could probably be built into a callback function from the scroller widget.

(Unless of course you happen to be using a system that doesn't support mmap, in which case this is a useless idea.)

  • Comment on Re: Displaying/buffering huge text files

Replies are listed 'Best First'.
Re^2: Displaying/buffering huge text files
by spurperl (Priest) on Feb 23, 2005 at 08:39 UTC
    I have a sinking feeling that mmap is not supported on Windows... I will check it more thoroughly, though, thanks.
      I realize this is probably to late to be be of use for your text-viewing task (which is presumably solved by now), but for future reference, you might want to look up a Windows port of the UNIX environment and tools provided by AT&T Research: http://www.research.att.com/sw/tools/uwin/.

      The ATT "UNIX for Windows" package is called "UWIN" (how clever?), it's free/open-source, and I noticed that the description of features includes:

      Memory mapping and shared memory:
      Both mmap() and the system V shared memory facilities are provided.