Re: Loading a part of the file to array using Tie::File (don't)

in reply to Loading a part of the file to array using Tie::File

But this could eat a lot of memory, as the files are huge. So I found Tie::File online, that happens not to load arrays in memory. 1st question, is that correct ?

Hi ansh007

Apparently Tie::File doesn't fast big files well it slow :D Don't use Tie::File its a toy :)

I hate to be there bearer of bad tidings here, but if you'd ever tried using Tie::File on a 34GB file, you'd never be suggesting it to others. It would take weeks to complete.
Oh, this is sad. I just had the script work on a 1G+ file, and extrapolated the time to about 17hrs of processing for a 34G file.
Using Tie::File on such a huge file -- or any file over a few (single digit) megabytes is stupid. It will use huge amounts of cpu and be very slow.
a file size that is too large to use with Tie::File? 1000 lines/100kB
Please do not suggest the use of Tie::File for use with files bigger than a few tens of megabytes. Using it makes processing such files 10s or 100s of times slower than using normal line-by-line access for no benefit.
Please try this yourself--say read the middle line in a 1 or 2GB file--before suggesting it to others.
Tie::File is slow, to write a 1mb file 137.506 seconds, versus 0.197 seconds without it
Tie::File: Speed and memory issue with large files

Comment on Re: Loading a part of the file to array using Tie::File (don't)

Replies are listed 'Best First'.
Re^2: Loading a part of the file to array using Tie::File (don't) by haukex (Archbishop) on Nov 22, 2017 at 07:18 UTC
Please don't use `<hN>` tags to highlight text, and use something like `<b>`, `<em>`, or `<strong>` instead. `<h1>` and `<h2>` are discouraged in particular.	[reply] [d/l] [select]
Re^3: Loading a part of the file to array using Tie::File (don't) by Anonymous Monk on Nov 22, 2017 at 23:16 UTC
Please don't use `<hN>` tags to highlight text, and use something like `<b>`, `<em>`, or `<strong>` instead. `<h1>` and `<h2>` are discouraged in particular. Is my message on point? Is it? I think you should abstain from trolling anyone, esp by consideration, about your personal preferences on formatting, Its not as if there aren't a myriad of options open to you on how to deal, but you chose the most heavyhanded/fascist one, consideration Oh look somebody is shouting an answer in a conversation, we can't have shouting in conversations on the internet, textual shouting, thats MORE INSANE THAN ALLCAPS insane insane insane insane insane insane hN tags is the most direct way to get relative emphasis effect	[reply] [d/l] [select]
Re^4: Loading a part of the file to array using Tie::File (don't) by haukex (Archbishop) on Nov 23, 2017 at 08:42 UTC
Yep, this is pretty much exactly the reaction I was expecting ;-) Is my message on point? The parent AM post makes an excellent point about the inefficiency of Tie::File, which is why it's unfortunate that the particular formatting was chosen - the points would have been just as valid if not "shouted". (And IMO "shouting" as Anonymous Monk doesn't help.) your personal preferences on formatting I wouldn't have posted if it was just my personal preference, but I linked to some site guidelines that recommend against `<h1>` and `<h2>`, here's another link. The rest of your post is of course just trolling, which is again unfortunate. You can always register an account so you can build a reputation, so that shouting isn't necessary, or at least so that you can edit your posts.	[reply] [d/l] [select]
A reply falls below the community's threshold of quality. You may see it by logging in.

In Section Seekers of Perl Wisdom