in reply to Re^2: most efficient way to scrape data and put it into a tsv file
in thread most efficient way to scrape data and put it into a tsv file

Perl buffers I/O by default. Files are block buffered, stdout is line-buffered to a terminal and block buffered when redirected, and stderr is unbuffered. This means waits until it sees the end of a line or the buffer to do an operating system write.

Are you writing your output to STDOUT or to a file? It sounded like the latter. Setting $| = 1 only unbuffers STDOUT. It is possible to unbuffer files, but this mainly used for logs where each line should be independent.

  • Comment on Re^3: most efficient way to scrape data and put it into a tsv file
  • Download Code

In Section Seekers of Perl Wisdom