On some operating systems, you need to actually close and reopen the file in order for it to see that the EOF has moved. You can use stat while the file is closed, to see whether the file has grown, hence whether it's worth opening it and reading the next chunk.

Hope ths helps.