Re: Opening a file at a designated offset based on closing of the file

Off topic, but let me get this straight. You have a log file that will grow from now until the end of time? That can make for a pretty large file, and every operating system that I know has a limit on how large a file can get. Do you have any way to rotate the log? We have logs from an app at work that write to a new log once a day, so every day has a new log.

Now, back to the matter at hand. Is there any way that you can uniquely identify your lines in the file? If so, you could (and should) set up a primary key/unique index on the database table that you're inserting in to. This will prevent the duplicate data from ever entering the database, so you'll be guarded on two fronts.

thor

Comment on Re: Opening a file at a designated offset based on closing of the file

Replies are listed 'Best First'.
Re: Re: Opening a file at a designated offset based on closing of the file by zengargoyle (Deacon) on Nov 11, 2003 at 02:17 UTC
i would agree, except i've found logs that look like: `-r-xr-x--- 1 log 12345 Jan 2003 log.1.gz -r-xr-x--- 1 log 13451 Jan 2002 log.2.gz ...` [download] some logs can grow real slow. why rotate daily/weekly/... when yearly or by size will do. guess it just depends...	[reply] [d/l]