It looks to me like you may not really need to keep track of what you've already done at all. Simply keeping track of when you last ran and then examining all files which have been modified since then should accomplish the same thing while also being substantially simpler.

Caveats:

1) This assumes that the time spent in each processing run is minimal. If it takes 5 minutes, then, yeah, it won't work because the logs are probably going to change during that time.

2) If your situation is such that you need to provide 'proof' that certain files have been processed at certain times then, of course, you'll need the records to satisfy those requirements.