Your technique is fine, not silly at all. A list of files isn't that big a memory hog, and that procedure prevents bad interactions between &wanted and the processing you do.

For instance, if your processing created backups in a yet unexamined directory, you would find yourself making backups of the backups, perhaps without end.

After Compline,
Zaxo