Store access control metadata, probably in a database correlated with the file by however you uniquely identify it in your system.