The modification time does not "take over the timestamp" of the latest changed or added file. The modification time of a directory only changes when a file is added or removed.