This is an issue your operating system has to care about, and it usally does take care.

If you are moving the files inside of one single partition, the data is not moved at all, just a bit of metadata, which is blazingly fast.

The only thing you might have to worry about are network file systems over a lagging connection.