You don't need super/root permissions to delete a directory, you just need to remove any files and subdirectories in it first.

The standard module File::Path exports the rmtree() function that recursively deletes directories. It's probably the easiest portable way to achieve this.