That's correct. The module is only compiled once, but the imports happen in each namespace where the script is used, and they take up space. You can always simply avoid importing by calling a module with an empty import list, like this:
use Foo::Bar ();