Yes, with and extends are just plain old exported subs; they don't execute at compile time unless you wrap them in BEGIN { ... }.

An alternative is to define your attribute in UNIVERSAL, so you don't need to inherit or import it explicitly. Defining attributes in UNIVERSAL is quite commonly done, and not as awful as it sounds.