Limiting the scope of @INC doesn't help because the following isn't true:

When I try to use some modules with same name as one used with previous servers(VirtualHost), its trying to load from the path added by previous servers (VirtualHost).

If a named module has already been loaded, Perl won't try to load it again. It doesn't get to the point of using @INC.

And if you forced your way around that too, it would still not work since you'd have two modules trying to occupy the same namespace.

Sorry, but an instance of Perl can't handle two modules with the same name.