Perhaps others will have some solution to your LWP question--I have not found anything helpful there. I looked deep into some of the dependencies of the package and found nothing at all addressing the IPv4/IPv6 protocols, so as far as I can tell, LWP is IPv4/6 agnostic, and knows nothing about them.

I suspect your problem is deeper than Perl, and has more to do with the underlying OS and network system. From the man pages of wget, I note the following points:

   Download Options
       --bind-address=ADDRESS
           When making client TCP/IP connections, bind to ADDRESS on t
+he local machine.  
           ADDRESS may be specified as a hostname or IP address.  This
+ option can be useful if
           your machine is bound to multiple IPs.

       --bind-dns-address=ADDRESS
           [libcares only] This address overrides the route for DNS re
+quests. If you ever need to circumvent 
           the standard settings from /etc/resolv.conf, this option to
+gether
           with --dns-servers is your friend.  ADDRESS must be specifi
+ed either as IPv4 or IPv6 address.  
           Wget needs to be built with libcares for this option to be
           available.

       --dns-servers=ADDRESSES
           [libcares only] The given address(es) override the standard
+ nameserver addresses,  e.g. as 
           configured in /etc/resolv.conf.  ADDRESSES may be specified
+ either as
           IPv4 or IPv6 addresses, comma-separated.  Wget needs to be 
+built with libcares for this 
           option to be available.



       -4
       --inet4-only
       -6
       --inet6-only
           Force connecting to IPv4 or IPv6 addresses.  With --inet4-o
+nly or -4, Wget will 
           only connect to IPv4 hosts, ignoring AAAA records in DNS, a
+nd refusing to connect
           to IPv6 addresses specified in URLs.  Conversely, with --in
+et6-only or -6, Wget 
           will only connect to IPv6 hosts and ignore A records and IP
+v4 addresses.

           Neither options should be needed normally.  By default, an 
+IPv6-aware Wget will use      
           the address family specified by the host's DNS record.  If 
+the DNS responds
           with both IPv4 and IPv6 addresses, Wget will try them in se
+quence until it finds one 
           it can connect to.  (Also see "--prefer-family" option desc
+ribed below.)

           These options can be used to deliberately force the use of 
+IPv4 or IPv6 address families 
           on dual family systems, usually to aid debugging or to deal
+ with broken
           network configuration.  Only one of --inet6-only and --inet
+4-only may be specified at the 
           same time.  Neither option is available in Wget compiled wi
+thout IPv6
           support.

       --prefer-family=none/IPv4/IPv6
           When given a choice of several addresses, connect to the ad
+dresses with 
           specified address family first.  The address order returned
+ by DNS is used without change
           by default.

           This avoids spurious errors and connect attempts when acces
+sing hosts that resolve to 
           both IPv6 and IPv4 addresses from IPv4 networks.  For examp
+le, www.kame.net
           resolves to 2001:200:0:8002:203:47ff:fea5:3085 and to 203.1
+78.141.194.  When the preferred 
           family is "IPv4", the IPv4 address is used first; when the 
+preferred
           family is "IPv6", the IPv6 address is used first; if the sp
+ecified value is "none", 
           the address order returned by DNS is used without change.

           Unlike -4 and -6, this option doesn't inhibit access to any
+ address family, it only 
           changes the order in which the addresses are accessed.  Als
+o note that the
           reordering performed by this option is stable---it doesn't 
+affect order of addresses 
           of the same family.  That is, the relative order of all IPv
+4 addresses and of
           all IPv6 addresses remains intact in all cases.
[download]

Note the several references to "DNS" in the above. It may help to check your DNS settings, and if you are only directing the script to a single IP, you might try adding an entry to it in your local /etc/hosts file in order to remove the delay in DNS lookup.