There's a line in:

  $PerlDir\site\lib\Win32\OLE\Const.pm
[download]

that reads

  __PACKAGE__->_Typelibs;
[download]

If I comment out this line then it loads instantly, if I uncomment this line, then I get my delay.

Update: it appears that _Typelibs() is defined in

  site\lib\auto\Win32\OLE\OLE.dll
[download]