When is the libgcc_s dependency even necessary?

Good point.
I wasn't sure if invoking -static-libgcc would work if perl had not been built with the same option - but I've just run a quick check (see below) and it seems that there's no problem in that regard.
So if the OP wants to distribute binaries compiled with gcc-4, but doesn't want to distribute the dll, then that's the way to go.

Actually, there's sometimes a dependency on a second dll (libstdc++-6.dll) so, to avoid having to distribute that dll, you'd also invoke -static-libstdc++.
These are linker flags, so they need to be given to $Config{ld}, which is g++ on Mingw-built perls.
As a quick test I ran:

use Inline C => Config =>
  BUILD_NOISY =>1,
  LD => 'g++ -static-libgcc',
  INC => '-IC:/MinGW/msys/1.0/local/include';

use Inline C => <<'EOC';

SV * foo(SV * x, SV * y) {
  return newSVnv(SvNV(x) * SvNV(y));
}

EOC

$x = foo(2.3, 1.1);
print $x, "\n";
[download]

Sure enough, the script ran fine and objdump revealed that the dll that was built had no dependency upon libgcc_s_dw2-1.dll:

C:\_32\pscrpt\inline>objdump -x _Inline/lib/auto/try_pl_0a36/try_pl_0a
+36.dll | grep "DLL Name"
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: perl516.dll
[download]

Whereas, if I commented out the LD => 'g++ ... assignment I then got

C:\_32\pscrpt\inline>objdump -x _Inline/lib/auto/try_pl_0a36/try_pl_0a
+36.dll | grep "DLL Name"
        DLL Name: libgcc_s_dw2-1.dll
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: perl516.dll
[download]

Cheers,
Rob