Thanks to everybody for their good and inspiring solutions!

In the end (or rather, in release 0.03), I went with a different approach alltogether:

When I install the source filter, I also install a fake warnings category experimental::signatures with it. This allows the code using Filter::signatures to also use no warnings 'experimental::signatures';, and more specifically, makes the switch between using the filter and removing all vestiges of this source filter a one-line change. CPAN testers will tell me how bad this idea was in the short run...