Since it is a common case, I'll note that if the third_party_code() is a use statement then to make the test useful you'll need to code it in either of the following two ways.

is( $^W, 1, 'warnings are on' );
require ThirdPartyModule;
ThirdPartyModule->import;
is( $^W, 1, 'warnings are still on' );
[download]

or

BEGIN{is( $^W, 1, 'warnings are on' )}
use ThirdPartyModule;
BEGIN{is( $^W, 1, 'warnings are still on' )}
[download]

This is because use executes at compile time, so if you write the naive version then the third party code executes before either of your tests.

It is likely to be less confusing to use the former version because otherwise these tests will run before other tests in your test suite.