As a consequence, the number of tests is likely to explode,

On the other side, I feel like I'm cheating doing all these repetitive tests, and having lots of "ok..." where I should really have one,

It sounds like you've adopted a mindset that looks at the raw numbers at the end of the run, instead of looking at the actual testing.

Having a large number of tests doesn't mean anything.

There's no "cheating" if the number of tests goes up.

There's no sin in re-testing the same thing in multiple files. For example, there's nothing wrong, and indeed everything right, with checking the result of every single constructor call, as in

my $foo = Foo::Bar->new();
isa_ok( $foo, 'Foo::Bar' );
[download]

in every single file. For that matter, there's nothing wrong, and indeed everything right, with check the result for each line of text:

while ( my $line = <> ) {
    my $foo = Foo::Bar->parse( $line );
    isa_ok( $foo, 'Foo::Bar' );
    ...
}
[download]

Don't worry about the test numbers. Keep throwing tests at it.