First of all, this

> $spinner->count('benchLoop', __FILE__, __LINE__);

can be reduced to $spinner->count('benchLoop') if you check caller to retrieve file and line inside the count() method.

Choroba's tip to use # line 123 directive is legit, see Plain-Old-Comments

So try to debug where the "strangeness" starts to happen in your file.

:~$ perl
$\="\n";   
print __LINE__;
# line 120
print __LINE__;
__END__
2
120
[download]

> Any idea why __LINE__ would be consistently off by 2?

Like Corion said, any module injecting code, like by using source filters could silently add additional lines.

Regarding your newline theory, try to end your file with __DATA__ and put this into your code right before calling ->count to check what the engine actually sees as different lines before compiling.

seek DATA,0,0;
my @lines = <DATA>;
say Dumper \@lines;
[download]

for instance

use v5.12; 
use warnings;
use Data::Dumper;
say __LINE__;
say __LINE__;
say __LINE__;


seek DATA,0,0;
my @lines = <DATA>;
say Dumper \@lines;

__DATA__
[download]

output

4
5
6
$VAR1 = [
          'use v5.12; 
',
          'use warnings;
',
          'use Data::Dumper;
',
          'say __LINE__;
',
          'say __LINE__;
',
          'say __LINE__;
',
          '
',
          '
',
          'seek DATA,0,0;
',
          'my @lines = <DATA>;
',
          'say Dumper \\@lines;
',
          '
',
          '__DATA__
'
        ];
[download]