Rather than trying to calculate a negative offset equating to the first record - and writing a multi-line comment explaining what you're doing - you can just reposition the file pointer to the start of the file with seek.

In the code below, $called_first_row represents what you called $first_row. Perhaps $first_wanted_row, or something similar, would have been a better choice of variable name; $fourth_row would have been completely accurate.

#!/usr/bin/env perl

use 5.010;
use strict;
use warnings;

use Text::CSV_XS;

my $csv_file = q{pm_csv_first_cell.csv};

open my $csv_fh, q{<}, $csv_file or die $!;

my $csv_obj = Text::CSV_XS::->new();

my $called_first_row = $csv_obj->getline_all($csv_fh, 3, 1);

say qq{@{$called_first_row->[0]}};

seek $csv_fh, 0, 0;

my $real_first_row = $csv_obj->getline_all($csv_fh, 0, 1);

say qq{@{$real_first_row->[0]}};

say q{First cell value: }, $real_first_row->[0][0];

close $csv_file;
[download]

Sample run and file contents:

ken@ganymede: ~/tmp
$ cat pm_csv_first_cell.csv
z,x,c,v,b,n
a,s,d,f,g,h
q,w,e,r,t,y
1,2,3,4,5,6
ken@ganymede: ~/tmp
$ pm_csv_first_cell.pl
1 2 3 4 5 6
z x c v b n
First cell value: z
ken@ganymede: ~/tmp
$
[download]

my $a_one = $csv->getline($fh)->[0];
<...>
my $first_row = $csv->getline_all( $fh ,2, 1 ); # decrease offset by 1
+ because one line was read earlier
[download]