After playing with your code and sample data for a bit, I had to conclude that the "BETWEEN ... AND ..." syntax is actually not supported by DBD::CSV. You can get the expected output by changing the query to:

SELECT * FROM perfmon WHERE Event_ID >= 7034 AND Event_ID <= 7037
[download]

(I'm not sure whether the "BETWEEN" values themselves are supposed to be part of the output set -- maybe you want ">" and "<".)

Here's a slightly cleaned up version of the OP code:

use strict;
use DBI;

my $query_string = "SELECT * FROM perfmon WHERE Event_ID >= 7034 and E
+vent_ID <= 7037"; 
my $csv_file_name = "perfdata.csv";
my $csv_dbh;
my $stmt_handle;

$csv_dbh =
    DBI->connect( "DBI:CSV:", undef, undef,
                  {
                      csv_eol => "\n",
                      csv_quote_char => "'",
                      csv_tables => { perfmon => { file => $csv_file_n
+ame }},
                      RaiseError => 1,
                  }
    ) or die $!;

$stmt_handle = $csv_dbh -> prepare( $query_string );
$stmt_handle -> execute();

while ( my @row = $stmt_handle->fetchrow_array() ) {
    my $rowstr = join(";", @row);
    print $rowstr,"\n";
}
$stmt_handle -> finish();
$csv_dbh -> disconnect();
[download]

(The "csv_quote_char" spec seems appropriate given the sample data you posted; the "csv_eol" I'm using might be specific to my own download of the data.)