a) testing if the directory exists before attempting to create it

b) using the perl built-in mkdir command

c) cache this info. IOW, don't try to recreate a directory you've already created before. The difference between system and mkdir and stat is quite vast, but in this situation it completely dwarfed by the sheer amount of unnecessary calls you'd make. update: this is the important suggestion. you can ignore the rest for this particular problem.

using the perl built-in mkdir command

The Perl equivalent of mkdir -p is mkpath if you don't want to do the chain of mkdir commands. On the other hand, I don't see why -p is needed on any but the first call.

By sorting, you can minimize the calls to mkdir, to open and to close. You can even sort such that the order is preserved in each file if so desired.

#!/usr/bin/perl -w

use strict;

#
# Run the script like:
#    tail -n +2 car.csv | sort | ./foo.pl
#
# Or if you want to preserve order:
#    tail -n +2 car.csv | sort -t, -k1,2 -s | ./foo.pl
#

use File::Path qw( mkpath );

my $out_dir = "/var/tmp/cars";

mkpath($out_dir, 0, 0777) or die;

my $last_make  = '---';
my $last_model = '---';
my $fh;
while (<>) {
   my ($make,$model) = split(/,/, $_, 4);

   if ($make ne $last_make) {
      mkdir("$out_dir/$make", 0777) or die;
      $last_make  = $make;
      $last_model = '---';
   }

   if ($model ne $last_model) {
      $fh = undef;
      open($fh, '>', "$out_dir/$make/$model") or die;
      $last_model = $model;
      $fh = undef;
   }

   print $fh $_;
}
[download]

++ obvious-in-retrospect bright idea :)

{
  my (%handles, %did_mkdir);
  sub get_out_file {
    my ($make, $model) = @_;
    if (!defined $handles{$make}{$model}) {
      if (! $did_mkdir{$make} && ! -d "$out_dir/$make") {
        system "mkdir -p $out_dir/$make";
        $did_mkdir{$make} = 1;
      }
      open $handles{$make}{$model}, "$out_dir/$make/$model" or die $!;
    }
    return $handles{$make}{$model};
  }
}
[download]

while (<>) {
  my ($make, $model) = split /,/, $_, 4;
  my $out = get_out_file($make, $model);
  print $out $line;
}
[download]

use strict;
use warnings;

# Read all input into data structure
my %cars;
my $i = 0;  # unique tag for hash keys
while (<>) {
    chomp;
    my ($make, $model, $rest) = split /,/, $_, 3;
    $cars{$make}{$model}{"$rest,$i"}++;
    $i++;
}

# Create each directory once
# Open/close each file once
my $out_dir = "/var/tmp/cars"; #Output directory for
for my $make (keys %cars) {
    my $dir = "$out_dir/$make";
    system "mkdir -p $dir";
    for my $model (keys %{ $cars{$make} }) {
        open my $fh, '>', "$out_dir/$make/$model" or die $!;
        for my $rest (keys %{ $cars{$make}{$model} }) {
            my @specs = split /,/, $rest, 2;
            print $fh (join ',', ($make, $model, @specs)), "\n";
        }
        close $fh;
    }
}
[download]

Update: On second thought, I think you can preserve input order by using an HoHoA instead of the HoHoH above. That should get rid of the smelly tag I introduced, too.

Can you pipe it in sorted? Then you could hash your data until you get a new make; then write it out and start a new hash.

And are you locked in to that directory structure? Or could you create a Honda.Civic file instead of ../Honda/Civic?

And maybe you can do some of that in parallel: Read in a make; write it to file in a child process; and read in the next make.

In the end, it depends on where your performance is lagging.