my $head = system head => -1 => $in_file; # Grab header/first line of file

system does not return the output of the command, it returns its exit code. There are ways to get the output in Perl*, but I would instead recommend opening and reading the file using the tools built into Perl (see e.g. Files and I/O):

use warnings;
use strict;
my $ENCODING = ":encoding(UTF-8)";

die "Usage: $0 FILENAME\n" unless @ARGV==1;
check_file($ARGV[0]);

sub check_file {
    my ($in_file) = @_;
    open my $fh, "<$ENCODING", $in_file
        or die "Failed to open $in_file: $!\n";
    my $head = <$fh>;
    close $fh;
    if ( $head =~ m/^ABC/ ) {
        print "This is a ABC File. Processing ABC File...\n";
        prcss_abc_file($in_file);
    }
    elsif ( $head =~ m/^ISA~ / ) {
        print "This is an EDI File. Processing EDI File...\n";
        prcss_edi_file($in_file);
    }
    else {
        print "I don't know this file type! Attempting to process BTN.
+..\n";
        prcss_btn($in_file);
    }
}
[download]

(* For example backticks, but as I wrote about at length here, I wouldn't recommend it for this task. If you really wanted to capture the output, at least use capturex from IPC::System::Simple.)

Update: Note that I am just guessing as to the purpose of the $encoding variable in the code you showed.