It isn't fully clear what you'd like to do from your question. Presuming you have a good reason for wanting to use Term::Readline (which you don't need to process command line variables) then something like this might be what you're looking for:

#!/usr/bin/perl
use strict;
use warnings;
use Term::ReadLine;

# Initialize the term variable for the keyborad input
my $commandLineMessage = $ARGV[0];
my $TERM = new Term::ReadLine 'Installer';
my $OUT = $TERM->OUT || \*STDOUT; 
my $prompt  = "Would you like to continue?\n";

my $ans = 'yes' if $commandLineMessage eq 'yes';
$ans = $TERM->readline($prompt) if $commandLineMessage ne 'yes';

chomp($ans);
print "ans is $ans \n";

if($ans =~ /yes/)
    {
       print "yes and continue\n";
    }
else 
    {
       print "no and take action\n";
    }
[download]

If you had used strict and warnings, you would have found the main syntactical error in this line:

$ans = ($TERM->readline($prompt)) ? : 'yes' :<STDIN> ;
[download]

which means "if a response is given, then $ans is yes, if no response is given, get a line from standard input". Which in turn means that $ans will always be set to yes if the user gives any input at all.

I hope that helps