UPDATE: First I added /www/application/localperl/bin to my $PATH:
I added it to ~/.bashrc file by adding following line:
export PATH=$PATH:/www/application/localperl/bin
Then ran:
source ~/.bashrc

Then I followed huck's advise to get devel package using sudo yum -y install make gcc mysql-devel mysql-libs mysql-server I tried to install DBD::mysql from CPAN shell (/www/application/localperl/bin/cpan).

It was failing during make test and the log entries just before failure are:

Running make test
Running Mkbootstrap for DBD::mysql ()
chmod 644 "mysql.bs"
PERL_DL_NONLAZY=1 "/www/application/localperl/bin/perl" "-MExtUtils::C
+ommand::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; t
+est_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00base.t .............................. 1/6 # Driver version is 4.04
+1
t/00base.t .............................. ok
t/05dbcreate.t .......................... 1/2 # Database 'test' access
+ible
t/05dbcreate.t .......................... ok
t/10connect.t ........................... 1/? # mysql_clientinfo is: 5
+.1.73
# mysql_clientversion is: 50173
# mysql_serverversion is: 50173
# mysql_hostinfo is: Localhost via UNIX socket
# mysql_serverinfo is: 5.1.73
# mysql_stat is: Uptime: 4761821  Threads: 1  Questions: 4535919  Slow
+ queries: 2  Opens: 357  Flush tables: 1  Open tables: 41  Queries pe
+r second avg: 0.952
# mysql_protoinfo is: 10
# SQL_DBMS_VER is 5.1.73
# Default storage engine is: MyISAM
t/10connect.t ........................... ok
t/15reconnect.t ......................... ok
t/16dbi-get_info.t ...................... ok
t/20createdrop.t ........................ ok
t/25lockunlock.t ........................ ok
t/29warnings.t .......................... ok
t/30insertfetch.t ....................... ok
t/31insertid.t .......................... ok
t/32insert_error.t ...................... ok
t/35limit.t ............................. ok
t/35prepare.t ........................... ok
t/40bindparam.t ......................... ok
t/40bindparam2.t ........................ ok
t/40bit.t ............................... ok
t/40blobs.t ............................. ok
t/40catalog.t ........................... ok
t/40keyinfo.t ........................... ok
t/40listfields.t ........................ ok
t/40nulls.t ............................. ok
t/40nulls_prepare.t ..................... ok
t/40numrows.t ........................... ok
t/40server_prepare.t .................... 1/31 DBD::mysql::st DESTROY 
+failed: MySQL server has gone away at t/40server_prepare.t line 98.
        (in cleanup) DBD::mysql::st DESTROY failed: MySQL server has g
+one away at t/40server_prepare.t line 98.
t/40server_prepare.t .................... ok
t/40server_prepare_crash.t .............. ok
[download]

So I sudo as root and tried to install DBD::mysql using CPAN shell (/www/application/localperl/bin/cpan) and the installation completed successfully. many tests from make test were skipped with message skipped: no database connection but I think that was because I didn't add db password by setting environment variable. I was not very comfortable doing it so I tried installation with setting environment variable and it still completed successfully.

Thank you again Ken and huck for your inputs. I am marking this as Solved.