Crude and slow, but it doesn't require anything but what you already have (strftime), and it should work (I haven't tried it)...

my $date_in_past = '20070101'; # YYYYMMDD
my $ONE_DAY = 24 * 60 * 60;
my $days_ago = 0;
my $now = time();

while (   $date_in_past
        < strftime('%Y%m%d',
                   localtime( $now - $days_ago*$ONE_DAY ) ) )
{
    $days_ago++;
}
[download]

I'm not proud. I guess without Date::Calc, I really am lost.