Hi, I'm new to Perl and I'm trying extract weather data from various sources and output this data into a standard format for import into a database.

One of the data sources is a CSV file which date and time stamps it's records using the local time on the server at the time it's written to the file. I'm looking to standardise the time and need to convert this time from the local time to UTC. I'm in the UK so this changes by 1 hour going forward on the last Sunday in March and back on the last Sunday in October at 2am.

The following code is what I have so far: 
#! c:\xampp\perl\bin\perl

use strict;
use warnings;

use LWP::Simple;
use Time::Local;

my $standard_filename = $ARGV[0];
my $extended_filename = $ARGV[1];

# Get yesterdays date time

my $epoc = time();
$epoc = $epoc - 86400;
my $yesterday = localtime($epoc);

my ($sec, $min, $hour, $day, $month, $year) = (localtime($epoc))[0,1,2
+,3,4,5];
$month++;
$year += 1900;

my $mm = $month;

if ($month < 10) {
    $month = "0" . $month;
}

if ($day < 10) {
    $day = "0". $day;
}

# Build the Standard and Extended filenames if not passed as parameter
+s

if (not defined $standard_filename) {
    $standard_filename = "C:\\wdisplay\\logfiles\\" . $mm . $year . "l
+gcsv.csv";
}

if (not defined $extended_filename) {
    $extended_filename = "C:\\WebHost\\customtextout.txt";
}

# Load Standard File in to an Array

my @standard_records;
open (standard_file, "<", $standard_filename)
    or die "Failed to open file: $!\n";

while (<standard_file>) {
    chomp;
    push @standard_records, $_;
}

close standard_file;

# Load Extended File into an Array

my @extended_records;
open (extended_file, "<", $extended_filename)
    or die "Failed to open file: $!\n";

while (<extended_file>) {
    chomp;
    push @extended_records, $_;
}

close extended_file;

# Process each Standard Record

foreach (@standard_records) {

    my @std_lines = split /\n/, $_;
    foreach my $line_std (@std_lines) {

        my @std_fields = split ",", $line_std;

        my $obs_day = $std_fields[0];
        my $obs_month = $std_fields[1];
        my $obs_year = $std_fields[2];
        my $obs_hour = $std_fields[3];
        my $obs_minute = $std_fields[4];
        my $obs_seconds = 0;

        if ($obs_day eq "day") {
            next;
        }

        my $utc_time = timelocal($obs_seconds, $obs_minute, $obs_hour,
+ $obs_day, $obs_month - 1, $obs_year - 1900);

        my ($utc_sec, $utc_min, $utc_hour, $utc_day, $utc_month, $utc_
+year) = (gmtime($utc_time))[0,1,2,3,4,5];
        $utc_month++;
        $utc_year += 1900;

        if ($obs_day < 10) {
            $obs_day = "0" . $obs_day;
        }

        if ($obs_month < 10) {
            $obs_month = "0" . $obs_month;
        }

        if ($obs_hour < 10) {
            $obs_hour = "0" . $obs_hour;
        }

        if ($obs_minute < 10) {
            $obs_minute = "0" . $obs_minute;
        }

        my $record_line = $obs_year . "," . $obs_month . "," . $obs_da
+y . "," . $obs_hour . "," . $obs_minute . "," . "00" . "," . "," . $u
+tc_year . "," . $utc_month . "," . $utc_day . "," . $utc_hour . "," .
+ $utc_min . "," . $utc_sec . "\n";

        print $record_line;

        my $file = "C:\\WEATHER_DATA_COLLECTOR\\temp.csv";

        open (TXT, ">>", $file);
        print TXT $record_line;
        close (TXT);

    }    

}

[download]
There is a bit of redundant code at the moment but the area of interest is within the foreach (@standard_records) loop and the date conversions within this section of the code. Initially I thought this was working but it does not handle the time when the clocks change correctly.

The following is a sample of the output as the clock goes back at the end of October 2014:

Local / UTC
25th 23:00 | 25th 22:00
26th 00:00 | 25th 23:00
26th 01:00 | 26th 01:00 (this should have been 00:00)
26th 01:00 | 26th 01:00 (hour should not have been repeated)
26th 02:00 | 26th 02:00
26th 03:00 | 26th 03:00

The local time goes back at 2am so 01:00 - 01:59 is repeated. UTC should be continuous, effectively 'catching-up' with the local time as it goes back. But, it jumps forward at 23:59 (missing out 00:00 - 00:59) and then going back at 2am like local time.

So, either side of the time change it looks correct it's just the transition it's not handling correctly.

I'd be very grateful for any help on what I'm doing wrong - Thanks Colin.

In reply to Converting local time to utc time by ureco

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.