in reply to Re: using a stochastic matrix to simulate weather conditions
in thread using a stochastic matrix to simulate weather conditions
thx, bliako, for this very useful script. I amaze at how quickly you can throw it together. You have just enough comments to teach me how this works as I work it. From the original, I had to dink around with the gridpoints by looking at the json from these urls in this form:
# https://api.weather.gov/points/{latitude},{longitude} # https://api.weather.gov/points/45.4836356,-122.4170501
, from which we glean the RHS of the url to be fetched:
my $query = "gridpoints/PQR/120,99/forecast";I customized the program to output orderly to a log as well, in time sequence and added times in Pacific Time.
#!/usr/bin/env perl use strict; use warnings; use DateTime::Format::Strptime; use REST::Client; use Data::Roundtrip qw/:all/; use 5.016; use Log::Log4perl; my $log_conf3 = "/home/hogan/Documents/hogan/logs/conf_files/3.conf"; my $log_conf4 = "/home/hogan/Documents/hogan/logs/conf_files/4.conf"; #Log::Log4perl::init($log_conf3); #debug Log::Log4perl::init($log_conf4); #info my $logger = Log::Log4perl->get_logger(); $logger->info($0); # this is our fetcher, similar to LWP::UserAgent # but better suited for this kind of web service: REST my $rest = REST::Client->new() or die "failed to construct client"; # see examples in # https://www.weather.gov/documentation/services-web-api # set the host $rest->setHost('https://api.weather.gov'); # https://api.weather.gov/points/{latitude},{longitude} # https://api.weather.gov/points/45.4836356,-122.4170501 # and this is our query with lat,long specified above #my $query = "gridpoints/TOP/$lat,$long/forecast"; my $query = "gridpoints/PQR/120,99/forecast"; # make the request and check the response code, 200 is good my $response = $rest->GET($query) or die "failed to GET($query)"; if ( $rest->responseCode() != 200 ) { die "failed to GET(" . $rest->getHost() . "/$query) with " . $rest->responseCode(); } # we get back JSON my $jsonstr = $response->responseContent(); # convert JSON string to a perl variable my $pv = json2perl($jsonstr); if ( !defined $pv ) { die "something wrong with this alleged json : '$jsonstr'"; } # go to the interesting part my $forecasts = $pv->{'properties'}->{'periods'}; # or print it all and examine it # print perl2dump($pv); # we have some dates in the data in ISO8601 format # this is a parser to convert that date to a DateTime # object which we can query about things (like seconds-unix-epoch) my $dateparser = DateTime::Format::Strptime->new( # parses 2020-04-26T06:00:00-05:00, # %F then literal T then %T then timezone offset pattern => '%FT%T%z' ) or die "failed to DateTime::Format::Strptime->new()"; # we store each prediction in this hash, keyed on start-end dates # see below $key ### hash no longer at this scope ### make an array container my @ordered; my $refordered = \@ordered; my $index = 0; for my $aforecast (@$forecasts) { print "go: " . $aforecast->{'name'} . "\n"; # extract various things from 1 prediction record # examine the record like : print perl2dump($aforecast); # like "This Afternoon" #print perl2dump($aforecast); my $period_str = $aforecast->{'name'}; # start and end times of the prediction, convert them to objects my $start_time_str = $aforecast->{'startTime'}; say "start is $start_time_str"; my $start_time_dateobj = $dateparser->parse_datetime($start_time_str +) or die "parsing date '$start_time_str'."; $start_time_dateobj->set_time_zone('America/Los_Angeles'); my $st = $start_time_dateobj->strftime('%Y-%m-%d-%H:%M +:%S %Z'); my $end_time_str = $aforecast->{'endTime'}; my $end_time_dateobj = $dateparser->parse_datetime($end_time_str) or die "parsing date '$start_time_str'."; $end_time_dateobj->set_time_zone('America/Los_Angeles'); my $et = $end_time_dateobj->strftime('%Y-%m-%d-%H:%M:%S %Z'); # sunny? my $forecast_str = $aforecast->{'shortForecast'}; # temperature as a number, see later for the units my $temp = $aforecast->{'temperature'}; # new scope for hash my %parsed; # store this record/prediction in our %parsed hash # keyed on this: my $key = $start_time_str . " to " . $end_time_str; $parsed{$key} = { 'date-human-str' => $period_str, # edit: added this 'date-from' => $start_time_str, 'date-from-local' => $st, 'date-to' => $end_time_str, 'date-to-local' => $et, 'date-span-hours' => ( $end_time_dateobj->epoch() - $start_time_dateobj->epoch() ) / +3600, 'date-from-epoch' => $start_time_dateobj->epoch(), 'date-to-epoch' => $end_time_dateobj->epoch(), 'forecast-string' => $forecast_str, # we append temp unit to the key, e.g. 'F' 'forecast-temp-' . $aforecast->{'temperatureUnit'} => $temp, }; my $refparsed = \%parsed; $ordered[$index] = $refparsed; $logger->info( perl2dump $refparsed); ++$index; } __END__
Log excerpt:
You still need to figure out how to convert the received data to machine-readable format, re: "sunny", and, most importantly, figure out how to store received data for easy access over time (SQLite database?, csv files?, json files? perl variable files which you can later eval? - in order of my preference) - what search keys, what identifies a unique record? etc.2020/05/01 08:59:21 INFO ./1.6.weather.pl 2020/05/01 08:59:21 INFO $VAR1 = { '2020-05-01T08:00:00-07:00 to 2020-05-01T18:00:00-07:00' => { 'date-from-epoch' => 1588345200, 'date-span-hours' => '10', 'date-to-local' => '2020-05-01-18:00:00 PDT', 'date-to-epoch' => 1588381200, 'date-to' => '2020-05-01T18:00:00-07:00', 'date-from-local' => '2020-05-01-08:00:00 PDT', 'date-from' => '2020-05-01T08:00:00-07:00', 'forecast-string' => 'Chance Rain Showers', 'date-human-str' => 'Today', 'forecast-temp-F' => 65 } }; 2020/05/01 08:59:21 INFO $VAR1 = { '2020-05-01T18:00:00-07:00 to 2020-05-02T06:00:00-07:00' => { 'date-from' => '2020-05-01T18:00:00-07:00', 'date-from-local' => '2020-05-01-18:00:00 PDT', 'date-to' => '2020-05-02T06:00:00-07:00', 'date-to-epoch' => 1588424400, 'date-to-local' => '2020-05-02-06:00:00 PDT', 'date-span-hours' => '12', 'date-from-epoch' => 1588381200, 'forecast-temp-F' => 50, 'date-human-str' => 'Tonight', 'forecast-string' => 'Rain Showers' } }; 2020/05/01 08:59:21 INFO $VAR1 = { '2020-05-02T06:00:00-07:00 to 2020-05-02T18:00:00-07:00' => { 'date-from-local' => '2020-05-02-06:00:00 PDT', 'date-to' => '2020-05-02T18:00:00-07:00', 'date-from' => '2020-05-02T06:00:00-07:00', 'date-span-hours' => '12', 'date-to-local' => '2020-05-02-18:00:00 PDT', 'date-from-epoch' => 1588424400, 'date-to-epoch' => 1588467600, 'forecast-temp-F' => 60, 'forecast-string' => 'Rain Showers', 'date-human-str' => 'Saturday' } };
When I get to the part where I'm keeping count for real, I think a SQLite database is the right tool.
Then you need to figure out how to create the sequence of "sunny"->"cloudy"->... given the time periods of the forecast etc.You know, it takes me all week to figure out a script that is written at a level that stretches my vistas, and then figure how I'm going to "make it my own," maybe extend it, and how I'm going to write it up. During this week and change, it has happened several times that I've been in situations where it was sunny and raining. I think we have to start somewhere other than where I suggested in the original post.
But it looks ;ole I missed something because you should be training your model with actual data and not forecasts. But that's trivial to correct hopefully, just change your urls.That's just it. I've been spending as much time and effort as I can to find actual data, but the most I see is GUI representations as opposed to JSON. I'm also realizing that conditions at Troutdale Airport might be completely different than here on Hogan Butte. Troutdale is right in the mouth of the Columbia Gorge, which is a very interesting formation for the way it channels wind, for example.
Q2 Can anybody see a way to get current conditions in JSON form?
I found more reading re sunlight v cvirus here: a bbc article.
Thanks all for comments, but especially:
for (@responses){ [bliako]++; }
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^3: using a stochastic matrix to simulate weather conditions
by hippo (Archbishop) on May 02, 2020 at 08:11 UTC |