Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Compiling and uploading a crontab to my Radioduino

by cavac (Parson)
on May 28, 2021 at 08:15 UTC ( [id://11133206]=CUFP: print w/replies, xml ) Need Help??

For background information: I'm working on my Radioduino, a souped up version of an Arduino Uno with lots of inbuild features like a real-time clock and external memory etc. It communicates via nRF24 radio with my nRF24 "modem", which in turn is accessible via Net::Clacks in my local network.

One of the features is a scheduler, similar to a Linux/Unix crontab. This is stored in FRAM in binary form and basically injects uplink radio commands at specified times into the command sequencer.

The structure of the crontab in the Radioduino is this:

typedef struct { uint8_t mode; uint8_t hour; uint8_t minute; uint8_t second; uint16_t offset; uint8_t command; uint8_t datalength; uint8_t data[16]; } SCHEDULEENTRY;

Of course, this is binary and a pain in the seating arrangement to edit by hand. So i made some perl tools to compile plain ASCII text files and upload them.

First, let's look at an example file of the plaintext crontab file:

#runonce hour minute second command offset values 0 0 0 0 DEEPSLEEP 0 0 19 +20 0 0 20 0 STREAM_TELEMETRY 0 0 0 0 20 30 SWITCHED_POWER 0 1 0 0 21 0 STREAM_TELEMETRY 0 0 0 0 22 0 WRITECOIL 0 2 4 1 0 0 23 0 STREAM_TELEMETRY 0 0 0 0 23 30 SWITCHED_POWER 0 0 0 0 24 0 DEEPSLEEP 0 0 59 0

Next up, we need to turn this thing into a binary file, compatible with the Radioduino.

#!/usr/bin/env perl use strict; use warnings; use Carp; use Data::Dumper; # Command map (text to command number). This should be parsed from rad +ioduino/globals.h, # but for now it's just hardcoded my %commandmap = ( WRITECONFIG => 2, READCONFIG => 3, RECONFIGURE_RADIO => 5, WRITEFRAM => 6, READFRAM => 7, PING => 9, WRITE_RTC => 16, READ_RTC => 17, STREAM_TELEMETRY => 19, READINPUTREGISTERS => 20, READHOLDINGREGISTERS => 22, WRITEHOLDINGREGISTERS => 24, READCOIL => 26, WRITECOIL => 28, READDISCRETE => 30, '32KHZ' => 50, SWITCHED_POWER => 52, DEEPSLEEP => 54, DEBUG => 60, ); # Open the file handles open(my $ifh, '<', 'crontab') or croak($!); open(my $ofh, '>', 'crontab.bin') or croak($!); binmode $ofh; # Format of the plaintext crontab #runonce hour minute second command offset values my $cnt = 0; while((my $line = <$ifh>)) { $cnt++; chomp $line; # Ignore comments next if($line =~ /^\#/); # Make sure there is only one space between columns $line =~ s/\ +/ /g; # Split elements and do some very basic validation my ($runonce, $hour, $minute, $second, $command, $offset, @values) + = split/\ /, $line; if(!scalar @values) { croak("Line $cnt: has not enough values: $line\n"); } # Turn the given command NAME into the correct command NUMBER if(!defined($commandmap{$command})) { croak("Line $cnt: Unknown command $command\n"); } my $numcommand = $commandmap{$command}; # Internally in the Radioduino, the "runonce" flag also serves as +"invalid record" flag, so turn the plaintext 0/1 boolean into the cor +rect number my $mode = 0; if($runonce == 1) { $mode = 2; } else { $mode = 1; } # Bulk up @values to the correct length of 16 bytes while((scalar @values) < 16) { push @values, 0; } # Turn everything into binary my $event = ''; $event .= chr($mode); $event .= chr($hour); $event .= chr($minute); $event .= chr($second); $event .= chr(($offset >> 8) & 0xff); $event .= chr($offset & 0xff); $event .= chr($numcommand); $event .= chr(scalar @values); foreach my $val (@values) { $event .= chr($val); } # Write binary entry to file print $ofh $event; print $line, "\n"; } # Fill in empty records, so we overwrite any old entries with empty/in +valid ones while($cnt < 86) { my $event = chr(0) x 24; print $ofh $event; $cnt++; } # Fill the remaining 8 bytes with zeroes as well for(1..8) { print $ofh chr(0); } # close filehandles close $ifh; close $ofh

The last step is to upload the file to Radioduino via the nRF24 link. For this, we send/receive via Net::Clacks through our nRF24 "modem" running on a different computer. The GSPSupport.pm implements some of the specifics of the radio protocol, which are currently under redesign. So it's probably moot to post that here and get directly to the actual uploader script. Edit: Posted the GSPSupport.pm at the end of the post, anyway.

#!/usr/bin/env perl use strict; use warnings; use Time::HiRes qw(sleep); use Data::Dumper; use Net::Clacks::Client; use XML::Simple; use Carp; # Load the temporary GSPSupport module (currently under redesign due t +o protocol change) # FIXME: This script uses all kinds of hardcoded values. This will nee +d to change after the protocol upgrade BEGIN { unshift @INC, './'; }; use GSPSupport; my $project = GSPSupport->parse(); # Load local config (e.g. with radio device we are talking to, relay r +outing etc) my $config = GSPSupport::loadClacksConfig(); # Slurb the binary crontab file my $tmptab = GSPSupport::slurpBinFile('crontab.bin'); my @crontab = split//, $tmptab; my $offset = 0; my $lastpacket; my $resendtime = 0; # Connect to Net::Clacks and listen to packets from our target device my $clacks = Net::Clacks::Client->new($config->{host}, $config->{port}, $config->{user}, $config->{password}, 'GSPDecoder'); $clacks->listen('GSP::RECIEVE::' . $project->{textid}); my $nextping = 0; my $done = 0; # Initiate stream by sending the first chunk sendNextChunk(); while(!$done) { my $now = time; if($nextping < $now) { $clacks->ping(); $nextping = $now + 30; } # Check for timeout and resend packet if required if($resendtime > 0 && $resendtime < time) { $clacks->set('GSP::SEND', $lastpacket); $clacks->doNetwork(); $resendtime = time + 5; } $clacks->doNetwork(); while((my $message = $clacks->getNext())) { if($message->{type} eq 'disconnect') { $clacks->listen('GSP::RECIEVE::' . $project->{textid}); $clacks->ping(); $clacks->doNetwork(); $nextping = $now + 30; next; } next unless($message->{type} eq 'set'); next unless($message->{name} eq 'GSP::RECIEVE::' . $project->{ +textid}); # We got some packet from our target Radioduino, check if it's + the right one (it may not be, it could be some other # telemetry stuff) decodeFrame($message->{data}); } sleep(0.01); } exit(0); my @frame; sub decodeFrame { my $line = shift; @frame = GSPSupport::packet2frame($line); if($frame[6] != $project->{id}) { # Not from this project return; } if($frame[8] == 255) { # ERROR FRAME. This happens if we did not disable the schedule +r manually before starting to upload # a new crontab GSPSupport::decodeErrorFrame(@frame); $done = 1; return; } if($frame[8] != 8) { # Not a COMMAND_DOWN_READFRAM, probably some other automatic t +elemetry stream return; } if($offset == 2048) { # Seems we have send all data, so we are done print "All sent!\n"; $done = 1; return; } # Ok, send the next chunk of data sendNextChunk(); } sub sendNextChunk { print "Sending $offset...\n"; # Get an empty frame with default values like routing already set my @outframe = GSPSupport::emptyframe($project); # Set command number $outframe[8] = 6; # COMMAND_UP_WRITEFRAM # Set FRAM address and data length $outframe[9] = ($offset >> 8) & 0xff; $outframe[10] = $offset & 0xff; $outframe[11] = 16; # Copy the data chunk for(my $i = 0; $i < 16; $i++) { my $char = shift @crontab; $outframe[12 + $i] = ord($char); } # Turn this array into a proper nRF24-over-Clacks frame packet my $packet = GSPSupport::frame2packet(@outframe); # Remember the packet in case we need to resend it $lastpacket = $packet; # Send it via clacks and remember the new timeout time $clacks->set('GSP::SEND', $packet); $clacks->doNetwork(); $resendtime = time + 5; # Move offset $offset += 16; return; }

Here is the GSPSupport.pm code:

package GSPSupport; use strict; use warnings; use XML::Simple; use Carp; use English; sub parse { open(my $ifh, '<', 'projectname.txt') or croak($!); my $projectname = <$ifh>; close $ifh; chomp $projectname; if(!length($projectname)) { croak("No projectname given!"); } my $devicesfile = '/home/cavac/src/gardenspaceagency/missioncontro +l/devices.xml'; if(!-f $devicesfile) { $devicesfile = '../devices.xml'; } my $deviceconfig = XMLin($devicesfile); my $devices = $deviceconfig->{device}; if(!defined($devices->{$projectname})) { croak("Unknown project $projectname"); } my $config = $devices->{$projectname}; my @rf24routing; push @rf24routing, $config->{id}; while((scalar @rf24routing < 5)) { push @rf24routing, 0; } $config->{rf24routing} = \@rf24routing; return $config; } sub loadClacksConfig { my $configfile = '/home/cavac/src/clacksconfig.xml'; if(!-f $configfile) { $configfile = '../clacksconfig.xml'; } my $config = XMLin($configfile); return $config; } sub emptyframe { my $config = shift; my @outframe = (0) x 30; # Sender, reciever $outframe[0] = 0x01; # Linksender MODEM for(my $i = 0; $i < 5; $i++) { $outframe[$i + 1] = $config->{rf24routing}->[$i]; } $outframe[6] = 0x01; # Real sender MODEM $outframe[7] = $config->{id}; # Real reciever return @outframe; } sub frame2packet { my @frame = @_; my $packet = ''; foreach my $byte (@frame) { my $lowbyte = ($byte & 0x0f) + 65; my $highbyte = ($byte >> 4) + 65; $packet .= chr($highbyte); $packet .= chr($lowbyte); } return $packet; } sub packet2frame { my $packet = shift; my @chars = split//, $packet; my @frame = (); # Decode to bytes while(@chars) { my $high = shift @chars; my $low = shift @chars; my $val = ((ord($high) - 65) << 4) + (ord($low) - 65); push @frame, $val; } return @frame; } sub decodeErrorFrame { my @frame = @_; if($frame[12] == 1) { print "MODBUS ERROR\n"; } elsif($frame[12] == 2) { print "INVALID PAYLOAD_LENGTH ERROR\n"; } elsif($frame[12] == 3) { print "MODBUS_SLAVE_NOT_SET ERROR\n"; } elsif($frame[12] == 4) { print "REBOOT_DETECTED ERROR\n"; } elsif($frame[12] == 5) { print "REQUEST_REJECTED ERROR\n"; } elsif($frame[12] == 6) { print "MOSFET_TEMPERATURE ERROR\n"; } return; } sub slurpBinFile { my $fname = shift; # Read in file in binary mode, slurping it into a single scalar. # We have to make sure we use binmode *and* turn on the line termi +nation variable completly # to work around the multiple idiosynchrasies of Perl on Windows open(my $fh, "<", $fname) or croak($ERRNO); local $INPUT_RECORD_SEPARATOR = undef; binmode($fh); my $data = <$fh>; close($fh); return $data; } 1;

I hope this post isn't too boring and helps inspire some ideas for your own projects.

perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'

Replies are listed 'Best First'.
Re: Compiling and uploading a crontab to my Radioduino
by jwkrahn (Abbot) on May 30, 2021 at 01:27 UTC

    From GSPSupport.pm (frame2packet):

    for my $byte ( @frame ) { my $lowbyte = ( $byte & 0x0f ) + 65; my $highbyte = ( $byte >> 4 ) + 65;

    I tested this subroutine with numbers ranging from 0 to 9,999,999,999 and I can tell you two things.

    1) $lowbyte is not a byte, it only has four bits. It is a nibble.

    2) $highbyte is not a byte as it can contain any value above 255 (0xFF).

      Oh, yes. It's badly named. This function actually turns the bytes (which ARE bytes) from the binary packet frame into "cavac-encoded" half-bytes for serial transmission.

      So, $highbyte (which should actually be $highnibble or something) can't go over 255).

      The reason i'm using this encoding is because it's dead simple to turn it back into real bytes, while allowing proper framing for transmitting over serial.

      perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
Re: Compiling and uploading a crontab to my Radioduino
by jwkrahn (Abbot) on May 29, 2021 at 03:25 UTC

    Hi, I'm not clear about one thing.

    typedef struct { uint8_t mode; uint8_t hour; uint8_t minute; uint8_t second; uint16_t offset; uint8_t command; uint8_t datalength; uint8_t data[16]; } SCHEDULEENTRY;

    According to the data structure, "datalength" is the length of the data in "data".

    According to your code, the "data" field is padded with zeros and then the length is taken so the "datalength" field is always 16.

    Is this correct, or should "datalength" reflect the actual bytes of data in "data"?

      Yes, the in-memory representation is fixed length (as it the radio packet length). But depending on the uplink command, it might use fewer of those bytes.

      An example would be changing some configuration bytes in the Radioduino EEPROM. You can send up to 16 bytes in one packet, but you might only want to change three bytes. The datalength says how many bytes to change, the offset is the address in the EEPROM.

      As i said, the packet format (and therefore also the scheduler data structure) is going to change in the future to make it more flexible. But i originally designed this a few years back as my first real Arduino project and my first time using nRF24 radio links. And now half of my home automation is also based on this format, and i have to design and plan an incompatible upgrade to the whole shebang.

      perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
        You can send up to 16 bytes in one packet, but you might only want to change three bytes. The datalength says how many bytes to change,

        Yes, that is what I assumed, but your code always sets the datalength value to 16.

        You might try it like this:

        while ( my $line = <$ifh> ) { # Ignore comments next if $line =~ /^#/; # Split elements and do some very basic validation my ( $runonce, $hour, $minute, $second, $command, $offset, @values + ) = split ' ', $line; @values or croak( "Line $.: has not enough values: $line\n" ); # Turn the given command NAME into the correct command NUMBER defined $commandmap{ $command } or croak( "Line $.: Unknown comman +d $command\n" ); my $numcommand = $commandmap{ $command }; # Internally in the Radioduino, the "runonce" flag also serves # as "invalid record" flag, so turn the plaintext 0/1 boolean into + the correct number my $mode = $runonce + 1; # Turn everything into binary my $event = pack 'C C C C n C C C16', $mode, $hour, $minute, $second, $offset, $numcommand, scalar @values, @values, ( 0 ) x 16; # Write binary entry to file print $ofh $event; print $line, "\n"; }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: CUFP [id://11133206]
Approved by kcott
Front-paged by Discipulus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others studying the Monastery: (6)
As of 2024-03-28 15:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found