rustybar has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I'm working on a project which requires me to read data from a weather sensor board thorough serial port and log them continuously into a text file. However, I have encounter a weird problem and hope you can help me with this.

I use lookfor to poll for data and it works great. However, after like 1 day of testing, there seems no data coming from the serial port anymore and the program just hang like at the lookfor statement there. Upon restarting the program, everything works fine again. I was wondering why after 1 day of reading from the serial port, the lookfor statement seem not to work anymore. Please advise me on this issue. Thanks!

Below is my partial code:

# Set up the serial port # 9600, 81N on the USB ftdi driver $port = Device::SerialPort->new("/dev/ttyUSB0"); $port->databits(8); $port->baudrate(9600); $port->parity("none"); $port->stopbits(1); $port->read_const_time(100); # const time for read (milliseconds +) $port->read_char_time(5); # avg time between read char $port->lookclear; # empty buffers $count_interval = 0; while (1) { readXML; sleep for the time interval before polling for data $e = $data->{Weather_Data_Interval}; $interval = $e->{value}; # Poll to see if any data is coming in $values = $port->lookfor(); # If we get data, then print it if ($values) { $values =~ s/#//g; $values =~ s/\$//g; $values =~ s/\r//g; @vars = split(/,/,$values); # high frequency logging writeValueXML; $count_interval++; # for displaying in website in the requested time inte +rval if($count_interval == $interval){ appendLoggedFile; $count_interval = 0; } #print "$values\n"; } }

Replies are listed 'Best First'.
Re: Continusously reading from serial port
by roboticus (Chancellor) on Apr 17, 2009 at 13:58 UTC
    rustybar:

    Have you verified that the sensor is still transmitting? I've seen cases where program startup logic can reset a sensor and start it, so put a serial line monitor2 on it and verify that the sensor is still transmitting when your program is hung.

    For example, perhaps some code is making the loop slow (a cron job decides to thrash the heck out of RAM and I/O), so the serial buffer fills up. The serial driver might signal to the sensor that it should stop sending for a while1. But the sensor might not understand the signal convention used. It might interpret an XOFF as a power-down command. Or it might think that a command is coming in, and wait (forever) for the rest of the command before sending its next report, or something.

    I've also seen cases where the driver uses XOFF to stop and XON to continue, but the device uses XOFF to stop and another XOFF to continue, so the device waits (forever) for an XOFF that's never going to arrive.

    The sensor might have just gotten confused for some other reason and gave up the ghost.

    NOTES:

    1 There are too darned many different ways applications can tell each other to stop sending and then proceed. XOFF/XON is a commonly used character-based protocol, though there are others. But there are also hardware handshaking versions using the RTS/CTS and DTR/DSR lines on your serial port. The long and short of it is, be sure that both ends of the wire agree!

    2 There's an entire range of serial line monitors. Some fancy protocol analyzers, eavesdroppers, etc. Often you can get away with a simple mini-tester, available (nearly everywhere) in both 9 pin and 25 pin versions. You can just watch the LEDs flicker to see which end of the cable is sending, and the current state of the hardware handshaking lines...

    Finally, I don't know how much serial communications mojo you have, so please don't be offended if I'm telling you the obvious...

    UPDATE: If you want something fancier, it's easy enough to put together a more advanced serial line monitor if you have a spare computer with a pair of serial ports. You just need an eavesdropping cable (details below) and a little software to monitor both ports. Print all characters from one port in one color, and the characters from the other port in a different color.

    Cable Details: (Pin numbers rearranged to simplify drawing) To original To original Eavesdropper system cable end computer Port 1 Port 2 DB-9 DB-9 DB-9 DB-9 MALE FEMALE FEMALE FEMALE 9------------9 9 9 8------------8 8 8 7------------7 7 7 6------------6 6 6 5------------5------------5-----------5 4------------4 4 4 1------------1 1 1 2--------*---2 +----2 +----2 3-----*------3 | 3 | 3 | | | | | +-----------+ | +--------------------------+
    ...roboticus
Re: Continusously reading from serial port
by Bloodnok (Vicar) on Apr 17, 2009 at 10:26 UTC
    I agree with targetsmart ... up to a point:

    Assuming appendLoggedFile does what it says on the tin, I can't see that that call can be problemmatic since (I assume) it's constantly writing to a file on the file system - if it were problemmatic i.e. the file system is full, then it wouldn't be possible to get the thing working by restarting - unless, of course, your script resets the output file before going into the loop i.e. outside the scope of the snippet you provide.

    If the behaviour of both readXML and writeValueXML were to be disclosed, then that opens up the possibility of discounting either, or both, of them as being the source(s) of your troubles.

    A user level that continues to overstate my experience :-))

      Hi thanks for the reply, I thought of it so but then I view the codes and could not find any possibility that they could have cause the hang up. Below are the codes, would appreciate if you all help me take a look.

      sub readXML{ # create object $xml = new XML::Simple (KeyAttr=>[]); # read XML file $data = $xml->XMLin($config_path . "sensors.xml"); } sub writeValueXML{ while(! (-x $config_path . "sensors.xml")){} #wait until other + process finish writing the file system("sudo chmod 644 $config_path" . "sensors.xml"); #set per +mission for root to rw only (exclusive write) my $output = new IO::File(">" . $config_path . "sensors.xml"); my $writer = new XML::Writer(OUTPUT => $output); my $flag = 0;; $count = 0; $writer->startTag("Sensors"); while($count < $DATANUM){ #write xml values $e = $data->{$sensor_type[$count]}; $writer->startTag("$sensor_type[$count]"); $writer->startTag("value"); $writer->characters($vars[$count]); $writer->endTag("value"); $writer->startTag("threshold"); $writer->characters($e->{threshold}); $writer->endTag("threshold"); $writer->endTag("$sensor_type[$count]"); if(($vars[$count]*1.0) > ($e->{threshold}*1.0)){ #*1 to for +ce it to float type if (-e $config_path . "SensorAlert.txt" && !$flag){ while(! (-x $config_path . "SensorAlert.txt")){} # +wait until other process finish writing the file unlink($config_path . "SensorAlert.txt"); #set perm +ission for root to rw only (exclusive write) } open WRITEFILE, "+>>", $config_path . "SensorAlert.txt"; print WRITEFILE "$count:$vars[$count]\n"; close(WRITEFILE); $flag++; } $count++; } $e = $data->{Weather_Data_Interval}; $writer->startTag("Weather_Data_Interval"); $writer->startTag("value"); $writer->characters($e->{value}); $writer->endTag("value"); $writer->endTag("Weather_Data_Interval"); $writer->endTag("Sensors"); $writer->end(); $output->close(); if($flag){ read_config_file; if ($config_data[10] ne "Sensors:disabled\n"){ $config_data[10] = "Sensors:true\n"; write_config_file; } system("sudo chmod 777 $config_path" . "SensorAlert.txt"); } system("sudo chmod 777 $config_path" . "sensors.xml"); } sub appendLoggedFile{ my ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $da +yOfWeek, $dayOfYear, $daylightSavings) = localtime(); $year = 1900 + $yearOffset; my $theTime = "$hour:$minute:$second, $weekDays[$dayOfWeek] $month +s[$month] $dayOfMonth, $year"; open WRITEFILE, "+>>", $data_path . "$months[$month]_$dayOfMonth +" . "_SensorLog.txt" or die $!; print WRITEFILE $values . ",\t\t" . $theTime . "\n"; close(WRITEFILE); }

      and roboticus, when the program hang, I end the process and use minicom to read from the port again, and the data is still transmittin. I cannot use 2 programs to read the same port at the same time, as it will say the port is being used by other process right now.

      Thanks!

Re: Continusously reading from serial port
by targetsmart (Curate) on Apr 17, 2009 at 07:44 UTC
    my Guess
    problem may be in readXML; or writeValueXML; or appendLoggedFile;
    Are you using any global array to push the contents read from the devices?, so that the array grows huge
    If you are using *nix you can check with 'top' command to see the memory usage of the program, if it is huge then you might have made a mess with handling variables.
    With the data provided I can't come to any conclusion. .

    Vivek
    -- In accordance with the prarabdha of each, the One whose function it is to ordain makes each to act. What will not happen will never happen, whatever effort one may put forth. And what will happen will not fail to happen, however much one may seek to prevent it. This is certain. The part of wisdom therefore is to stay quiet.