With serial drivers one always worries about overruns and framing errors. These are usually the reason for lost data. Probably also the reason sysread is choking. I would expect to see some error handling in your read loop but I don't see any. Something like:

 if ($LatchErrorFlags & CE_FRAME) { warn "Framing Error"; }

Does your device implement rts/cts flow control? Does the serial cable contain the rts/cts lines. Do any adapters omit those lines?