Hello Monks, I've spent several hours over the last couple days trying to get this to work. My script connects to a website that streams JSON continuously, what I need to do is decode the JSON, filter it then insert the filtered results into a database. The script below currently connects but JSON::SL doesn't seem to be identifing the JSON objects and decoding them the script just sits there waiting to decode.
#!/usr/bin/perl use strict; use IO::Socket::SSL; #qw(debug3); use JSON::SL; print "JSON Stream parser test\r\n"; my $p = JSON::SL->new; #Look for everything past the first JSON pointer $p->set_jsonpointer(["/^"]); # simple client my $cl = IO::Socket::SSL->new( PeerAddr => 'example.com', PeerService => 'https', PeerPort => 443, Proto => 'tcp', Reuse => 1 ); die "no connection" unless $cl -> connected(); print $cl "GET /stream/report?token=IAMAFAKETOKEN HTTP/1.0\r\n\r\n +"; print <$cl>; local $/ = \5; #read only 5 bytes at a time while (my $line = <$cl>) { $p->feed($line); #parse what you can fetch anything that comple +ted the parse and matches the JSON Pointer while (my $obj = $p->fetch) { print "$obj->{Value}{sequenceNumber}: $obj->{Value}{sender +Callsign}\n"; #Inserted for testing #do filtering and database stuff here } } close $cl;
Using this code to test strictly the decode it works.
#!/usr/bin/perl use strict; use warnings; use JSON::SL; my $p = JSON::SL->new; #look for everthing past the first level (i.e. everything in the array +) $p->set_jsonpointer(["/^"]); local $/ = \5; #read only 5 bytes at a time while (my $buf = <DATA>) { $p->feed($buf); #parse what you can #fetch anything that completed the parse and matches the JSON Poin +ter while (my $obj = $p->fetch) { print "$obj->{Value}{sequenceNumber}: $obj->{Value}{senderCall +sign}\n"; } } __DATA__ [ {"sequenceNumber":1401615035,"frequency":7076402,"mode":"JT65","sNR +":-1,"flowStartSeconds":1492192912,"senderCallsign":"DJ0FX","senderLo +cator":"JN67KT","receiverCallsign":"OZ7IO","receiverLocator":"JO65gq" +,"receiverDecoderSoftware":"WSJT-X v1.7.0 r7405"}, {"sequenceNumber":1401615039,"frequency":7077903,"mode":"JT65","sN +R":-11,"flowStartSeconds":1492192973,"senderCallsign":"R7NO","senderL +ocator":"KN98","receiverCallsign":"OZ7IO","receiverLocator":"JO65gq", +"receiverDecoderSoftware":"WSJT-X v1.7.0 r7405"}, {"sequenceNumber":1401615040,"frequency":7076811,"mode":"JT65","sN +R":-11,"flowStartSeconds":1492193032,"senderCallsign":"F6AVP","sender +Locator":"JN28UX","receiverCallsign":"OZ7IO","receiverLocator":"JO65g +q","receiverDecoderSoftware":"WSJT-X v1.7.0 r7405"} ]
Here's a sample of the data as received, in the working decode script I had to add a comma at the end of each line.
{"sequenceNumber":1401615035,"frequency":7076402,"mode":"JT65","sNR":- +1,"flowStartSeconds":1492192912,"senderCallsign":"DJ0FX","senderLocat +or":"JN67KT","receiverCallsign":"OZ7IO","receiverLocator":"JO65gq","r +eceiverDecoderSoftware":"WSJT-X v1.7.0 r7405"} {"sequenceNumber":1401615036,"frequency":7076996,"mode":"JT65","sNR":- +6,"flowStartSeconds":1492192912,"senderCallsign":"RU6MO","senderLocat +or":"KN97KF","receiverCallsign":"OZ7IO","receiverLocator":"JO65gq","r +eceiverDecoderSoftware":"WSJT-X v1.7.0 r7405"} {"sequenceNumber":1401615037,"frequency":7077360,"mode":"JT65","sNR":- +5,"flowStartSeconds":1492192913,"senderCallsign":"RV3QD","senderLocat +or":"KO91NQ","receiverCallsign":"OZ7IO","receiverLocator":"JO65gq","r +eceiverDecoderSoftware":"WSJT-X v1.7.0 r7405"}

In reply to Trying to decode JSON streaming from a website by ve6sar

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.