Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^7: Perl Contempt in My Workplace

by hippo (Bishop)
on May 28, 2021 at 09:43 UTC ( [id://11133208]=note: print w/replies, xml ) Need Help??


in reply to Re^6: Perl Contempt in My Workplace
in thread Perl Contempt in My Workplace

JSONs are not possible for 10_000_000 records

JSON is a data format. It has no limitations on the size of the data.


🦛

Replies are listed 'Best First'.
Re^8: Perl Contempt in My Workplace
by cavac (Parson) on May 28, 2021 at 10:34 UTC

    Technically, yes. But nearly all JSON parsers i've seen are designed to slurp in everything all at once and turn it into a in-memory data structure. So for very large files, you might (or might not) have to cobble together a custom parser that can do a stream-as-you-go approach.

    Of course, that's where Perl comes into its own. Munching insanely huge text files is what it was designed for in the first place ;-)

    perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'

      You're an oracle man, right? Oracle has JSON_TABLE to alleviate the JSON weirdness, I think.

      Postgres has the same functionality, but not yet those JSON_TABLE API/functions (that the SQL Standard prescribes). That JSON_TABLE work (for postgres) is largely done, although not yet committed. Somewhat understandably, there seems to be a lack of interest: most DBAs look down a bit upon the strangeness of JSON data type, and prefer tables of more conventional data types. I take it that in the oracle-world there is the same reluctance towards this encroachment of NoSQL-y types. I feel the same reluctance myself.

        I used to use Oracle back in the day. Nowadays, it's PostgreSQL all the way.

        There are a few cases where i use JSON datatypes and similar, but they are very limited.

        perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'
    A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11133208]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (9)
As of 2024-04-18 11:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found