Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^3: Best way to store/access large dataset?

by erix (Prior)
on Jun 22, 2018 at 19:53 UTC ( [id://1217254]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Best way to store/access large dataset?
in thread Best way to store/access large dataset?

The ultimate goal of this will to be pulling data from a database and creating the binaries on the fly through a series of calculations, and then using this script to determine the next series of data points to pull from the database. (This serves as a filter.)

I have to ask:

"ultimate goal is pulling data from a database"? Then why were you talking about these .txt files in the OP?

"creating the binaries"? What are "binaries"?

Why pull data from a database to do "calculations" (apparently external from the db) when a database can do efficient calculations for you?

  • Comment on Re^3: Best way to store/access large dataset?

Replies are listed 'Best First'.
Re^4: Best way to store/access large dataset?
by Speed_Freak (Sexton) on Jun 28, 2018 at 19:09 UTC

    I missed this response, but I think I've answered the questions throughout the post. But if not I'll give it a shot now.

    The database doesn't exist yet, and I need to do the work as a proof of concept. So once the database exists, the script will be changed to point there instead of the files.
    Binaries are just a presence/absence representation of an attribute. They are calculated from a series of raw values by evaluating the relationships of those values a few different ways.
    I'm all for the database doing the calculations if it can. I'm in completely unfamiliar territory here, so recommendations are appreciated.

[reply]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://1217254]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others chilling in the Monastery: (2)
As of 2024-04-19 21:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found