in reply to Re: multiple hash compare, find, create
in thread multiple hash compare, find, create
Hello Eily. I should have provided a better explanation regarding the creation of the large hash tables. I'm using the Splunk API to pull large datasets overnight. However, it is impossible to pull all of the information we need from Splunk in one query. In addition, the Splunk administrators have incorporated limits on the amount of data that can be pulled and the number of daily searches that any single user can pull. I know, rather draconian. Therefore I am gathering all of the raw data in three separate data pulls from Splunk and making extensive use of Perl to parse the raw flat files into usable .csv files.
I then ingest the three large .csv files put them into the three hash tables (takes about 6 minutes) hoping to enable efficient grouping into a single hash table with each key to be associated with three values.
I am stumped
Again:
hash_1 (input)
key => [value_hash_1]hash_2 (input)
key => [value_hash_2]hash_3 (input)
key => [value_hash_3]hash_4 (output)
key => [value_hash_1, value_hash_2, value_hash_3]
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^3: multiple hash compare, find, create
by Eily (Monsignor) on Dec 10, 2018 at 23:57 UTC | |
|
Re^3: multiple hash compare, find, create
by AnomalousMonk (Archbishop) on Dec 10, 2018 at 22:07 UTC |