in reply to Re: Data structure advice
in thread Data structure advice

Hi,

Thanks for the replies.

In this particular case, I define "best" as "quickest lookup" as there are only 48 values per day (I am processing each data separately) and I will be doing the lookup approx. 8000 times per day.

I'd like to do the lookup something like this: my $pressure = get_pressure("2003-10-12 16:09:06"); Where $pressure is set to the most recent pressure value prior to "2003-10-12 16:09:06"

I am prefectly capable of writing a function that searches through a list of values to return the required value. I was hoping that there would be a nice perl trick to save me the bother!!!

R.

--

Robin Bowes | http://robinbowes.com

Replies are listed 'Best First'.
Re: Re: Re: Data structure advice
by dragonchild (Archbishop) on Jan 05, 2004 at 15:17 UTC
    The nice Perl trick is a SQL trick. And, don't write the function - it's already written! Use the tools you have. Plus, once you have it in a database, you're going to find other capabilities. Trust me.

    ------
    We are the carpenters and bricklayers of the Information Age.

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

[reply]
      Hi,

      I am aware of the capabilities of using SQL, but feel that it is overkill for this application.

      I will have, at most, 48 pressure values per day to look up; these will be read from a .csv format file. I could, I agree, use DBI (I think DBD::CSV would be most appropriate) but I can't help thinking that the performance overhead will be too much.

      I suppose I could always just try it, i.e. profile rather than speculating.

      I'll wait to see if there are any other suggestions then knock something up tomorrow.

      Cheers,

      R.

      --

      Robin Bowes | http://robinbowes.com

[reply]
Re: Re: Re: Data structure advice
by duff (Parson) on Jan 05, 2004 at 15:28 UTC

    Hmm. How do you arrive at 48 values per day? The snippet you showed has 3 values per hour, which for 24 hour days is 72 values per day :-)

    duff

[reply]
      Doh! There is a logical explanation for this...

      The data was extracted from the production system, following data processing. The raw data is generated every 30 mins at 20 and 50 mins past the hour. The values on the hour are averages calculated from the previous and following values.

      R.

      --

      Robin Bowes | http://robinbowes.com

[reply]
Re: Data structure advice
by Abigail-II (Bishop) on Jan 05, 2004 at 16:16 UTC
    For a "quickest lookup", one would use a lookup table. Considering your query with second precisions, you'd need to store 62400 entries per day, so if you have to query data from lots of days, you might need to invest in memory. But it gives you the quickest lookup time.

    Abigail

[reply]
      ...you'd need to store 62400 entries per day...

      I think you mean "86400 entries per day" otherwise I have no clue where you pulled that number from.

      duff

[reply]
        Indeed.

        Abigail

[reply]

In Section Seekers of Perl Wisdom