AssFace has asked for the wisdom of the Perl Monks concerning the following question:

I have some scripts that go and grab data, and then do work on that data. They currently look to see if it is a weekend, and if so, they kill themselves quietly.

But if there is a weekday holiday, they will still execute, happily downloading and analyzing duplicate data, blissfully ignorant to their wasted effort.

Specifically I am working with stock market data, and so I would like to add in code that will take into account if "today" is a holiday that the stock markets observe and therefore are closed, meaning the script should die quietly.

I have an account on a Pair server, so I have access to a huge number of modules - I sifted through their list and then popped over to the CPAN Search as well.
I've looked through a bit of the Date::Calendar and its friends, but I'm curious if anyone here knows of:
1) ideally a module that is stock market specific that has holiday info
or
2) a module that is holiday smart (and then I will then add the extra code on my side to be aware of the stock market code)

Thanks!

-Eric



-------------------------------------------------------------------
There are some odd things afoot now, in the Villa Straylight.

Replies are listed 'Best First'.
Re: Is there a "Holiday" module?
by Solo (Deacon) on Apr 25, 2003 at 21:12 UTC
    Date::Manip has decent support for holiday calendars. Calendars are defined in "GlobalCnf" files (as the POD calls them).

    # example holiday section in file monk.conf *Holiday 1/1 = New Year's Day third Monday in Feb = Presidents' Day fourth Thu in Nov = Thanksgiving 4/25 = The bestest holiday ever! # The Friday after Thanksgiving is an unnamed holiday most places fourth Thu in Nov + 1 day = 1*0:0:0:0:0:0*EASTER = Easter 1*11:0:11:0:0:0*CWD = Veteren's Day (observed) 1*0:0:0:0:0:0*EASTER,PD5 = Good Friday # quickie Perl script use Date::Manip; Date_Init("TZ=EDT","GlobalCnf=monk.conf"); if ( Date_IsHoliday(ParseDate('today')) ) { print "Today is " . Date_IsHoliday(ParseDate('today')); }

    Update: Added the code example.

    --Solo

    --
    There's no mystical energy field that controls my destiny. It's all a lot of simple tricks and nonsense.

Re: Is there a "Holiday" module?
by robartes (Priest) on Apr 25, 2003 at 20:41 UTC
    Have a look at Date::Calendar. You can pass a hash containing a 'holiday profile'. There are some profiles predefined in Date::Calendar::Profiles. The docs for that also contain a way to define your own profiles, which seems to be what you want.

    BTW, I got all of this by typing 'holidays' into http://search.cpan.org - this is a great resource.

    CU
    Robartes-

      Okay thanks - I'll give that a look. I did in fact look at that site, and at Date::Calendar, but apparently didn't read that one :)



      -------------------------------------------------------------------
      There are some odd things afoot now, in the Villa Straylight.
•Re: Is there a "Holiday" module?
by merlyn (Sage) on Apr 25, 2003 at 20:48 UTC
    The Holiday schedule for a bank will differ from the Holiday schedule for the stock market or a large high-tech corporation. There's no universal holiday schedule.

    Hence, the only real answer is to figure out some other way to know if the data you're pulling is new or stale.

    And then you won't need a Holiday module.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

      Well, technically, in the case of these scripts - there is a universal holiday schedule.
      The scripts take in data from the NYSE, NASDAQ, and AMEX exchanges and nothing else. Those markets share the same holiday schedule.

      I was trying to weigh out whether it was better to check that and never hit the net (do more work), or if it was better to download something and determine it from that.

      There are arguements for and against each - but I suppose the best arguement for the "always download" approach isn't that they don't share the same holiday schedule - but the case where one is closed for a reason that the others wouldn't close (ie something happens to the physical structure of one and not the others). Even then - historically, they all close... but just in case, I suppose that is the arguement that the data should determine it in this case.

      Were there banks or other markets involved, it wouldn't be the case and I'd agree with you - but in this case, I think the holidays would work just fine.
      I was mainly curious if someone had already done a holiday module to save me time/effort - and as someone pointed out, it looks like there is something in part of the Date::Calendar module.

      -------------------------------------------------------------------
      There are some odd things afoot now, in the Villa Straylight.
        AssFace,
        I think merlyn, and others, are right for a different reason. Even though there is no universal Holiday schedule, you could possibly find, or more likely, build your own that fits your situation. What is the problem with that?

        Automation is only as smart as the person who programmed it.

        During the 9/11 tragedy the markets were closed for an "un-scheduled outage". If your code can't efficiently determine that today is a down day - you will eventually end up having your script waste time and resources.

        Cheers - L~R

        I was mainly curious if someone had already done a holiday module to save me time/effort - and as someone pointed out, it looks like there is something in part of the Date::Calendar module.
        But I think I just pointed out why there can't be exactly what you need. Was some part of my previous post unclear?

        You'll need to determine if the particular place from which you are sucking data is closed. Only that site will know. Anything else is a guess. And can you really afford guesses?

        -- Randal L. Schwartz, Perl hacker
        Be sure to read my standard disclaimer if this is a reply.

Re: Is there a "Holiday" module?
by dws (Chancellor) on Apr 25, 2003 at 23:15 UTC
    I would like to add in code that will take into account if "today" is a holiday that the stock markets observe and therefore are closed, meaning the script should die quietly.

    A safer approach might be to figure out how "the markets are closed today" is reported, and detect that on a page you've pulled. Then you're safe for both holidays and for unscheduled market closures (one of which we've had in recent memory).

Re: Is there a "Holiday" module?
by space_cadet (Initiate) on Apr 25, 2003 at 20:24 UTC
    If the problem is simply that duplicate data is downloaded, you could just attempt a download every day. Match some identifier (such as the date posted) in your newest data to the same identifier in the previous data downloaded. If the identifiers are equal, stop the process and try again tomorrow.
      Yeah, that is actually how this question really came about. I have something like that now that I put in today while it was a slow day at work.

      but that still requires the download of data - and when that happens over thousands of files... well, I'd rather it didn't. I'd rather have them start, see that today isn't their day, and then just close down without ever going to a net connection.

      granted, I could have it do it on the first file, and the "assume" that if the first file (or set it so if it passes some threshold of files that have dupes) has duplicate data, they all are going to, so die out... but as far as I can tell, in programming (and much else in life) assumption is pretty much always a bad call.

      -------------------------------------------------------------------
      There are some odd things afoot now, in the Villa Straylight.
Re: Is there a "Holiday" module?
by digger (Friar) on Apr 25, 2003 at 20:38 UTC
    What about using a small tagfile with the date in it. Just download that small file first. If it's the same as the tagfile that is on the local system, die silently. Then you don't download the big stuff.

    hth,
    digger
Re: Is there a "Holiday" module?
by mojotoad (Monsignor) on Apr 26, 2003 at 16:14 UTC
    You seem to be concerned with needlessly downloading data when it is, in fact, duplicate data on a "down day".

    Why not use a canary index to check for down days? For example, before downloading data for all of your tickers, perform a quick check on the DJIA -- if the DJIA has not been updated, then it's a fair bet none of the tickers will have been updated either.

    That way you eliminate the need for figuring out holidays. You script is adaptive and not wasteful of resources.

    Matt