Re^2: Persistent timed events

Thanks, I think I'm being dense here, do you mean create 1440 directories per date and then have the actioning servers pick files out of the due directory until it's empty? Otherwise I'd have tens of thousands of files per date-directory. Or do you mean several records per file?

Great idea about the time-sharing servers, I'll definitely use that.

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian W. Kernighan

Comment on Re^2: Persistent timed events

Replies are listed 'Best First'.
Re^3: Persistent timed events by BrowserUk (Patriarch) on Nov 03, 2005 at 15:57 UTC
No. One directory per date, one file per message named for the time (to the minute; hence the 1440) of the action + disambiguator. This was based upon the assumption that your "several hundred thousand" figure would be spread across several days. `20051103 - 0502.xxx1 0502.xxx2 0503.xxx1 0503.xxx2 0505.xxx1 0506.xxx1 20051104 - ...` [download] If I was misreading you, and your mean several hundred thousand per day, and daily directories would be too big, then I'd sub-divide them into hours `20051103 - 00 - 01 .... - 23 - 5901.xxx1 - 5901.xxx2 - 5902.xxx1` [download] I guess you could go to minutes, if required to keep the final directory sizes small for searching. Having 1440 subdirs in a directory isn't a problem as you are never searching that space, just going through it with a hard-coded path. I'd stick to one record per file, to avoid the need for using (cooperative) locking, as it only takes for someone to use a non-cooperating utility at an inappropriate moment to create a dealock situation. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal? "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: Persistent timed events
by BrowserUk (Patriarch) on Nov 03, 2005 at 15:57 UTC

No. One directory per date, one file per message named for the time (to the minute; hence the 1440) of the action + disambiguator. This was based upon the assumption that your "several hundred thousand" figure would be spread across several days.

20051103 - 0502.xxx1
           0502.xxx2
           0503.xxx1
           0503.xxx2
           0505.xxx1
           0506.xxx1
20051104 - ...
[download]

If I was misreading you, and your mean several hundred thousand per day, and daily directories would be too big, then I'd sub-divide them into hours

20051103 - 00
         - 01
         ....
         - 23 - 5901.xxx1
              - 5901.xxx2
              - 5902.xxx1
[download]

I guess you could go to minutes, if required to keep the final directory sizes small for searching. Having 1440 subdirs in a directory isn't a problem as you are never searching that space, just going through it with a hard-coded path.

I'd stick to one record per file, to avoid the need for using (cooperative) locking, as it only takes for someone to use a non-cooperating utility at an inappropriate moment to create a dealock situation.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

[reply]
[d/l]
[select]