in reply to Whats quicker - passing along as variable to sub, or global var?
My recommendation would be that you should use some form of a database/external-file structure here. In situations like these, I happen to find SQLite to be very hard to beat, even when I am creating a file on-the-spot as working storage and demolishing the whole thing at the end. (P.S.: when using SQLite, transactions are very crucial to performance.)
You grab each article into memory, work with it, put it back, and then repeat the process, effectively re-using the same memory pages each time. Disk I/O will still be going on ... since all memory is virtual, you cannot avoid this ... but the nature of that I/O will be in line with what the virtual memory manager and the filesystem manager, respectively, expect and are prepared to deal with. The pages will undoubtedly sit in a stable working-set, while the I/O that takes place will be file I/O against an efficient indexed file. The OS, whatever it is, knows very well how to serve the needs of an application that behaves in this way.
Even though the technical capacity of hardware these days is vastly larger than it once was, it is still the case that, when you exceed those limits or when you go against the software designers’ core assumptions in their design of a subsystem, “it all falls down” rather quickly and badly.
Given that you’re asking the OQ of “what’s quicker,” I know that the app is in-pain already. The answer should be, “it doesn’t matter, and if it does seem to matter, that is not the true root cause of the problem.” The virtual memory subsystem is not happy, so to speak. Fact is, it is crying in pain (and so are any users, if any, who have to share the same system with you), because this app is “going against the grain.”
IMHO.™ HTH.™ TMTOWTDI.™
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Whats quicker - passing along as variable to sub, or global var?
by BrowserUk (Patriarch) on Apr 11, 2011 at 19:04 UTC |