Re: Re: Re: Force perl to release memory back to the operating system

Yes I did make a couple of assumptions about what look like finacial transactions in multiple currencies that are cumulated and then indexed against against customer details to get the most active customer list by transaction value. I can't think why that looked like a DB task :-)

But of course you are right. I won't be putting my web or squid logs into a DB anytime soon although I do rotate them daily and use YAML to serialize data we parse out into flat files so we can get it back as required (rarely and for static HTML anyway)

As always tools and tasks.....

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

Comment on Re: Re: Re: Force perl to release memory back to the operating system

Replies are listed 'Best First'.
Re: Re: Re: Re: Force perl to release memory back to the operating system by demerphq (Chancellor) on Sep 25, 2003 at 12:13 UTC
Yes I did make a couple of assumptions Er, I hope you didnt misunderstand me. My reply wasnt to your whole node or the analysis you offered, just more of a heads up to people that DB's are not the universal pancea for handling large volumes of data. (Not so much for you, as Im familiar with you and was confident you knew the caveats, but for other monks out there that might be tempted to bulk load a few million records just to sum one field in them.) I find theres a tendency amongst DB'ish types to treat DBMS as the only way to solve problems. There are huge classes of problems where using a DBMS is a no brainer (transactions, distribution, integrity, locking etc), but there are also huge classes of problems where a DB is probably not the best solution. As you said. Tools for tasks... Anyway, im hoping this exchange opens one or two minds to alternate approaches which can only be a good thing. Cheers, :-) --- demerphq _{First they ignore you, then they laugh at you, then they fight you, then you win. -- Gandhi}	[reply] [d/l]

Replies are listed 'Best First'.

Re: Re: Re: Re: Force perl to release memory back to the operating system
by demerphq (Chancellor) on Sep 25, 2003 at 12:13 UTC

Yes I did make a couple of assumptions

Er, I hope you didnt misunderstand me. My reply wasnt to your whole node or the analysis you offered, just more of a heads up to people that DB's are not the universal pancea for handling large volumes of data. (Not so much for you, as Im familiar with you and was confident you knew the caveats, but for other monks out there that might be tempted to bulk load a few million records just to sum one field in them.)

I find theres a tendency amongst DB'ish types to treat DBMS as the only way to solve problems. There are huge classes of problems where using a DBMS is a no brainer (transactions, distribution, integrity, locking etc), but there are also huge classes of problems where a DB is probably not the best solution.

As you said. Tools for tasks...

Anyway, im hoping this exchange opens one or two minds to alternate approaches which can only be a good thing.

Cheers, :-)

_{First they ignore you, then they laugh at you, then they fight you, then you win.

-- Gandhi}

[reply]
[d/l]