|
Re: Need to increase the processing speed?
by BrowserUk (Patriarch) on Jun 29, 2005 at 08:12 UTC
|
The optimisation is evil crowd might suggest
- Buy or rent a faster processor
- Make use of more than one processor.
If that's not convenient, then probably the best way is to make your program do less.
Making your program do less usually involves simplifying your code by modifying your algorithm.
Another effective way, assuming that your program processes that same datafile more than once, is to index the file so that you can quickly find the bits you need each time.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
| [reply] |
Re: Need to increase the processing speed?
by Zaxo (Archbishop) on Jun 29, 2005 at 05:38 UTC
|
What does it need to do and how fast does it need to be?
You probably can't increase the processor speed.
| [reply] |
|
|
Hi zaxo,
i want to clarify the doubt that
is any processor(system) related commands available
to speed up the execution of program
so that i was asked this question.
any way thanks for your suggestion.
| [reply] |
Re: Need to increase the processing speed?
by sk (Curate) on Jun 29, 2005 at 06:54 UTC
|
are you asking about UNIX commands that can speed up processes? Try man nice and man renice Note that this is not "improving your program's preformance but just a temporary hack to give your program a higher priority.
cheers
SK
Update: going back to holli's point. If your code is not extermely complicated to post (length wise) then it would be better to for the monk's to look at it and then suggest some improvements. You mention 2GB file size. Is the program writing back to disk also or is it ready only? If you are writinig a lot of stuff and your system's IO is not that great try writing out compressed file. Also are you trying to read entire file in mem? etc. code will surely help! | [reply]
[d/l]
[select] |
|
|
Unless your system is specially configured, under most situations 'nice' and 'renice' can only be used to slow down a process (make it behave more nicely, and give up its cycles). Root is the only user (or someone with root-like powers), who can increase the priority of a process to a level above its default value (in the scope of 'nice', giving it a lower 'nice' value).
Unfortunately, most systems are set up so that everything runs at the highest priority -- so you have to renice everything else that is running, and can't just tweek the one program. (some multi-user systems are set up differently, but I don't know of any OSes that come set up this way out of the box). But of course, if you renice the wrong stuff, that's actually needed by the OS, you can have bad things happen.
I'd say the way to go is hardware related ( faster disk, more memory, etc ), or reworking the program in question to deal with whatever the bottleneck is. If the system has been up for a while, a reboot before the process might help if anything leaked memory.
| [reply] |
Re: Need to increase the processing speed?
by holli (Abbot) on Jun 29, 2005 at 06:38 UTC
|
Get a faster harddisk. that is likely to have a greater influence than using a faster cpu.
Besides that, maybe your program is just inefficient? Show us your code.
| [reply]
[d/l] |
Re: Need to increase the processing speed?
by TedPride (Priest) on Jun 29, 2005 at 12:36 UTC
|
You haven't supplied enough information for us to give a good answer. What exactly are you trying to do to the file? There are undoubtedly ways to make your code run faster, but if we don't know what your code does, we can't suggest ways to optimize it. If you're reading line by line, for instance, you can speed things up some by reading multiples of the disk sector size and then splitting.
Short of that, get a faster hard disk, like holli suggests. This will most likely give you the largest speed improvement. | [reply] |
Re: Need to increase the processing speed?
by kprasanna_79 (Hermit) on Jun 29, 2005 at 08:45 UTC
|
Hai Perlsen,
I dont think there is any command to increase processor speed...so better go for higher configuration, if u want to do soo...
And more over if my understanding is correct. U can read the file and process line by line instead of processing the whole file to increase performance ....
say...
open(FIN,">monster_file) || die "Cant open";
while(<FIN>) {
$line = $_;
.....
.....
}
The above mentioned is to give an idea on how to handle the large files..wrap up ur ideas over it..
--Prasanna.K | [reply]
[d/l] |
Re: Need to increase the processing speed?
by rjsaulakh (Beadle) on Jun 29, 2005 at 11:30 UTC
|
the first thing is u need to decrease the processing time not increase
opening a file as big as 2 mb can take time so
you can split the files and get them into an array and then pass this array to ur text processing code
hope that helps
| [reply] |
|
|
You seem to be confused. Merely opening a file should be constant time, irrelevant of how much space it takes up on the disk.
| [reply] |
|
|
Hi rjsaulakh,
Can you please elaborate the way to split files and get them into an array ?? and passing this array to process code...
A necessary code will certainly help.
Thanks in Advance,
xnavala
| [reply] |
|
|
Hi rjsaulakh,
Can you please help me by giving necessary code in order to split the file and get them into an array and to pass this array to process the code. The file which i am using is an unix file and it contains some 14k lines of code.
Thanks in Advance,
xnavala
| [reply] |
|
|
Hi rjsaulakh,
Can you please help me by giving necessary code in order to split the file and get them into an array and to pass this array to process the code. The file which i am using is an unix file and it contains some 14k lines of code.
Thanks in Advance,
xnavala
| [reply] |
|
|
This thread is 8 years old. Maybe you should start a new one for your question?
| [reply] |