Efficiency issues in text parsing

texuser, while your code is correct and it is great that you now found a solution on your own, I do have some thoughts to share with you on your code:

When checking whether opening of files was successful, you should provide the name and possibly the mode, i.e. instead of writing open(IN, "<temp.in") || die "\n Can't open file\n $!\n"; rather write something like open(IN, '<', "temp.in") or die "Error opening file temp.in for input: $!\n";. This helps you (and the user) fixing the error more quickly as the code grows and you possibly migrate into another environment where your read/write access rights vary.
You shoud do this error checking for all files that you open, especially for output files, since most of the times when you have the right to write, you do have the right to read, but not neccessarily vice versa.
You should open a file only when you need it, i.e. open the output file after the second thime you open the input file.
Some RegEx efficiency issues: In the first RegEx [^<]+ would be faster than .+?, since the RegEx engine does not have to check the rest of the RegEx (well, the next character) before proceding to the next possible number of charachters. In the second RegEx, something similar for .*: You might want to substitute it by [^<]+, so that the RegEx engine oes not have to waste time on backtracking. But those are only speed considerations, which just matter if you have large or numerous files.
You open and read the input file twice. Remember that disk read/write operations are much slower than the execution of a piece of program code. If possible, you should try to read the file only once.

Last thing: I tested both your and my code on a file named "temp.in", and both worked. The input file consisted of your three input lines, the output file of your three output lines in both cases. Considering the sbove, if you would like to get the program running, as to parse your input file in one go, I'llbe happy to help you.
Oh, yes, and a ++ for you for putting together your own piece of code that works for you.
Cheers, CombatSquirrel.

Comment on Efficiency issues in text parsing Select or Download Code

Replies are listed 'Best First'.
Loop and Find & Replace by texuser74 (Monk) on Aug 25, 2003 at 04:33 UTC
Hi, Thanks for your comments. One more small doubt i have. here is the input data `<input> This is to test. this is to test <p>This is to test. This is to test</p> <p>This is to test. This is to test</p> This is to test. this is to test </input> <output> This is to test. this is to test <p>This is to test. This is to test</p> <p>This is to test. This is to test</p> This is to test. this is to test </output>` [download] i.e. i want to make the <p>...</p> as single line. i mean delete the carrage returns only inside <p>...</p> my following code does the job, but only for the last <p>...</p>. i don't know how to loop it here. pls suggest `$infile = $ARGV[0]; open(IN, '<', "temp.in") \|\| die "\nCan't open temp.in \n"; open(OUT, '>' "temp.out"); $/=""; while(<IN>) { if($_=~s/(.)<p>(.)\<\/p\>(.*)//ms) { $pre = $1; $par = $2; $pos = $3; $par=~s#\n# #ig; print OUT "$pre<p>$par\<\/p\>$pos"; } } close(IN); close(OUT);` [download] Note: also please let me know how to include the source code in this page, any special tags for that? i mean the code formatting is often getting messed when i post _{edited by ybiC: Reformatted - balanced <code> tags around sample input and code}	[reply] [d/l] [select]
Re: Re: Efficiency issues in text parsing by CombatSquirrel (Hermit) on Aug 25, 2003 at 08:51 UTC
Your formatting is pretty much screwed up. To be honest, I don't see what you are saying. Try to fix it and I'll do my best to help you. In the meantime I'd recommend the HTML::TokeParser Tutorial. In general, if you do extensive HTML or XML processing, consider using a module. Cheers, CombatSquirrel.	[reply]
Re: Efficiency issues in text parsing by texuser74 (Monk) on Aug 27, 2003 at 00:44 UTC
Hi, Thanks for your comments. One more small doubt i have. here is the input data `<input> This is to test. this is to test <p>This is to test. This is to test</p> <p>This is to test. This is to test</p> This is to test. this is to test </input>` [download] `<output> This is to test. this is to test <p>This is to test. This is to test</p> <p>This is to test. This is to test</p> This is to test. this is to test </output>` [download] i.e. i want to make the <p>...</p> as single line. i mean delete the carrage returns only inside <p>...</p> my following code does the job, but only for the last <p>...</p>. i don't know how to loop it here. pls suggest `$infile = $ARGV[0]; open(IN, '<', "temp.in") \|\| die "\nCan't open temp.in \n"; open(OUT, '>' "temp.out"); $/=""; while(<IN>) { if($_=~s/(.)<p>(.)\<\/p\>(.*)//ms) { $pre = $1; $par = $2; $pos = $3; $par=~s#\n# #ig; print OUT "$pre<p>$par\<\/p\>$pos"; } } close(IN); close(OUT);` [download] Note: also please let me know how to include the source code in this page, any special tags for that? i mean the code formatting is often getting messed when i post _{edited by ybiC: Reformatted to avoid lateral scrolling in browser window - balanced <code>tags around example input+output and code}	[reply] [d/l] [select]
Re: Re: Efficiency issues in text parsing by CombatSquirrel (Hermit) on Aug 27, 2003 at 01:04 UTC
Re: Efficiency issues in text parsing by texuser74 (Monk) on Aug 27, 2003 at 06:25 UTC
Some notes below your chosen depth have not been shown here