Offset Reading - two files

gio001 has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone, I am a newbie and hope you forgive if this question is simple to your eyes. I have been looking at this code for some time:

perl -sple"BEGIN{open BIG};($undef,$l)=split;read(BIG,$_,$l)" -- -BIG=bigfile.dat index.dat >outfile.dat

The index file contains two columns, one for the offset and the second for a number of characters to read. The code works fine but it seems to be processing the bigfile sequentially, by this I mean it starts at 0 reads the first value in column 2 of the index file and writes out that many characters, next it continues from there and writes out the next group of x characters as identified by the next value in column 2 of the index file.

How can I force a reposition of the read inside the big file by using the value in column 1 of the index file anytime a new read has to occur?

Can the logic be added to the one liner above?

Additionally how do I add a CR to each record that is going to the output file?

Thanks to all.

Comment on Offset Reading - two files Download Code

Replies are listed 'Best First'.
Re: Offset Reading - two files by BrowserUk (Patriarch) on Dec 14, 2012 at 14:13 UTC
Let's see a little history: In 2008, you asked; I replied. Two and a half years later you re-posted my code and asked. And now nearly two years on, you do the same again. You haven't explained what you are trying to do; shown examples of the files in question; or explained why it should be a one-liner. You seem to have unlimited time to solve the problem; but also seem to have made no progress in the intervening 5 years. What are you actually trying to solve? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP Neil Armstrong	[reply]
Re^2: Offset Reading - two files by gio001 (Acolyte) on Dec 14, 2012 at 14:39 UTC
OK, yes I understand your valid points. Here is a little history: when I originally asked you the question I was facing a problem on retrieving data from a large file whose record structure information was stored in an index file. At that time the structure of the index file was sequential that is the next record was always starting right after the end of the prior record, so no offset was actually involved. The code you supplied was perfectly doing the retrieval by sequentially picking up chunks from the big file. Now the structure of the big file has been changed by the vendor and there is extraneous data in between each record for which I have to make use of the offset information each time before I can get the record I am interested in. I understand your surprise and the disappointment for my ignorance, I can only say I wish I had more time to dedicate to the discovery of Perl abilities ..... Thanks for all you did	[reply]
Re^3: Offset Reading - two files by BrowserUk (Patriarch) on Dec 14, 2012 at 14:50 UTC
Now the structure of the big file has been changed by the vendor and there is extraneous data in between each record for which I have to make use of the offset information each time before I can get the record I am interested in.... I wish I had more time to dedicate to the discovery of Perl abilities That change appears to have happened 20 months ago. How have you been handling the problem in the interim? Have you heard of seek? With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP Neil Armstrong	[reply]
Re^4: Offset Reading - two files by gio001 (Acolyte) on Dec 14, 2012 at 15:04 UTC
Re^5: Offset Reading - two files by BrowserUk (Patriarch) on Dec 14, 2012 at 15:10 UTC
Some notes below your chosen depth have not been shown here
Re^4: Offset Reading - two files by gio001 (Acolyte) on Dec 14, 2012 at 15:06 UTC
Re: Offset Reading - two files by Anonymous Monk on Dec 14, 2012 at 14:08 UTC
As Tutorials: Basic debugging checklist teaches, use B::Deparse to see how perl interprets the code Since you did not know about B::Deparse you are not qualified to write oneliners, so stop that :P	[reply]