how to read data for every 280 character ?

bh_perl has asked for the wisdom of the Perl Monks concerning the following question:

Hi..

I want to read every 280 data but I just found that my code captured the character including the new line. Then once I remove the new line then the character will be less from 280 to 27?. That is my problem..

So, how could i removed the new line and captured 280 data ? anyone have any ideas ?

local $/ = \280 if ($decode !~ /csv/);

$printout = "";
if ($outfile) {
    open (OUTDATA, ">$outfile");
    $printout = 1;
}

if ($fname && (defined($decode) || defined($encode))) {
    open (DATA, "$fname");
    while (my $data = <DATA>) {
        chomp($data);
        if (defined($decode) && $decode =~ /csv/) {
            csvout($data);
        } elsif (defined($decode) && $decode !~ /csv/) {
            $data =~ s/\n//g;
            print "$data"; # ..Problem is here..
            #decode($data);
        } elsif (defined($encode) && $encode =~ /csv/) {
            csv_encode($data);
        } elsif (defined($encode) && $encode !~ /csv/) {
            std_encode($data);
        }
    }
    close (DATA);
} else {
    usage();
[download]

20080818 Janitored by Corion: Removed PRE tags, as per Writeup Formatting Tips

Comment on how to read data for every 280 character ? Download Code

Replies are listed 'Best First'.
Re: how to read data for every 280 character ? by dHarry (Abbot) on Aug 18, 2008 at 07:51 UTC
So what kind of data are you trying to read? (The 280 sounds very familiar to me.) Like anonymous monk suggests open the file in binmode. In fact it doesn’t harm to always do this. You could for example read the bytes (skipping the new lines) and count the number of bytes you have read. So you read byte for byte with a while loop and when you encounter a new line you use a next to continue with the next iteration. HTH	[reply]
Re^2: how to read data for every 280 character ? by bh_perl (Monk) on Aug 18, 2008 at 07:59 UTC
Its Huawei NGN data.. do you familiar with it ? Actually it is a binary data... i used xxd command first to convert from bin to hex then i used perl to read the data.. is it a good idea ?	[reply]
Re^3: how to read data for every 280 character ? by Anonymous Monk on Aug 18, 2008 at 08:10 UTC
Don't bother using xxd. Perl can read in binary files just fine if you use binmode.	[reply]
Re^4: how to read data for every 280 character ? by bh_perl (Monk) on Aug 19, 2008 at 01:50 UTC
Re^5: how to read data for every 280 character ? by blazar (Canon) on Aug 19, 2008 at 11:06 UTC
Re: how to read data for every 280 character ? by JavaFan (Canon) on Aug 18, 2008 at 10:17 UTC
You are reading in chucks of 280 bytes at the time. From those chucks, you remove newlines, and then you are surprised you're left with chucks that are less than 280 characters. If you would have a huge container of marbles, of all kinds of different colours; you then scope out 280 marbles, and from those 280, you remove all the yellow ones, will you then be surprised if you ended up with 276 marbles? If you read in 280 bytes in a string, and you want to keep 280 bytes, don't go removing bytes from it.	[reply]
Re: how to read data for every 280 character ? by pat_mc (Pilgrim) on Aug 18, 2008 at 07:54 UTC
Hi, bh_perl - I am not sure I fully understand what you are trying to do - and in which framework you are trying to do it. My guess, however, would be that you have a typical end-of-line type of problem. Could it be that your are using data from different operating systems including Windows? Remember that the line break in Linux is `\n` while unser Windows it is `\n\r`. Removing just one type of line breaks from a file containing mixed line break types may therefore result in unpredictable results. I have had this kind of problems many times myself (in fact, you may want to have a look at one of my earlier posts). Hope this helps. Cheers - Pat	[reply] [d/l] [select]
Re^2: how to read data for every 280 character ? by Anonymous Monk on Aug 18, 2008 at 08:14 UTC
would be that you have a typical end-of-line type of problem. `$/=280;` means readline reads 280 bytes at a time, not lines (so line-ending characters don't matter). `C:\>perl -e"die unpack q~H*~, $/" 0a at -e line 1.` [download]	[reply] [d/l] [select]
Re: how to read data for every 280 character ? by Anonymous Monk on Aug 18, 2008 at 07:42 UTC
You have opened your file in binary mode, yes? See `binmode`.	[reply]
Re: how to read data for every 280 character ? by Anonymous Monk on Aug 18, 2008 at 07:12 UTC
But, once i tried to calculate the char length it become 276 not 280.. why its become like this ?.. Because you modified $data ? Because $data is unicode and 280 bytes is 276 chars in unicode?	[reply]
Re: how to read data for every 280 character ? by TGI (Parson) on Aug 18, 2008 at 16:19 UTC
You may wish to consider using read instead of grabbing whole lines. Something like this untested code ought to do the trick. `my $data; read( $fh, $data, 280 ); #read in initial 280 bytes. while( my $count = $data =~ s/\n//g ) { read( $fh, $data, $count, length $data ); }` [download] TGI says moo	[reply] [d/l]