Hi Monks,

I recently implement the Byte allign compression and i want your opinion if i used the most efficient way...

Byte Allign Encode ...(compress just 20 docs for example..)
(i dont compress anything else except Dod IDS) 
my $posting;
for my $docid(1 .. 20)
{
    $posting. = ByteEncode($docid);
}
#At the end i pack the whole bit string 
my $compressed = pack "b*",$posting; # i am not sure if its the best f
+ormat...

#Decode the compressed Bit string
my $df = 20; #number of compressed docs..
my $bits=7;  
my $decode = unpack "b*",$posting;
    my @w;my $nr=0;
    my $binary='';
    my $start=0;
    while()
    {
                #ckeck the MSB, if it is 1 then its the last byte for 
+the current Docid
        my $temp = substr($decode,0,$bits+1,'');
        # its the last one...
        if($temp=~ m/^1/)
        {
            substr($temp,0,1,'');
            $binary.=$temp;
            push @w,bin2dec($binary);
            undef $binary;
            $nr++;            
            last if($nr==$df);
        }
                #else remove the MSB and keep the rest and go to the n
+ext byte
        else
        {
            substr($temp,0,1,'');
            $binary.=$temp;
        }
        
    }
    return @w;

[download]

My question is how can i read from the bit string 1 byte or n Bits per time and how can i check the MSB from the packed format without to unpack it and work it with the substr..

Another thing that i dont know is how can i import delimiters in the bit string because i want to save for each doc id the corresponding positions for each term..

Like this way: Term1 -> DocID1PositionsDocID2Positions..e.t.c..

Thanks


In reply to Byte allign compression in Perl.. by MimisIVI

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.