Re^7: encoding hdmi video byte

Okay. I took your spec and coded this slow and labourious implementation and applied it to the bytes 0 .. 255:

#! perl -slw
use strict;
use Data::Dump qw[ pp ];

my %map;

for my $i ( 0 .. 255 ) {
    my @Q;
    my @D = split '', unpack 'b8', chr $i;
    $Q[ 0 ] = $D[ 0 ];
    $Q[ 1 ] = $D[ 1 ] ^ $Q[ 0 ];
    $Q[ 2 ] = $D[ 2 ] ^ $Q[ 1 ];
    $Q[ 3 ] = $D[ 3 ] ^ $Q[ 2 ];
    $Q[ 4 ] = $D[ 4 ] ^ $Q[ 3 ];
    $Q[ 5 ] = $D[ 5 ] ^ $Q[ 4 ];
    $Q[ 6 ] = $D[ 6 ] ^ $Q[ 5 ];
    $Q[ 7 ] = $D[ 7 ] ^ $Q[ 6 ];
    my $outb = ord pack 'b8', "@Q";
    $map{ $i } = $outb;
}

pp \%map;
[download]

And it produced this output:

{
  "0" => 0,
  1   => 85,
  2   => 84,
  3   => 1,
  4   => 80,
  5   => 5,
  6   => 4,
  7   => 81,
  8   => 64,
  9   => 21,
  10  => 20,
  11  => 65,
  12  => 16,
  13  => 69,
  14  => 68,
  15  => 17,
  16  => 0,
  17  => 85,
  18  => 84,
  19  => 1,
  20  => 80,
  21  => 5,
  22  => 4,
  23  => 81,
  24  => 64,
  25  => 21,
  26  => 20,
  27  => 65,
  28  => 16,
  29  => 69,
  30  => 68,
  31  => 17,
  32  => 0,
  33  => 85,
  34  => 84,
  35  => 1,
  36  => 80,
  37  => 5,
  38  => 4,
  39  => 81,
  40  => 64,
  41  => 21,
  42  => 20,
  43  => 65,
  44  => 16,
  45  => 69,
  46  => 68,
  47  => 17,
  48  => 0,
  49  => 85,
  50  => 84,
  51  => 1,
  52  => 80,
  53  => 5,
  54  => 4,
  55  => 81,
  56  => 64,
  57  => 21,
  58  => 20,
  59  => 65,
  60  => 16,
  61  => 69,
  62  => 68,
  63  => 17,
  64  => 0,
  65  => 85,
  66  => 84,
  67  => 1,
  68  => 80,
  69  => 5,
  70  => 4,
  71  => 81,
  72  => 64,
  73  => 21,
  74  => 20,
  75  => 65,
  76  => 16,
  77  => 69,
  78  => 68,
  79  => 17,
  80  => 0,
  81  => 85,
  82  => 84,
  83  => 1,
  84  => 80,
  85  => 5,
  86  => 4,
  87  => 81,
  88  => 64,
  89  => 21,
  90  => 20,
  91  => 65,
  92  => 16,
  93  => 69,
  94  => 68,
  95  => 17,
  96  => 0,
  97  => 85,
  98  => 84,
  99  => 1,
  100 => 80,
  101 => 5,
  102 => 4,
  103 => 81,
  104 => 64,
  105 => 21,
  106 => 20,
  107 => 65,
  108 => 16,
  109 => 69,
  110 => 68,
  111 => 17,
  112 => 0,
  113 => 85,
  114 => 84,
  115 => 1,
  116 => 80,
  117 => 5,
  118 => 4,
  119 => 81,
  120 => 64,
  121 => 21,
  122 => 20,
  123 => 65,
  124 => 16,
  125 => 69,
  126 => 68,
  127 => 17,
  128 => 0,
  129 => 85,
  130 => 84,
  131 => 1,
  132 => 80,
  133 => 5,
  134 => 4,
  135 => 81,
  136 => 64,
  137 => 21,
  138 => 20,
  139 => 65,
  140 => 16,
  141 => 69,
  142 => 68,
  143 => 17,
  144 => 0,
  145 => 85,
  146 => 84,
  147 => 1,
  148 => 80,
  149 => 5,
  150 => 4,
  151 => 81,
  152 => 64,
  153 => 21,
  154 => 20,
  155 => 65,
  156 => 16,
  157 => 69,
  158 => 68,
  159 => 17,
  160 => 0,
  161 => 85,
  162 => 84,
  163 => 1,
  164 => 80,
  165 => 5,
  166 => 4,
  167 => 81,
  168 => 64,
  169 => 21,
  170 => 20,
  171 => 65,
  172 => 16,
  173 => 69,
  174 => 68,
  175 => 17,
  176 => 0,
  177 => 85,
  178 => 84,
  179 => 1,
  180 => 80,
  181 => 5,
  182 => 4,
  183 => 81,
  184 => 64,
  185 => 21,
  186 => 20,
  187 => 65,
  188 => 16,
  189 => 69,
  190 => 68,
  191 => 17,
  192 => 0,
  193 => 85,
  194 => 84,
  195 => 1,
  196 => 80,
  197 => 5,
  198 => 4,
  199 => 81,
  200 => 64,
  201 => 21,
  202 => 20,
  203 => 65,
  204 => 16,
  205 => 69,
  206 => 68,
  207 => 17,
  208 => 0,
  209 => 85,
  210 => 84,
  211 => 1,
  212 => 80,
  213 => 5,
  214 => 4,
  215 => 81,
  216 => 64,
  217 => 21,
  218 => 20,
  219 => 65,
  220 => 16,
  221 => 69,
  222 => 68,
  223 => 17,
  224 => 0,
  225 => 85,
  226 => 84,
  227 => 1,
  228 => 80,
  229 => 5,
  230 => 4,
  231 => 81,
  232 => 64,
  233 => 21,
  234 => 20,
  235 => 65,
  236 => 16,
  237 => 69,
  238 => 68,
  239 => 17,
  240 => 0,
  241 => 85,
  242 => 84,
  243 => 1,
  244 => 80,
  245 => 5,
  246 => 4,
  247 => 81,
  248 => 64,
  249 => 21,
  250 => 20,
  251 => 65,
  252 => 16,
  253 => 69,
  254 => 68,
  255 => 17,
}
[download]

That's a byte to byte mapping table. I then used that to build the translation table I posted.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"I'd rather go naked than blow up my ass"

Comment on Re^7: encoding hdmi video byte Select or Download Code

Replies are listed 'Best First'.
Re^8: encoding hdmi video byte by AnomalousMonk (Archbishop) on Feb 24, 2010 at 23:54 UTC
A couple of points I don't understand about Re^7: encoding hdmi video byte. First, in the statement `my $outb = ord pack 'b8',``"@Q"``;` will not `"@Q"` interpolate the array with spaces between the elements of `@Q` and so give an oddball result (i.e., only 16 unique values) since a space (ASCII space == 0x20) is packed by `b` as a 0 bit and half the elements of `@Q` will be ignored? I.e., shouldn't it be `my $outb = ord pack 'b8',``join '', @Q``;` instead? The other point I don't understand is that I took the OP statement `Data byte D(0:7) encode this to make 9bit data q_m(0:8) q_m[0] = D(0); q_m(1) = q_m(0) XOR D(1); q_m(2) = q_m(1) XOR D(2); q_m(3) = q_m(2) XOR D(3); q_m(4) = q_m(3) XOR D(4); q_m(5) = q_m(4) XOR D(5); q_m(6) = q_m(5) XOR D(6); q_m(7) = q_m(6) XOR D(7); q_m(8) = 1;` [download] to be pseudo-code, and to be the equivalent (on a byte-for-byte basis) of `$q_m_byte = (0x100 \| ($D_byte ^ (0xff & ($q_m_byte << 1))));` [download] with a `$q_m_byte` 'byte' of 9 bits always ending up with its bit-8 set, and bit-7 of the original value of `$q_m_byte` always being lost. But `$Q[ 0 ] = $D[ 0 ]; $Q[ 1 ] = $D[ 1 ] ^ $Q[ 0 ]; $Q[ 2 ] = $D[ 2 ] ^ $Q[ 1 ]; $Q[ 3 ] = $D[ 3 ] ^ $Q[ 2 ]; ...` [download] seems to be the equivalent of `$Q[ 0 ] = $D[ 0 ]; $Q[ 1 ] = $D[ 1 ] ^ $D[ 0 ]; $Q[ 2 ] = $D[ 2 ] ^ $D[ 1 ] ^ $D[ 0 ]; $Q[ 3 ] = $D[ 3 ] ^ $D[ 2 ] ^ $D[ 1 ] ^ $D[ 0 ]; ...` [download] which is very different.	[reply] [d/l] [select]
Re^9: encoding hdmi video byte by BrowserUk (Patriarch) on Feb 25, 2010 at 00:12 UTC
You're right on the `"@Q"` part. Dumb mistake. With that corrected, the mapping makes a lot more sense. As for the right interpretation of the spec, I guess you'll have to ask the OP. I believe, (with the "deliberate" error:) and constant 9th bit excepted), I coded a direct implementation of the spec. If the spec is wrong, or my interpretation of it is wrong, then the OP will have to correct it. The main point was to answer the OPs question of the best way to perform the mapping efficiently, and that is tr/// with the appropriate translation strings. On a gig of data, doing 10 x 100MB tr///s, (or even 1 x 1GB if he has the memory), is going to beat performing half a dozens boolean operations on every byte, hands down. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "I'd rather go naked than blow up my ass"	[reply] [d/l]
Re^9: encoding hdmi video byte by hdmiguru (Initiate) on Feb 25, 2010 at 16:36 UTC
Hi AnomalousMonk, Here is the quote from my req... "...the LSb of the output (Q[0] = D[0]) matches the LSb of the input. With the starting value established, the remaining 7bits of the output (9th bit always 1) is derived from sequential XORing of each bit of the input with the previously derived bit...." Therefore i coded it this way. In a very simple way. pls let me know if it is clear now. Also i was looking for some input to make my code smaller/smarter than it is now. my %map; for my $i ( 0 .. 255 ) { my @Q = qw(0 0 0 0 0 0 0 0 0); my $result; #Step 1 : $Q[0] = D[0] $Q[0] = ($i & 0x01); #Step 2 : $Q[1] = D[1] ^ $Q[0] $Q[1] = (($i & 0x02) ^ ($Q[0] << 1)); #Step 3 : $Q[2] = D[2] ^ $Q[1] $Q[2] = ($i & 0x04) ^ ($Q[1] << 1); #Step 4 : $Q[3] = D[3] ^ $Q[2] $Q[3] = ($i & 0x08) ^ ($Q[2] << 1); #Step 5 : $Q[4] = D[4] ^ $Q[3] $Q[4] = ($i & 0x10) ^ ($Q[3] << 1); #Step 6 : $Q[5] = D[5] ^ $Q[4] $Q[5] = ($i & 0x20) ^ ($Q[4] << 1); #Step 7 : $Q[6] = D[6] ^ $Q[5] $Q[6] = ($i & 0x40) ^ ($Q[5] << 1); #Step 8 : $Q[7] = D[7] ^ $Q[6] $Q[7] = ($i & 0x80) ^ ($Q[6] << 1); #Step 9 : $Q[8] = 1 $Q[8] = 0x100; $result = ($Q[0] \| $Q[1] \| $Q[2] \| $Q[3] \| $Q[4] \| $Q[5] \| $Q[6] \| + $Q[7] \| $Q[8]); $map{ $i } = $result; #print "$result "; } pp \%map; [download]	[reply] [d/l]
Re^10: encoding hdmi video byte by AnomalousMonk (Archbishop) on Feb 25, 2010 at 18:16 UTC
My (mis)understanding on first reading the OP was that the needed data transformation was q_m = f(q_m, D), whereas BrowserUk understood it to be q_m = f(D). Looking at Re^9: encoding hdmi video byte, it is clear that BrowserUk had the right idea, and a quick check of the output of this implementation and BrowserUk's (after the `"@Q"` bit is fixed, and ignoring those pesky high bits) shows they are functionally the same. All the rest is as BrowserUk wrote. Just use the info in `%map` to create a `tr///` translator. This you can do by hand or, as I think BrowserUk wrote somewhere, create a translator function on the fly with eval. This will zip through the input data pretty quickly (either in one big Gig or in several smaller chunks). Then, the only problem remaining is the pesky high-order constant bit(s?), bit-8 (and -9?), which, again as BrowserUk wrote, it would be simplest and quickest just to spit out alternately with the transformed input bytes. Happy hdmi-ing!	[reply] [d/l] [select]
Re^8: encoding hdmi video byte by hdmiguru (Initiate) on Feb 24, 2010 at 22:01 UTC
Thanks a lot! Mr. BrowserUK.... that is really a ery big help !!!!	[reply]
Re^9: encoding hdmi video byte by AnomalousMonk (Archbishop) on Feb 25, 2010 at 00:43 UTC
Just to clarify, is the proper interpretation of your OP that of Re^7: encoding hdmi video byte, or is it more like `Data byte D(0:7) encode this to make 9bit data q_m(0:8) new previous bit of bit of q_m(0) = D(0); q_m(1) = q_m(0) XOR D(1); q_m(2) = q_m(1) XOR D(2); q_m(3) = q_m(2) XOR D(3); q_m(4) = q_m(3) XOR D(4); q_m(5) = q_m(4) XOR D(5); q_m(6) = q_m(5) XOR D(6); q_m(7) = q_m(6) XOR D(7); q_m(8) = 1;` [download]	[reply] [d/l]
Re^10: encoding hdmi video byte by BrowserUk (Patriarch) on Feb 25, 2010 at 10:37 UTC
For the record. Once the translation tables are correct, whatever the proper encoding is, tr/// will process 1GB in 17 seconds, including the read and write time, Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "I'd rather go naked than blow up my ass"	[reply]
Re^10: encoding hdmi video byte by hdmiguru (Initiate) on Feb 25, 2010 at 17:13 UTC
Yes what you have said is correct. The encoding starts with taking the bit 0 (D[0]) of input data and that becomes the bit 0 of result (Q[0]). Then XOR the bit 0 of the result with bit 1 of input data. That becomes bit 1 of the result (Q1). Then XOR the bit 1 of the result with the bit 2 of input data. That becomes bit 2 of the result (Q2). Then XOR the bit 2 of the result with bit 3 of input data. That becomes bit 3 of the result (Q3). Then XOR the bit 3 of the result with the bit 4 of input data. That becomes bit 4 of the result (Q4). Then XOR the bit 4 of the result with bit 5 of input data. That becomes bit 5 of the result (Q5). Then XOR the bit 5 of the result with the bit 6 of input data. That becomes bit 6 of the result (Q6). Then XOR the bit 6 of the result with bit 7 of input data. That becomes bit 7 of the result (Q7). Th bit 8 of the result is always 1 (which indicates that we used XOR to the receiver). example input data 1010 0101 output data 1 0110 0011	[reply]