If you don't want to change your main code just write a simple script to pre-process the data and concatenate the second page onto the first. That way your data will be as you expected when you wrote the script. Quick and dirty but probably the easiest fix.

All you need to do is work out the point(s) of difference between a single page table and a multipage one. For example it seems from what you present that the data fields always match m/^\|\s+[a-z]/ ie a | spaces then a lower case letter code (the headers, blank lines etc don't) so you could do:

perl -ne 'print if m/^\|\s+[a-z]/' infile > outfile
[download]

This will put all the data rows into the outfile (no headers or anything else). Mod the main script so it does not expect to skip headers.....