Incomplete Output When Printing Hash

FAX has asked for the wisdom of the Perl Monks concerning the following question:

Dear Gentlemen and Ladies ... first, allow me to say that you are the "go to" resource for Perl information and I greatly appreciate this site. Also, this is my first question and, although I've read the guidelines, should I make a mistake in presenting my quandary, please don't interpret that as disrespect or an attempt to irritate you.

Sadly, my question is so simple, I hesitate to even ask. However, I cannot seem to find a solution or answer having Googled my brains out and, rather than turn to alcoholism, I thought I'd ask.

Here's the problem ... I'm printing a hash. Real simple. Done it a thousand times. The hash is generated from a SQL query (also done a thousand times). The query results in 63 hash elements ...

use strict;
use warnings;
my $count = keys %myobjtdets; - hash created from query
print "$count"; - results in the number 63
exit;
[download]

Is that number correct? I counted the entries in the SQL table. It's 63. However, when I trawl through the hash, and print the entire result, I can only see about 19 or so entries. It's as though the list is truncated for some reason. It's insane. The same effect on Firefox 12.0 and various releases of IE, so the problem appears to be browser unrelated.

print "<html><meta http-equiv='content-type' content='text/html; chars
+et=UTF-8'><BR/><BR/>";
print "<BR/>FtchObjtDets<BR/><BR/>";
foreach my $mykey (sort keys %myobjtdets) { print "$mykey  - $myobjtde
+ts{$mykey}<BR/>"; }
print "</html>";
exit;
[download]

When this failed after numerous attempts, I noticed that some of you like the Data::Dumper::Simple module. Thinking that, perhaps, the issue is simply some odd inability of the browser to accept my entire list, I loaded that module and used it. Same result ... a truncated list.

What's in the data, you might ask? Nothing strange. Letters and numbers ... what many call alphanumeric characters. There's nothing in the hash list that, by and of itself, might cause the list to break. In case I missed something, however, I changed the data to basic numbers 1 .. 63 and recreated the same problem. What, pray tell, am I doing wrong? I do not believe I am a moron. However, I'm willing to confess that I am and stamp a red "M" on my forehead should you prove it.

Please help, I beg you for I am rapidly losing what remains of my mind.

Best, FAX

Comment on Incomplete Output When Printing Hash Select or Download Code

Replies are listed 'Best First'.
Re: Incomplete Output When Printing Hash by McA (Priest) on Aug 18, 2013 at 18:53 UTC
The story sounsd wierd, but try this snippet to get a hint on the output: `use Data::Dumper; print "<html><meta http-equiv='content-type' content='text/html; chars +et=UTF-8'><BR/><BR/>"; print "<BR/>FtchObjtDets<BR/><BR/>"; print "<pre>\n"; print Dumper(\%myobjtdets), "\n"; print "</pre>\n"; print "</html>"; exit;` [download] And show us the output. Update: I don't know, what you meant with "print". Is the only output to a browser via your snippet? You should look at the Dumper output on the console. What you don't do: You do not encode HTML entities. When the elements of your hash contain HTML special characters like "<", you're generating HTML tags. McA	[reply] [d/l]
Re^2: Incomplete Output When Printing Hash by FAX (Novice) on Aug 18, 2013 at 20:33 UTC
Tested your code. Unfortunately, it appears to have a problem since, at form submission, I am prompted with a download option (typical when there's some syntax issue). I'll tinker with it a bit and try to find the problem. (Maybe I copy/pasted something wrong and broke a line). FAX Okay ... modified your code ... here's the version that worked on Firefox 12.0 ... `use Data::Dumper; print "<html><meta http-equiv='content-type' content='text/html; chars +et=UTF-8'><BR/><BR/>"; print "<BR/>FtchObjtDets<BR/><BR/>"; print "<pre><BR/>"; print Dumper(\%myobjtdets); print "<BR/></pre><BR/>"; print "</html>"; exit;` [download] Interesting that Firefox didn't like the newline syntax. Odd, that. As for the output, Here 'tis. Again, a truncated list. Should I start drinking now? `$VAR1 = { 'prilabst' => 0, 'firstname' => 'Test', 'OBJECT.listmenuid' => '1650', 'odetnone' => 0, 'useremail' => 'test@test.com', 'clntobid' => '2', 'objtocat' => 'User', 'directory' => 'testuser', 'objtotyp' => 'Users', 'PROCESS.javaloc.Pages1' => 'Pages1', 'OBJECT.directory' => 'testuser', 'email' => 'test@test.com', 'usroidst' => 1, 'password' => 'test', 'clientid' => '2', 'PROCESS.javaloc.Menus1' => 'Menus1',` [download] It's the little things that drive you crazy. Still, thank you so very much for your interest and assistance. FAX	[reply] [d/l] [select]
Re^3: Incomplete Output When Printing Hash by Monk::Thomas (Friar) on Aug 19, 2013 at 07:39 UTC
To me this really looks like there's some value in the keys/values which keeps perl from outputting the whole page. A) You could try to print only the keys and see which one comes after 'PROCESS.javaloc.Menus1'. This ~should~ print all 63 keys. Then you can change 'sort keys' into 'reverse sort keys' and if there's only one malformed key/value then you should end up with 46 lines of output. If there are less then that, then there are more then one bogus keys/values. B) Can perl write files to the filesystem? Normally the answer is yes, but you seem to be working on a webserver and who knows how it's configured. (Maybe someone really nailed it down and prevents write access to the filesystem via a SELINUX policy...) If you can write, then you could print your results to a file and check the contents of that file. At least you don't have to worry about proper HTML entity escaping, browser bugs and possible buffering issues. C) Do you have access to the webserver error log? If not: Get familiar with eval and catching exceptions and try to catch the error message, why perl terminates abnormally. Hint: Try::Tiny is really nice. The documentation also explains what can go wrong with eval. But pay attention to the trailing semicolon 'try {} catch {};' <--- Strange stuff happens if you ommit it.	[reply]
Re^2: Incomplete Output When Printing Hash by FAX (Novice) on Aug 18, 2013 at 20:27 UTC
I can't thank you enough for your help on this insane situation. I am printing to the browser. The reason for this is that the SQL query and resulting hash is initiated by an online form submission (a log-in page, actually). The page has basic, simple input and I've used it without any issue. Anyhow, since I'm testing by submitting the form, it seems an easy matter to merely print the hash list to the browser window I'm already looking at. To be perfectly candid, that's how I've always reviewed things like this and have never had a problem in the past. It's what makes this particular deal so incredibly irritating. It's a very simple request ... print the hash to a browser page. Nevertheless, if I would be better off printing to the console, I must confess that, since I've never actually done that, I don't know how to do it. In the real world, this hash list must be generated by the form submission. That's why I test it the same way. I take it that you recommend I research means and methods for printing to the console, however? Thanks again (to all), FAX	[reply]
Re: Incomplete Output When Printing Hash by 2teez (Vicar) on Aug 18, 2013 at 20:27 UTC
Hi FAX, Can you try printing out the hash keys and it's corresponding values, without the HTML tags and see what you have first, like so: `use Data::Dumper; ... print Dumper \%myobjtdets;` [download] Then if all your keys and values are printed without truncation of your data, then you know where the issue is. If not, then let's take a look into your dataset. If you tell me, I'll forget. If you show me, I'll remember. if you involve me, I'll understand. --- Author unknown to me	[reply] [d/l]
Re^2: Incomplete Output When Printing Hash by FAX (Novice) on Aug 18, 2013 at 21:12 UTC
Used your code as written (sans ellipse)and generated a partial, truncated list again as follows; $VAR1 = { 'prilabst' => 0, 'firstname' => 'Test', 'OBJECT.listmenuid' => '1650', 'odetnone' => 0, 'useremail' => 'test@test.com', 'clntobid' => '2', 'objtocat' => 'User', 'directory' => 'testuser', 'objtotyp' => 'Users', 'PROCESS.javaloc.Pages1' => 'Pages1', 'OBJECT.directory' => 'testuser', 'email' => 'test@test.com', 'usroidst' => 1, 'password' => 'test', 'clientid' => '2', 'PROCESS.javaloc.Menus1' => 'Menus1', 'objecttype' => 'Users', 'OBJECT.useremail' => 'test@test.com', 'OBJECT.savemenuid' => '381', [download] It's beginning to feel as though there is something else wrong that neither strict nor warnings is telling me about. This is far too simple a command to fail without good reason ... or so it would seem. Thanks for your help on this, though. I greatly appreciate it. FAX	[reply] [d/l]
Re^3: Incomplete Output When Printing Hash by AnomalousMonk (Archbishop) on Aug 19, 2013 at 01:20 UTC
I'm still confused about what you're doing. Do you say that you can run code like `use Data::Dumper; my %hash = initialize_a_hash_somehow(); print 'key/value pairs: ', scalar keys %hash; print Dumper \%hash;` [download] ... and see printed to the console the number 63 for key/value pairs and a listing of 19 key/values from `Dumper`? (Also: where's the closing curly bracket for the `Dumper` output?) If this is the case, something is seriously b0rken in your system! It seems to me that the first step is to get consistent data printed to the console and then figure out how to print it to a browser.	[reply] [d/l] [select]
Re: Incomplete Output When Printing Hash by RichardK (Parson) on Aug 18, 2013 at 23:22 UTC
What a fun problem ;) to me, the only things that stand out are Firefox 12 is really quite old, the current version is 23, but I'm not sure that will make much difference you're missing your body tag, html really should look something like this:- `<html> <head> </head> <body> content here </body> </html>` [download] browsers do try to guess what you meant, and usually do the right thing -- but in this case who knows? I usually use CGI to generate this type of html, I find it much easier.	[reply] [d/l]
Re^2: Incomplete Output When Printing Hash by Anonymous Monk on Aug 19, 2013 at 11:47 UTC
The first thing to do (and something a lot of people fail to do) when debugging CGI is to get rid of HTML: `print "Content-Type: text/plain\r\n\r\n";` [download]	[reply] [d/l]
Re: Incomplete Output When Printing Hash by Anonymous Monk on Aug 19, 2013 at 03:07 UTC
https://metacpan.org/module/HTML::Entities#encode_entities-string	[reply]
Re: Incomplete Output When Printing Hash by CountZero (Bishop) on Aug 19, 2013 at 15:24 UTC
Are you entirely sure that all the entries in your SQL table are unique? If not, putting those not unique entries into a hash will only keep one entry per "unique" value and hence a reduction of the total number of keys. CountZero A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James My blog: Imperial Deltronics	[reply]
Re^2: Incomplete Output When Printing Hash by Monk::Thomas (Friar) on Aug 20, 2013 at 08:35 UTC
We can rule out uniqueness related problems due to: `my $count = keys %myobjtdets; - hash created from query print "$count"; - results in the number 63` [download]	[reply] [d/l]
Re: Incomplete Output When Printing Hash by Laurent_R (Canon) on Aug 18, 2013 at 19:00 UTC
I have no idea if this is going to be of any help, but, quite possibly, this is going to give you a more reasonable count on your data: `my $count = values %myobjtdets;`	[reply] [d/l]
Re^2: Incomplete Output When Printing Hash by AnomalousMonk (Archbishop) on Aug 18, 2013 at 19:49 UTC
... quite possibly, this is going to give you a more reasonable count on your data: `my $count = values %myobjtdets;` I don't understand. How could `my $count = values %myobjtdets;` produce anything at all different from `my $count = keys %myobjtdets;` for the `$count` scalar?	[reply] [d/l] [select]
Re^3: Incomplete Output When Printing Hash by Laurent_R (Canon) on Aug 18, 2013 at 21:28 UTC
Thank you for your comment, AnomalousMonk. I thought that something like: `print scalar values %hash;` would return a count on only the defined values. After a quick try, this is obviously not the case. My error. This leads me to another possible solution illustrated in the following session under the Perl debugger: `DB<1> %hash = (jan => 1, feb => 2, mar => undef, apr => undef, may = +> 5); DB<2> x %hash 0 'feb' 1 2 2 'jan' 3 1 4 'may' 5 5 6 'mar' 7 undef 8 'apr' 9 undef DB<3> print scalar grep {defined $_} values %hash; 3 DB<4> print scalar grep {defined $_} keys %hash; 5` [download]	[reply] [d/l] [select]