larsen has asked for the wisdom of the Perl Monks concerning the following question:

I have strange problems inserting data in a MySQL (3.23.36) table. In a apparently casual way, my data are truncated. Here the code I'm using:

sub insert_msg { my $author = $query->param('author'); my $subject = $query->param('subject'); my $body = $query->param('body'); my $parent = $query->param('parent'); my $thread; if ( $parent ) { my $sth = $dbh->prepare(qq|select id, thread from messaggi whe +re id=?|); $sth->execute( $parent ) || die $DBI::errstr; my $hr = $sth->fetchrow_hashref(); $thread = ($hr->{'thread'} == 0) ? $hr->{'id'} : $hr->{'thread +'}; } else { $thread = 0; } $body = validate( $body ); $body = HTML::Entities::encode( $body ); $body =~ s/\[(.*?)\|(.*?)\]/target($1, $2)/eg; $body =~ s/\[(.*?)\]/target($1)/eg; my $sth = $dbh->prepare( qq|insert into messaggi values(null, ?, n +ull, ?, ?, ?, null)| ); $sth->execute( $thread, $author, $subject, $body ) || die $DBI::er +rstr; $template->process( $config->{'template'}->{'insert'} ); }

Here the table's description:

CREATE TABLE `messaggi` ( `id` int(10) unsigned NOT NULL auto_increment, `thread` int(10) unsigned default NULL, `data` timestamp(14) NOT NULL, `author` varchar(100) default NULL, `subject` varchar(200) default NULL, `body` text, `category` int(10) unsigned default NULL, PRIMARY KEY (`id`), FULLTEXT KEY `body` (`body`), ) TYPE=MyISAM

As you can guess, it's a mysql-based webforum. Very simple.
First of all, I fetch information about the thread where the message has to be inserted. Then, I validate the $body using HTML::Parser, stripping HTML tags that are not valid. Then, I replace links with appropriate HTML, like in PerlMonks. Eventually, I insert the message in the database.

Sometime the $body is truncated, casually, apparently. For example, inserting the same message after some minute, the body is inserted correctly.

Thank you for every advice.

Replies are listed 'Best First'.
Re: Truncated data using MySQL and DBI
by jryan (Vicar) on Sep 04, 2001 at 18:17 UTC
    You can always consider switching body from text to longtext. I have a similar system at work, and with longtext, it has never given me a trunacation problem.
      As far as I know, TEXT fields can contain up to 216, that is something like 65k, but I have the problem described even with messages of two lines.
(ichimunki) Re: Truncated data using MySQL and DBI
by ichimunki (Priest) on Sep 04, 2001 at 19:20 UTC
    I am assuming you've validated that $body is being correctly assigned from the param() call, and is still the proper length after validate()? Have you tried this as a $dbh->do() command with similar problems?
      ...err. I was too much self-confident about my code. The problem seems to be in validate() :) So, may I ask your fresh eyes to examine my code? :)

      sub validate { my $text = shift; my $new_text = ''; my %is_approved = (); my @approved_tags = ( 'a', 'p', 'br', 'b', 'i', 'strong', 'tt', 'small', 'strike', 's', 'u', 'sub', 'sup', 'blockquote', 'table', 'th', 'tr', 'td', 'caption', 'thead', 'tfoot', 'tbody', 'colgroup', 'dl', 'dt', 'dd', 'ul', 'li', 'ol', ); foreach( @approved_tags ) { $is_approved{ $_ } = 1 } my $validator = HTML::Parser->new( start_h => [sub { my $self = shift; my ($tagname, $attr, $origtext) = @_; $new_text .= $origtext if ($is_approved{ $tagname }); }, "self, tagname, attr, text"], end_h => [sub { my $self = shift; my ($tagname) = @_; $new_text .= "</$tagname>" if ($is_approved{ $tagname }); + }, "self, tagname"], text_h => [sub { my $self = shift; my ($origtext) = @_; $new_text .= $origtext; }, "self, text"] ); $validator->parse( $text ); return $new_text; }