in reply to WWW::Robot hangs

Did you check your database log files to see if anything strange is happening? You do a INSERT INTO index ... in sub invoke_on_contents {...}, but you never do an explicit commit to the database. I guess this is because you rely on the fact the you have DBI AutoCommit on. In fact AFAIK AutoCommit should default to on. Anyway DBI man page says "Explicitly defining the required AutoCommit behavior is strongly recommended and may become mandatory in a later version.". My guess is that AutoCommit on may not be working properly for you, and after a certain number of INSERT (that are not automatically commited) you may end up with some condition of database rollback segment full, and this may causes DBI:ODBC to crash (OK, I don't know why you don't get back any error).
I already had some problem with AutoCommit, so I usually explicitly set AutoCommit to off and then perform an explicit commit when I want.
I hope this helps.

marcos

Replies are listed 'Best First'.
Re: Re: WWW::Robot hangs
by smitz (Chaplain) on Jun 19, 2002 at 08:22 UTC
    Ill be honest, most of the DBI instructions were added on for debugging purposes, and I'm not sure I know what some of them do, particularly AutoCommit.
    I'm using this on a Win2000 Box with MS Access, how do I check the server log?
    Dont tell me, its not possible with Access... :-(
    Further, how do I do a hard commit? And should I be doing this after each INSERT or just occasionaly?

    Thanks,
    SMiTZ
[reply]
[d/l]
[select]
      Unfortunately I can give you no help with Access, and I don't know if Access support commit and rollback ... sorry, Don't you have any other DB to test you script with?. Anyway, For the other questions:
      AutoCommit is a database handler attribute: from the DBI man page: "If true, then database changes cannot be rolled-back (undone). If false, then database changes automatically occur within a "transaction", which must either be committed or rolled back using the commit or rollback methods.": you can read further details in DBI man page. You can set AutoCommit in the connect statement in the same way as you set RaiseError, for example: 
      my $dbh = DBI->connect(
  "DBI:ODBC:$databaseName",
  {
    AutoCommit => 0,
    RaiseError => 1
  }
)
  or die "cannot connect to Oracle!";

      [download]

      You can perform an explicit commit on the database simply saying: 
      $dbh->commit;  # or call $dbh->rollback; to undo changes

      [download]
      This makes sense only when you turn AutoCommit off.
      You can do a commit after every INSERT to see if it works, but this may be time consuming, so you may decide to commit every 10 or 50 INSERT, or whatever is suitable for your application.

      marcos
[reply]
[d/l]
[select]
      forget MS Access and get a real DB. installation is a breeze, usage is quite friendly with a frontend.

      I tried writing NT services using Access/ODBC (even ADO) but they all dies or locks (without further notice) after 1-2 days. with MySql and DBI I have months of uptime and not a problem.

      cheers,
      Aldo 

      __END__
$_=q,just perl,,s, , another ,,s,$, hacker,,print;

      [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom