Casey2255 has asked for the wisdom of the Perl Monks concerning the following question:

Hi all.

I'm working with a fleet of devices running Perl on a ARM chipset (armhf) that are experiencing an infrequent, random crash in SQL::Abstract (via DBIx::Class). I've not yet been able to reproduce it manually, our only clue to go off of has been the following log statement.

panic: stack_grow() negative count (-16777216) at /usr/share/perl5/SQL/Abstract.pm line 1493

I've found similar threads of https://rt-cpan.github.io/Public/Bug/Display/108578/ and https://github.com/Perl/perl5/issues/15013. But neither of which were fruitful.

My first observation was that -16777216 is 0xFF000000 as a 32-bit signed value. This makes me think it's due to a unsigned to signed cast gone wrong. However, adding 16 MiB to the stack seems incorrect as well. To me this screams memory corruption or an uninitialized value.

I ran it with the -d flag to get the following stack trace: 

@ = SQL::Abstract::_join_sql_clauses(ref(DBIx::Class::SQLMaker), 'and'
+, ref(ARRAY), ref(ARRAY)) called from file '/usr/share/perl5/SQL/Abst
+ract.pm' line 678
@ = SQL::Abstract::_where_HASHREF(ref(DBIx::Class::SQLMaker), ref(HASH
+), undef) called from file '/usr/share/perl5/SQL/Abstract.pm' line 54
+5
@ = SQL::Abstract::_recurse_where(ref(DBIx::Class::SQLMaker), ref(HASH
+)) called from file '/usr/share/perl5/SQL/Abstract.pm' line 525
@ = SQL::Abstract::where(ref(DBIx::Class::SQLMaker), ref(HASH), ref(HA
+SH)) called from file '/usr/share/perl5/SQL/Abstract.pm' line 469
@ = SQL::Abstract::select(ref(DBIx::Class::SQLMaker), ref(ARRAY), 'me.
+id, me.name, me.value, me.modified_at', ref(HASH), ref(HASH)) called 
+from file '/usr/share/perl5/DBIx/Class/SQLMaker.pm' line 172
@ = DBIx::Class::SQLMaker::select(ref(DBIx::Class::SQLMaker), ref(ARRA
+Y), ref(ARRAY), ref(HASH), ref(HASH)) called from file '/usr/share/pe
+rl5/DBIx/Class/Storage/DBI.pm' line 1679
@ = DBIx::Class::Storage::DBI::_gen_sql_bind(ref(DBIx::Class::Storage:
+:DBI::Pg), 'select', ref(ARRAY), ref(ARRAY)) called from file '/usr/s
+hare/perl5/DBIx/Class/Storage/DBI.pm' line 1666
@ = DBIx::Class::Storage::DBI::_prep_for_execute(ref(DBIx::Class::Stor
+age::DBI::Pg), 'select', ref(ARRAY), ref(ARRAY)) called from file '/u
+sr/share/perl5/DBIx/Class/Storage/DBI.pm' line 1810
@ = DBIx::Class::Storage::DBI::_execute(ref(DBIx::Class::Storage::DBI:
+:Pg), 'select', ref(ARRAY), ref(ARRAY), ref(HASH), ref(HASH)) called 
+from file '/usr/share/perl5/DBIx/Class/Storage/DBI.pm' line 2409
@ = DBIx::Class::Storage::DBI::_select(ref(DBIx::Class::Storage::DBI::
+Pg), ref(ARRAY), ref(ARRAY), ref(HASH), ref(HASH)) called from file '
+/usr/share/perl5/DBIx/Class/Storage/DBI.pm' line 2586
@ = DBIx::Class::Storage::DBI::select_single(ref(DBIx::Class::Storage:
+:DBI::Pg), ref(ARRAY), ref(ARRAY), ref(HASH), ref(HASH)) called from 
+file '/usr/share/perl5/DBIx/Class/ResultSet.pm' line 1104

[download]

This puts us at the following function and line of the crash: https://github.com/dbsrgits/sql-abstract/blob/2972827e573b0217735b901088e69c994ba8d226/lib/SQL/Abstract.pm#L1493 

sub _join_sql_clauses {
  my ($self, $logic, $clauses_aref, $bind_aref) = @_;

  if (@$clauses_aref > 1) {
    my $join  = " " . $self->_sqlcase($logic) . " ";
    my $sql = '( ' . join($join, @$clauses_aref) . ' )';
    return ($sql, @$bind_aref);
  }
  elsif (@$clauses_aref) {
    ##### CRASH TRIGGERED ON LINE 1493 BELOW #####
    return ($clauses_aref->[0], @$bind_aref); # no parentheses
  }
  else {
    return (); # if no SQL, ignore @$bind_aref
  }
}

[download]

However, trying to add a breakpoint to stack_grow() fails. From further research this is an internal symbol, so does not have a subroutine to match.

My questions are:

Versions: Perl version: 5.32.1 DBIx::Class version: 0.082841 SQL::Abstract version: 1.87

I appreciate any and all feedback, thank you all in advance. - Casey

Replies are listed 'Best First'.
Re: Crash in stack_grow() with SQL::Abstract
by dave_the_m (Monsignor) on Jun 03, 2025 at 07:21 UTC
    Perl_stack_grow() is a C function: it's part of the perl interpreter. You would have to debug it by using a C debugger such as gdb and run the perl interpreter under it.

    It will be called each time an OP being being executed by the interpreter wants to push one or more items onto perl's argument stack. In that line of code, it likely extends it by 1 to push $clauses_aref->[0] and then extends it by the size of the @$bind_aref array. It is most likely crashing due to the latter: so either the array has got extremely large, or the internal state of the array has got corrupted in some fashion.

    What version of perl is this, and is it compiled as 32-bit or 64-bit? The commands perl -v and perl -V |grep ivsize should tell you these.

    Dave.

[reply]
[d/l]
[select]

      Tip: perl -V |grep ivsize can be replaced with perl -V:ivsize.

      The argument is an anchored regex, so you can also do stuff like perl -V:'install.*'.

[reply]
[d/l]
[select]
      Thank you for all the great info. For reference, these are the current versions we're running:
      • Perl version: 5.32.1
      • DBIx::Class version: 0.082841
      • SQL::Abstract version: 1.87
      And per the outputs of those commands:
      • Target: arm-linux-gnueabihf-thread-multi-64int
      • IVSize: 8
[reply]
        One more... Can you run a test on the target hardware that pushes an insane number of things onto the stack and see what message it crashes with?

        The reason I ask is that it's possible to produce automatic SQL that does WHERE column IN (?, ?, ?, ?, ............, ?) where it generated a placeholder for each of millions of rows that came from another resultset, so you could legitimately run out of Perl stack on that function call if you're now asking the perl stack to grow to match that array's length.

[reply]
[d/l]

Back to Seekers of Perl Wisdom