I have merrily allocated structures without a care to deallocate the space
If you allocate any memory using malloc then perl will not deallocate it for you, you have to free the structures yourself.

In the case of a string, why do I not get a reference to a perl scalar
Perl references are not the same as C pointers. newSVpv creates a perl scalar with the value of a string and returns a pointer to it, not a reference. To create a reference use newRV or newRV_inc.

Although Data::Dumper is very useful, for looking at the details of variables created in this way Devel::Peek is invaluable, for example:

use Devel::Peek;

print mkstr() . "\n";
my $str = mkstr();
print Dump($str),"\n\n";


my $arr = mkarr();
print ref($arr) . "\n";
print Dump($arr) . "\n";

__END__
__C__
... your code ...
[download]

Gives (on Linux 5.12.0):

This is another string!
SV = PV(0x8e2f128) at 0x8cebf60
  REFCNT = 1
  FLAGS = (PADMY,POK,pPOK)
  PV = 0x8dade30 "This is another string!"\0
  CUR = 23
  LEN = 24


ARRAY
SV = IV(0x8cebfcc) at 0x8cebfd0
  REFCNT = 1
  FLAGS = (PADMY,ROK)
  RV = 0x8e51cb0
  SV = PVAV(0x8d55470) at 0x8e51cb0
    REFCNT = 2
    FLAGS = ()
    ARRAY = 0x8e40b10
    FILL = 9
    MAX = 9
    ARYLEN = 0x0
    FLAGS = (REAL)
    Elt No. 0
    SV = IV(0x8e1d3f4) at 0x8e1d3f8
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = 0
    Elt No. 1
    SV = IV(0x8e1d4a4) at 0x8e1d4a8
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = 1
    Elt No. 2
    SV = IV(0x8e1d3a4) at 0x8e1d3a8
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = 2
    Elt No. 3
    SV = IV(0x8d9971c) at 0x8d99720
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = 3
Use of uninitialized value in concatenation (.) or string at ./gash.pl
+ line 18.
[download]

This clearly shows that your mkstr() function returns a scalar value as a string, and it looks like there is a problem with the memory allocation in mk_arr().

Personally I do not create Perl arrays in my XS code, I return lists instead, and then the caller can stuff the list where it wants (I don't think av_make() is a common way of creating a Perl array). Typically I would loop creating the list elements on the stack using:

XPUSHs(sv_2mortal(newSVpvn (szString, iLen)));
[download]

Finally, I recommend "Extending and Embedding Perl" by Tim Jenness and Simon Cozens (Manning).