cormanaz has asked for the wisdom of the Perl Monks concerning the following question:

Good day Monks. I have some accelerometer data that is noisy, so I would like to smooth it using spline fitting. I have been trying to do this using Python scipy.interpolate and it is driving me ABSOLUTELY BONKERS with type conflicts.

So I thought I would try my go-to language: beautiful, typeless Perl. Alas, I'm not smart enough to figure out how to use Math::Spline for this purpose, and strangely after some searching around can't find any examples.

Some toy data is:

use strict; use feature ':5.10'; my @accel = [[-0.7437,0.1118,-0.5367], [-0.5471,0.0062,-0.6338], [-0.7437,0.1216,-0.5255], [-0.4437,0.3216,-0.3255], ];
What I'd like to do is turn this into, say, 200 rows with interpolated values. Can anyone point me in the right direction?

Replies are listed 'Best First'.
Re: How to smooth values of an x,y,z array using Math::Spline
by hv (Prior) on May 10, 2023 at 17:09 UTC

    Did you try following the approach shown in the synopsis or example sections of the Math::Spline docs? What happened when you tried it?

    I'll take a wild guess that the triples in your "toy data" represent 3D (x, y, z) values, at regular time intervals (t0, t1, t2, t3). I'm not sure what results you're hoping for, but my guess is extrapolated 3D positions at interpolated times, in which case I expect you're going to need to instantiate three Math::Spline evaluators, one each for (t -> x), (t -> y) and (t -> z).

      Great idea, hv.

      Updating my code to implement three splines relative to t rather than two splines relative to x, there isn't a divide-by-zero problem:

      use warnings; use strict; use feature ':5.10'; my @accel = ([-0.7437,0.1118,-0.5367], [-0.5471,0.0062,-0.6338], [-0.6437,0.1216,-0.5255], [-0.4437,0.3216,-0.3255], ); # note: I changed from an array whose only element is an arrayref +of arrayrefs to an array who has n arrayrefs, to be a simple 2D array use Math::Spline; my (@x,@y,@z,@t); # generate x, y, and z arrays, and a t array my $pt = 0; for my $xyz (@accel) { push @t, $pt++; push @x, $xyz->[0]; push @y, $xyz->[1]; push @z, $xyz->[2]; } # create spline calculators for x&y and x&z my $spline_tx = eval { Math::Spline::->new(\@t, \@x) } or do { die "ty +: $@" }; my $spline_ty = eval { Math::Spline::->new(\@t, \@y) } or do { die "ty +: $@" }; my $spline_tz = eval { Math::Spline::->new(\@t, \@z) } or do { die "tz +: $@" }; my @interp_accel = (); my $NSTEPS = 200; my $dt = $pt/$NSTEPS; # $NSTEPS+1 values from xmin to xmax, inclusive for my $i (0..$NSTEPS) { my $t = $i * $dt; my $x = $spline_tx->evaluate($t); my $y = $spline_ty->evaluate($t); my $z = $spline_tz->evaluate($t); push @interp_accel, [$x,$y,$z]; # store for later printf "interpolate # %d => t=%.2f => [%s,%s,%s]\n", $i, map {$_// +'<undef>'} $t, $x, $y, $z; # debug print }

      I then get the four fixed points at t=0, t=1, t=2, and t=3, with interpolated values everywhere else (I snipped intermediate values, except near the fixed points)

      interpolate # 0 => t=0.00 => [-0.7437,0.1118,-0.5367] interpolate # 1 => t=0.02 => [-0.73780958368,0.10862255968,-0.53961481 +072] interpolate # 2 => t=0.04 => [-0.73192386944,0.10544767744,-0.54252728 +576] ... interpolate # 48 => t=0.96 => [-0.54759113856,0.00641293056,-0.6335783 +4624] interpolate # 49 => t=0.98 => [-0.54723036832,0.00624379232,-0.6337463 +9728] interpolate # 50 => t=1.00 => [-0.5471,0.0062,-0.6338] interpolate # 51 => t=1.02 => [-0.5472023792,0.0062833216,-0.633737511 +2] interpolate # 52 => t=1.04 => [-0.5475304256,0.00649236480000001,-0.63 +35600576] ... interpolate # 98 => t=1.96 => [-0.6433479104,0.1142666112,-0.532585126 +4] interpolate # 99 => t=1.98 => [-0.6436376048,0.1179228224,-0.529056384 +8] interpolate # 100 => t=2.00 => [-0.6437,0.1216,-0.5255] interpolate # 101 => t=2.02 => [-0.64352802112,0.12529626272,-0.521917 +57408] interpolate # 102 => t=2.04 => [-0.64312404096,0.12901093376,-0.518309 +84064] ... interpolate # 148 => t=2.96 => [-0.45563928704,0.31328743424,-0.333929 +71136] interpolate # 149 => t=2.98 => [-0.44967201088,0.31744352928,-0.329715 +11392] interpolate # 150 => t=3.00 => [-0.4437,0.3216,-0.3255] interpolate # 151 => t=3.02 => [-0.43772798912,0.32575647072,-0.321284 +88608] interpolate # 152 => t=3.04 => [-0.43176071296,0.32991256576,-0.317070 +28864] ... interpolate # 198 => t=3.96 => [-0.24427595904,0.51418906624,-0.132690 +15936] interpolate # 199 => t=3.98 => [-0.24387197888,0.51790373728,-0.129082 +42592] interpolate # 200 => t=4.00 => [-0.2437,0.5216,-0.1255]
      Gah...sorry! I thought I had defined the array. Yes, it's accelerator x,y,z values (columns) for four observations (rows).
Re: How to smooth values of an x,y,z array using Math::Spline
by pryrt (Abbot) on May 10, 2023 at 17:12 UTC
    I assume the rows of @accel are x,y,z values. Also, @accel=[,[],[]] creates a 3deep array: the outer array has a single element, which is an arrayref; inside that arrayref are four arrayrefs with three elements each. To simplify that structure, I changed the outer brackets [] to parentheses () so it's a 2deep array instead of 3deep array, so each element of @accel is an arrayref of three values, which is what I think you wanted.

    When I tried running

    use warnings; use strict; use feature ':5.10'; my @accel = ([-0.7437,0.1118,-0.5367], [-0.5471,0.0062,-0.6338], [-0.7437,0.1216,-0.5255], [-0.4437,0.3216,-0.3255], ); # note: I changed from an array whose only element is an arrayref +of arrayrefs to an array who has n arrayrefs, to be a simple 2D array use Math::Spline; my (@x,@y,@z); my ($xmin, $xmax) = (9.99e99,-9.99e99); # for tracking min and max x v +alues # generate x, y, and z arrays for my $xyz (@accel) { push @x, $xyz->[0]; push @y, $xyz->[1]; push @z, $xyz->[2]; if($x[-1] < $xmin) { $xmin = $x[-1]; } if($x[-1] > $xmax) { $xmax = $x[-1]; } } # create spline calculators for x&y and x&z my $spline_xy = eval { Math::Spline::->new(\@x, \@y) } or do { die "xy +: $@" }; my $spline_xz = eval { Math::Spline::->new(\@x, \@z) } or do { die "yz +: $@" }; my @interp_accel = (); my $NSTEPS = 200; my $dx = ($xmax-$xmin)/$NSTEPS; # $NSTEPS+1 values from xmin to xmax, +inclusive for my $i (0..$NSTEPS) { my $x = $xmin + $dx * $i; my $y = defined($spline_xy) ? $spline_xy->evaluate($x) : undef; my $z = defined($spline_xz) ? $spline_xz->evaluate($x) : undef; push @interp_accel, [$x,$y,$z]; # store for later printf "interpolate # %d => [%s,%s,%s]\n", $i, map {$_//'<undef>'} + $x, $y, $z; # debug print }

    ... I found that $spline_xy=Math::Spline::->new(...) and $spline_xz both die when they are created; if I change the third row to having an x value of -0.6437 instead of -0.7437, it fixes it, so I am assuming that it's because those x-values are the same.

    Were you intending the middle two of the four points to be points on the curve (so the interpolation would hit them exactly); or are they "control points", where the interpolation doesn't hit them directly, but instead only touches on the outer two, and the inner two just define directions (like this image from Bézier_curve)? Because, from what I can tell, Math::Spline makes sure that it hits the points from the instantiation-list rather than treats them as controls.

    If what you really wanted was more along the lines of a Bézier curve, let me know, because that's pretty simple to code up without a module.

      Yes, I think a Bezier curve would be fine. Mainly just trying to smooth-out some noise. I'd appreciate you sharing that code.
        Given what you said Re^2: How to smooth values of an x,y,z array using Math::Spline, I think my second answer Re^2: How to smooth values of an x,y,z array using Math::Spline is your best bet. That answer allows for any number of observations to be smoothed. My original thought for the Bezier would have been to only pay attention to four points, only hitting the outer two; and if I expanded that idea to multiple "fixed points", it would require that the number of observations be 3N+1, and it would "waste" (go toward but not hit) roughly 2/3 of the points.

        But, if you really want the Bezier, and thus not hitting most of the points you list:

        Whether this or Re^2: How to smooth values of an x,y,z array using Math::Spline is better for "smoothing" the data that you have is really up to you.

        addenda: Please note that in all of my solutions, @interp_accel will contain the complete list of interpolated values, even if my manual editing or in-code logic doesn't print them all. I just skipped most of the printing to save space in the posts, so the reduced output was enough to show what was going on.