How are you measuring "results are not that great"

Not very scientifically, I've just looked at the largest difference in the coordinates.

I've also plotted the results: https://imgur.com/a/W7j3yC9

what are the results against the two test cases you have posted

target_41:

        339        508  345.92214  512.55747
        314        518  318.48381   516.9291
        265        533  274.68006  530.37545
        245        537  248.82152  537.19123
        228        544  227.20604  544.26418
[download]

target_996:

        207        550  213.15752  551.70605
        206        569  209.79236  571.16027
        205        602  208.69494  603.29318
        203        621  207.30388  622.07696
        201        638  207.08208  637.97918
[download]

Perhaps "not that great" was not that great a description; this algorithm in fact produces comparable results to those in other replies in the thread.

Note that I've messed up the direction of the transformation in my reply, the lines at the end should be

my ($px, $py) = fitwarp2d($red_out_x, $red_out_y, $red_in_x, $red_in_y
+, 2);

my ($blue_new_x, $blue_new_y) = applywarp2d($px, $py, $blue_in_x, $blu
+e_in_y);

#say for $blue_out_x, $blue_out_y, $blue_new_x, $blue_new_y, $blue_out
+_x - $blue_new_x, $blue_out_y - $blue_new_y;
[download]