This might be faster than your inner loop:

splice @arr, $from, $to-$from+1, map 1+$_, @arr[$from..$to];
[download]

...but I haven't tested it. If we had a full runnable example, we could do some Benchmarking.