Turns out when I said "now we are starting to collect data", I spoke too soon. There are bugs in the software that drive the logger and, after months of back and forth between us and the manufacturer, we haven't been able to get the system to work correctly.

The data consists of measurements in two axis from an induction magnetometer at 64Hz. The analysis looks for signals at fairly low frequency (7Hz maybe) that are related to some ionospheric phenomena I've forgotten about. Data files are a day's worth of data can come out to around 21MB. My test script looks at the first 16 seconds (1024 points) and plots a simple frequency/amplitude graph just to check that the data looks sane (basically that it has wiggly bits). The data below is fairly typical and shows a spike just below 8Hz - the interesting bit to the scientists. Absolute amplitude isn't very interesting for a quick assessment of the system.

Now, the version that still uses Math::FFT, but PDL::Graphics::Simple only for visualisation (note the lack of needing to scale etc) - you need PDL, PDL::Graphics::Simple, and one underlying plotter, I recommend PDL::Graphics::Gnuplot:

use strict;
use warnings;
use Math::FFT;
use PDL;
use PDL::Graphics::Simple;

my $filePath = $ARGV[0] // "testData.txt";

my @data = do { open my $fh, $filePath or die "Can't open $filePath: $
+!"; local $/; split /\s+/, <$fh> };

my $fft = Math::FFT->new(\@data);
my $spectrum = $fft->spctrm;
shift @$spectrum; # Remove DC signal component

line pdl($spectrum);
print "ret> "; <STDIN>;
[download]

Even if you prefer other methods of calculation, I hope you agree that the quick visualisations available with P:G:S are a game-changer. (The last line is just so the plot window doesn't get immediately closed because your program terminated - that's not a problem if you use a REPL like perldl)

And finally (cue human-interest story), the version using PDL::FFTW3; unfortunately for me I had to learn about power spectra and then convert Math::FFT's spctrm code into a small PDL function:

use strict;
use warnings;
use PDL;
use PDL::FFTW3;
use PDL::Graphics::Simple;
my $filePath = $ARGV[0] // "testData.txt";
my @data = do { open my $fh, $filePath or die "Can't open $filePath: $
+!"; local $/; split /\s+/, <$fh> };

sub spectrum {my ($d)=@_;
  my $hn=$d->dim(0)-1;
  my $a2=$d->abs2/(($hn*2)**2);
  $a2->slice([1,$hn-1])*=2;
  $a2;
}
my $spectrum = spectrum(rNfft1 pdl(\@data));
$spectrum = $spectrum->slice('1:-1'); # Remove DC signal component

line pdl($spectrum);
print "ret> "; <STDIN>;
[download]

If you're into IRC, then next time you feel you might benefit from using PDL but don't feel 100% clear on where to start, do feel free to join #pdl on irc.perl.org and ask questions! A lightweight way to do so is on the MetaCPAN page for various PDL modules, with the "Chat with us" link.

What? Software/hardware with bugs??

Once I'd emotionally recovered from dealing with that idea, I simplified your code into this (with your data above in a file you can give as an argument, or in the obvious default filename):

use strict;
use warnings;
use Tk;
use Tk::Canvas;
use Math::FFT;
my $filePath = $ARGV[0] // "testData.txt";
my @data = do { open my $fh, $filePath or die "Can't open $filePath: $
+!"; local $/; split /\s+/, <$fh> };
my $spectrum;
my $fft = Math::FFT->new(\@data);
$spectrum = $fft->spctrm;
shift @$spectrum; # Remove DC signal component
my $mw = MainWindow->new (-title => "Magnetometer Plotter");
my $canvas = $mw->Canvas (-height => 700, -width => 1024)->pack ();
my $i = 0;
$canvas->createLine(
  (map +(2 + 2 * $i++, $_), NormData($spectrum, 680)), -fill => 'blue'
);
$mw->MainLoop;
sub NormData {
    my ($data, $span) = @_;
    my ($min, $max);
    for my $datum (@$data) {
        $min //= $datum;
        $max //= $datum;
        $min = $datum if $min > $datum;
        $max = $datum if $max < $datum;
    }
    my $scale = $span / ($max - $min);
    map $span - ($_ - $min) * $scale + 10, @$data;
}
[download]

There are bugs in the software that drive the logger and, after months of back and forth between us and the manufacturer, we haven't been able to get the system to work correctly.

You wouldn't happen to have more details on that sensor setup? Maybe a datasheet, interface description, or anything of that sort? Maybe i could help to come with an alternative solution of getting the data out of the sensor.

PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
Also check out my sisters artwork and my weekly webcomics

The sensor is just a solenoid about 2m long with, I'd guess, tens of thousands of turns. The electronics is a very sensitive DC coupled amplifier with extremely low low frequency (especially 1/f) noise followed by an ADC. The system is looking at low frequency changes in the local Earth's magnetic field due to changes in current flowing in the ionosphere which is influenced by the solar wind.

A complication for our situation is that everything is running off solar power and there is no network connection available (cellular, POTS or fiber). The main issue we have with the logger is that it doesn't reliably recover from low power situations. It doesn't help that the system is powered over a 60m cable which generates about 1V drop between the 14V boosted supply and the logger. Things were much worse before I put a voltage boost system in at the supply end.

Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond