Turns out when I said "now we are starting to collect data", I spoke too soon. There are bugs in the software that drive the logger and, after months of back and forth between us and the manufacturer, we haven't been able to get the system to work correctly.

The data consists of measurements in two axis from an induction magnetometer at 64Hz. The analysis looks for signals at fairly low frequency (7Hz maybe) that are related to some ionospheric phenomena I've forgotten about. Data files are a day's worth of data can come out to around 21MB. My test script looks at the first 16 seconds (1024 points) and plots a simple frequency/amplitude graph just to check that the data looks sane (basically that it has wiggly bits). The data below is fairly typical and shows a spike just below 8Hz - the interesting bit to the scientists. Absolute amplitude isn't very interesting for a quick assessment of the system.

-1054355 -934448 -1083864 -1289846 -1362502 -1251771 -956710 -826765 -
+1036961
-1107361 -861209 -822089 -1070997 -1151255 -1022497 -1007843 -1045329 
+-965611
-1006518 -1105524 -969199 -834501 -836961 -889544 -1083336 -1161387 -1
+013110
-1032829 -1164512 -1140949 -1048894 -950036 -982239 -1167254 -1152554 
+-1052439
-1157364 -1219241 -1084558 -914330 -959233 -1212299 -1235228 -1022051 
+-1043316
-1235664 -1215237 -1076013 -1097741 -1196784 -1130571 -950403 -986063 
+-1163735
-1073039 -987492 -1225996 -1301753 -1054177 -984974 -1118071 -1046077 
+-934674
-1002840 -1032935 -1170383 -1080441 -871046 -2101892 -3012085 -1410560
+ -144178
-426156 -584073 -714243 -1039425 -1129146 -1032533 -1098582 -1225863 -
+1092377
-1158012 -1348995 -1113281 -957586 -1204903 -1693565 -1845858 -1093056
+ -625118
-1010615 -987534 -867248 -1541651 -1733959 -846189 -425518 -1062193 -1
+599191
-1093092 -512186 -1157578 -1724370 -835494 -101072 -648677 -1262616 -1
+219483
-864121 -837411 -1277211 -1363956 -926084 -789910 -1048298 -1262266 -1
+364887
-988971 -1229990 -2729175 -1789255 1095947 896226 -1005292 -1448384 -1
+241273
-1157566 -973054 -1087618 -1647032 -1592968 -1049262 -1077479 -1141678
+ -1168128
-1489736 -298666 757862 -1191923 -2392586 -1298409 -740727 -938564 -11
+58873
-1243077 -1225406 -926746 -784887 -1305888 -1536915 -1224183 -1068087 
+-1132526
-1189950 -1129911 -1072430 -1214240 -1399874 -1359004 -1151840 -986897
+ -1026991
-1112164 -1087705 -1239457 -1426727 -1292056 -1001813 -806323 -824692 
+-996581
-1074320 -990701 -1078806 -1294623 -1150505 -996057 -1112120 -1026471 
+-898243
-1125728 -1402701 -1256597 -1021666 -743675 -484198 -1251500 -1787720 
+-1476381
-1737003 -1318660 -420744 -738887 -1025414 -952742 -1335695 -1441729 -
+1159485
-1171768 -1128103 -712072 -473309 -765466 -1198700 -1285089 -1348419 -
+1556346
-1327956 -971620 -1075261 -1120926 -1099457 -1471453 -1488643 -936739 
+-795910
-1015253 -1117698 -1274139 -1335401 -1178533 -1082915 -1199835 -137339
+8 -1226060
-1015291 -1051447 -997198 -1073021 -1428076 -1435493 -1098443 -1116945
+ -892701
174420 -83292 -1668146 -1877307 -1508544 -1280587 -504159 -607975 -142
+8293
-1301471 -664190 -733295 -1547665 -1889257 -1438016 -929650 -635668 -9
+66941
-1709318 -1526262 -652677 -626175 -1104370 -1161787 -1066086 -1021897 
+-1204104
-1547559 -1260868 -1043039 -1427884 -320452 516777 -1458465 -2573263 -
+1364426
-326671 -762368 -1766573 -1689927 -972438 -634770 -1046064 -1612223 -1
+623946
-1322425 -866368 -876067 -1189239 -934223 -881000 -1333817 -1397151 -1
+148026
-989331 -909200 -1025935 -1210654 -1219816 -1258397 -1316399 -1190640 
+-1113434
-1132030 -1068965 -1001816 -966472 -1267534 -1705439 -1276088 -680003 
+-888269
-1081254 -1002823 -970752 -977912 -1219760 -1427306 -1174881 -981218 -
+1224692
-1385454 -1194642 -1012963 -1104891 -1207310 -1220396 -1395342 -127909
+0 -845828
-1116560 -1437570 -1084730 -1159214 -1190535 -581785 -806557 -1415148 
+-1108899
-1013638 -1332982 -1215643 -1151227 -1200647 -968004 -899747 -1083941 
+-984857
-862614 -1092052 -1230587 -1167887 -1151473 -1140504 -1089388 -1087093
+ -1302907
-1332549 -1039985 -1436184 -1781060 -994629 -719614 -1290174 -1240472 
+-904403
-972206 -1244797 -1336973 -570295 -290696 -1693055 -2201096 -965058 -7
+70006
-1837310 -1719518 -770618 -361287 -586334 -1817225 -2347376 -837843 -2
+54105
-1492344 -1693600 -910120 -427102 -468579 -1605636 -2252801 -1104692 -
+140762
-848383 -1711137 -1131908 -630674 -1153093 -1475118 -1556041 -1417374 
+-833023
-760893 -1291887 -1430175 -1198517 -1146028 -1075614 -1099305 -1353092
+ -614445
-12355 -1355060 -2046745 -1349995 -1430142 -1206187 -98425 -430436 -16
+06793
-932603 -247395 -1634486 -1715870 -657143 -1626560 -2297483 -1002247 4
+62080
35877 -1586491 -1914975 -1532303 -1265764 -1183126 -1540281 -1654876 -
+932297
-85782 -1145971 -2049147 389008 879489 -2274618 -2809777 -1539767 -122
+2566
-880632 -1363944 -2055611 -1070531 -75843 -682017 -1416002 -1534265 -1
+508300
-1211030 -1108849 -1163247 -778277 -868413 -1526254 -1669297 -1274194 
+-798747
-752338 -1077686 -1162498 -1089816 -1196821 -1340222 -1368628 -1252791
+ -1054124
-1404053 -1889470 -1355151 -745665 -939884 -1186114 -1169937 -1105376 
+-1147347
-1181510 -890566 -773205 -1359124 -1502163 -796794 -906557 -1586946 -1
+377471
-1047865 -1030444 -1167082 -1588821 -1263972 -644461 -1008479 -1495011
+ -1342273
-1028058 -988989 -1158112 -1305765 -1315860 -1204892 -1050168 -848403 
+-1023304
-1482909 -1455809 -1284749 -1286722 -1159969 -1103699 -1106097 -943406
+ -952245
-1223480 -1299913 -1160612 -1080231 -1075582 -1058265 -1093522 -132459
+2 -1444696
-1195728 -1058915 -1149150 -1039824 -958094 -1193442 -1444084 -1485409
+ -1314866
-1219947 -1413455 -1335332 -837657 -819562 -1286799 -1379894 -1219435 
+-1124044
-1088228 -1260950 -1245597 -1092072 -1460263 -1665419 -1049773 -689335
+ -1060283
-1219379 -1033441 -975453 -966997 -1156753 -1512748 -1284994 -789371 -
+915644
-1169182 -1147150 -1197323 -1207674 -1067278 -933085 -912254 -1066721 
+-1266233
-1259568 -1045466 -1055480 -1359943 -1403499 -1186318 -1118476 -123297
+3 -1377352
-1290560 -1091953 -1136144 -1225847 -1210870 -1217571 -1231031 -118506
+9 -1113054
-1185330 -1240263 -1120751 -1083007 -993931 -883322 -1175803 -1493373 
+-1355860
-1063971 -1090329 -1345197 -1346019 -1172782 -1127428 -1075348 -110641
+7 -1380152
-1534778 -1463694 -1340902 -1313475 -1568310 -1514233 -954053 -873890 
+-1160319
-1197488 -1313393 -1451298 -1428033 -1021950 -166024 -271752 -1325705 
+-1737896
-1714508 -1495359 -934726 -727873 -838379 -733421 -736044 -1172448 -14
+24758
-1321563 -1314633 -1137709 -1170825 -1620191 -1250956 -635781 -1031522
+ -1500431
-1290616 -1021328 -1033316 -1074291 -1211802 -1334660 -1098954 -994158
+ -1316236
-1431913 -1191996 -1091457 -1165761 -1181486 -1213391 -1344006 -140999
+4 -1435200
-1465518 -1186385 -871016 -976628 -1179847 -1115625 -876088 -1004670 -
+1333060
-1284424 -1258914 -1180806 -833710 -904164 -1264856 -1219181 -1096460 
+-1234681
-1142026 -951652 -1160463 -1429389 -1354783 -1119268 -1141540 -1318320
+ -1315135
-1188595 -1141624 -1284897 -1377030 -1271011 -1023826 -862551 -918367 
+-949460
-1356037 -1786649 -1447122 -1153710 -1019925 -769824 -1136060 -1548436
+ -1111902
-818063 -1244032 -1361190 -1028930 -991652 -1018209 -1012953 -1295930 
+-1371684
-1180156 -1183918 -1240893 -1306134 -1325088 -1220611 -1324199 -134221
+6 -1023687
-872554 -913886 -957518 -1168996 -1446499 -1388738 -1089972 -930928 -9
+29609
-993027 -1015646 -1091791 -1305690 -1343565 -1165748 -1027871 -1012640
+ -1087786
-1186404 -1248425 -1198453 -1129382 -1193703 -1219190 -1148161 -120247
+3 -1261168
-1135200 -1121100 -1289463 -1250003 -1054067 -1120204 -1327277 -123378
+5 -1178537
-1604371 -1739003 -1218323 -1013978 -1188695 -1118849 -1110974 -112792
+3 -1056728
-1406357 -1491787 -959987 -905970 -1147927 -1064593 -1072575 -1156833 
+-1138653
-1216013 -1530899 -1760041 -1134627 -368485 -722795 -1403416 -1340010 
+-1021996
-1173518 -1490663 -1550328 -1375197 -1063157 -979212 -1067480 -1100436
+ -1256527
-1314402 -1182906 -1188332 -1242387 -1169014 -1093018 -1123957 -130452
+7 -1361532
-1192471 -1174671 -1136630 -973948 -987736 -1110964 -1264928 -1347238 
+-1307836
-1253470 -1170125 -1048140 -954800 -1021785 -1249522 -1450832 -1435361
+ -1167651
-980650 -1169147 -1170247 -820999 -1203242 -1873838 -1531581 -1050236 
+-1170669
-1117612 -940681 -1286462 -1517041 -1023648 -834009 -1215297 -1313855 
+-1338401
-1396640 -1132578 -950547 -1069690 -1148066 -1155598 -1273501 -1260452
+ -746425
-262995 -697223 -1568030 -1828812 -1702889 -902338 982211 1114728 -146
+2920
-2892609 -2117097 -1508292 -1451639 -1178200 -1153484 -1570561 -162302
+0 -1300785
-1068096 -1012333 -1261148 -1598204 -1564795 -1260201 -1236513 -133819
+6 -1107258
-936999 -986849 -1042388 -1261079 -1432733 -1208353 -1092080 -1253497 
+-1063788
-963919 -1329327 -1306502 -1044184 -1318947 -1537245 -1113093 -862395 
+-1103857
-1213260 -1183151 -1163644 -1117863 -1269569 -1515117 -1418782 -126965
+2 -1464783
-1353886 -843847 -815884 -1156185 -1418300 -1609264 -1370371 -910253 -
+1057106
-1470584 -1183571 -676737 -885380 -1272998 -1339584 -1383560 -1296692 
+-1141617
-1217674 -1179373 -903166 -896103 -1140247 -1177038 -1331049 -1919074 
+-1977233
-1138244 -516190 -773997 -1457803 -1489718 -935451 -903971 -1207930 -1
+307803
-1270151 -1159960 -1099448 -1090811 -1159112 -1266938 -1183558 -108122
+3 -997369
-1012252 -1529823 -1577220 -829769 -771159 -1241174 -1298038 -1143512 
+-1041211
-1042029 -1233145 -1466345 -1331412 -1146916 -1217168 -1108874 -104956
+7 -1149499
-1181035 -1363742 -1467882 -1281255 -1172830 -1020790 -858906 -1936347
+ -2879456
-1876257 -1128414 -810627 -210514 -434927 -585082 -472618 -1204344 -18
+93927
-1519070 -878879 -743883 -822486 -846772 -1018272 -1371794 -1550654 -1
+315731
-988149 -875586 -840765 -953618 -1241970 -1475921 -1401490 -1068513 -1
+151692
-1326206 -1016491 -1040386 -1406092 -1461914 -1335829 -1175231 -115881
+3 -1210189
-1037623 -919895 -1017856 -1110112 -1219289 -1346192 -1264686 -1107212
+ -1107887
-1086946 -1177693 -1436808 -1343801 -1201297 -1450223 -1395247

[download]
Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond

In reply to Re^4: Quick script to check data logger data by GrandFather
in thread Quick script to check data logger data by GrandFather

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.