in reply to Re: Calculate aircraft headings from GPS data for google kml display
in thread Calculate aircraft headings from GPS data for google kml display

Thank you for all the comments. But my main problem is still how to loop around all these big dataset and calculate the headings. i think the logic should be something like this:

for ($aircraft_id){ $current_point_x = $latitude $current_point_y = $longitude $waypoint_x = $latitude_next $waypoint_y = $longitude_next //do some calculations on calculate headings ! //output headings }

Any ideas to parse through all these data and do calculations, the data is organised in a way that there are different aircraft in the dataset, but it can be identified by aircraft_id, and they are ordered by gps_time column, how could i get my two continuous points and calculate the heading. In Python, i could use dictionary to do that, as to perl, i don't know. help please !

thank you!

Si

  • Comment on Re^2: Calculate aircraft headings from GPS data for google kml display
  • Download Code

Replies are listed 'Best First'.
Re^3: Calculate aircraft headings from GPS data for google kml display
by BrowserUk (Patriarch) on Jun 10, 2014 at 06:01 UTC
    In Python, i could use dictionary to do that

    In Perl, a "dictionary" is called a hash.

    As you have multiple aircraft, you cannot use the rowids to determine consecutive coordinates for a given aircraft (unless there are always and only 2), so I've used a hash of arrays of hashes to accumulate the data in:

    $planes{ $aid }[ n ]{ ... };

    This print the headings for each aircraft as it reads the data:

    C:\test>junk53 Aircraft: 599 heading: 0.000 Aircraft: 591 heading: 90.000 Aircraft: 599 heading: 0.000 Aircraft: 591 heading: 90.000

    Would've been nice if the sample contained some heading changes, and I'm somewhat suspicious of the exact 90° difference in the aircraft headings, so check the math. but this should get you started:

    #! perl -slw use strict; use Data::Dump qw[ pp ]; my @headers = split ' ', do{ my $raw = <DATA>; $raw =~ tr[",][ ]; $raw + }; #" my %planes; while( my $line = <DATA> ) { $line =~ tr[",][ ]; #" my @fields = split ' ', $line; my $rowid = shift @fields; my $aid = shift @fields; my %row; @row{ @headers } = @fields; push @{ $planes{ $aid } }, \%row; next unless @{ $planes{ $aid } } > 1; my $lat1 = $planes{ $aid }[ -2 ]{ latitude }; my $lat2 = $planes{ $aid }[ -1 ]{ latitude }; my $dlat = $lat2 - $lat1; my $lon1 = $planes{ $aid }[ -2 ]{ longitude }; my $lon2 = $planes{ $aid }[ -1 ]{ longitude }; my $dlon = $lon2 - $lon1; my $y = sin( $dlon / 180 ) * cos( $lat2 /180 ); my $x = cos( $lat1 /180 ) * sin( $lat2/180 ) - sin( $lat1/180 ) * +cos( $lat2/180 ) * cos( $dlon/180 ); printf "Aircraft: %d heading: %6.3f\n", $aid, atan2( $y, $x ) * 57 +.2957795; } __DATA__ "rowid","aircraft_id","actual_date_time","latitude","longitude","radio +_altitude","ground_speed","thrust_engine_1","thrust_engine_2" 3828994,599,2012-11-04 14:00:00,51.47592545,-0.437049866,-9.0,2.0,0.04 +4653355,0.041778761,1.74208527619370935228499282740898271690901934191 +164070390564,15.22092881857486030932218666006017736321211426212568671 +811012 3828995,591,2012-11-04 14:00:00,51.47598267,-0.435331702,-0.875,0.0,0. +066208174,0.068682498,0.913095793953437657822846430541224159086453865 +40627114668956,4.8940906898850944664550883969746212941234464464906348 +3492349 3828996,599,2012-11-04 14:00:01,51.47592545,-0.437049866,-9.0,2.0,0.04 +4653355,0.041778761,1.74208527619370935228499282740898271690901934191 +164070390564,15.22092881857486030932218666006017736321211426212568671 +811012 3828997,591,2012-11-04 14:00:01,51.47598267,-0.435331702,-0.75,0.0,0.0 +64970428,0.068682498,0.9055060385715144395793861983988312690635603870 +3789731965392,4.92096836684002218971971504299700119099094370033875757 +916224 3828998,599,2012-11-04 14:00:02,51.47592545,-0.437049866,-9.0,2.0,0.04 +4653355,0.041778761,1.74208527619370935228499282740898271690901934191 +164070390564,15.22092881857486030932218666006017736321211426212568671 +811012 3828999,591,2012-11-04 14:00:02,51.47598267,-0.435331702,-0.625,0.0,0. +066208174,0.068682498,0.913095793953437657822846430541224159086453865 +40627114668956,4.8940906898850944664550883969746212941234464464906348 +3492349

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Awesome man, thanks for the tips, that's very helpful !

      However, i have few more questions regarding your clever code, :-)!

      first, what does following few lines of code do ? i think the first one is to trim the raw data and put it into array, but i never saw lots of expressions you put there before, can you give a bit hint, especially the # at the end, :-)!

      my @headers = split ' ', do{ my $raw = <DATA>; $raw =~ tr[",][ ]; $raw + }; #" $line =~ tr[",][ ]; #" push @{ $planes{ $aid } }, \%row; next unless @{ $planes{ $aid } } > 1;

      second, i understand the key method is to get it into a hash dictionary, by using:

      my $lat1 = $planes{ $aid }[ -2 ]{ latitude }; my $lat2 = $planes{ $aid }[ -1 ]{ latitude };

      But this does not seem to pick up the right number, i did a printf to check, it seems always 0.

      third, for the heading calculations, i think the math is fine, because the data represents the aircrafts at the take-off or landing stage, so most of heading should be 90 or 180 degree. also i don't need it to be that accurate, it's only for display purpose on the google earth. i did a check on one pair of coordinates by using your formula, it seems work fine.

      my $lats1=51.4774704; my $lats2=51.4774704; my $lons1=-0.458850861; my $lons2=-0.459365845; my $dlats = $lats2 - $lats1; my $dlons= $lons2 - $lons1; my $y1 = sin( $dlons / 180 ) * cos( $lats2 /180 ); my $x1 = cos( $lats1 /180 ) * sin( $lats2/180 ) - sin( $lats1/180 +) * cos( $lats2/180 ) * cos( $dlons/180 ); printf "heading: %d\n", atan2( $y1, $x1 ) * 57.2957795

      I also upload more data here, in case you want to check !

      _Data_

      "rowid","aircraft_id","actual_date_time","latitude","longitude","radio +_altitude","ground_speed","thrust_engine_1","thrust_engine_2","co2"," +nox" 11782888,685,2012-11-04 15:25:14,51.4774704,-0.458850861,-9.25,64.5,0. +028000256,0.028713875,1.577894599,22.32443440 11782889,721,2012-11-04 15:25:15,51.47083282,-0.478762537,-7.0,0.0,0.0 +2504996,0.029270458,2.672078122,25.016723749 11782890,685,2012-11-04 15:25:15,51.4774704,-0.459365845,-9.5,60.75,0. +028000256,0.029427142,1.590756897,22.13159408 11782891,709,2012-11-04 15:25:15,51.46648407,-0.468292236,-9.5,15.25,0 +.077369954,0.068239666,4.124230104,28.73529887 11782892,704,2012-11-04 15:25:15,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782893,721,2012-11-04 15:25:16,51.47083282,-0.478762537,-7.0,0.0,0.0 +2504996,0.029270458,2.678124860,24.952298408 11782894,704,2012-11-04 15:25:16,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782895,685,2012-11-04 15:25:16,51.4774704,-0.459880829,-9.5,57.25,0. +028713875,0.029427142,1.603534644,21.93343790 11782896,709,2012-11-04 15:25:16,51.46648407,-0.468292236,-9.5,15.25,0 +.077369954,0.068239666,4.124230104,28.73529887 11782897,721,2012-11-04 15:25:17,51.47083282,-0.478762537,-7.0,0.0,0.0 +2504996,0.029270458,2.678553119,24.966366785 11782898,685,2012-11-04 15:25:17,51.4774704,-0.460224152,-9.5,54.0,0.0 +28713875,0.029427142,1.603534644,21.93343790 11782899,709,2012-11-04 15:25:17,51.46648407,-0.468120575,-9.25,15.5,0 +.077369954,0.068239666,4.124230104,28.73529887 11782900,704,2012-11-04 15:25:17,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782901,721,2012-11-04 15:25:18,51.47083282,-0.478762537,-7.0,0.0,0.0 +25653887,0.029270458,2.684827917,24.909981212 11782902,685,2012-11-04 15:25:18,51.4774704,-0.460567474,-9.5,50.5,0.0 +28713875,0.030140059,1.616481248,21.74572268 11782903,704,2012-11-04 15:25:18,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782904,709,2012-11-04 15:25:18,51.46648407,-0.467948914,-9.0,15.75,0 +.076671196,0.068239666,4.101445938,28.85420529 11782905,721,2012-11-04 15:25:19,51.47083282,-0.478762537,-7.0,0.0,0.0 +25653887,0.028668525,2.684902618,24.912091811 11782906,709,2012-11-04 15:25:19,51.46648407,-0.467948914,-9.0,16.0,0. +076671196,0.068239666,4.101445938,28.85420529 11782907,704,2012-11-04 15:25:19,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782908,685,2012-11-04 15:25:19,51.4774704,-0.460910797,-9.5,47.25,0. +029427142,0.030852625,1.642374213,21.37011001 11782909,721,2012-11-04 15:25:20,51.47083282,-0.478762537,-7.0,0.0,0.0 +2625748,0.028668525,2.722969965,24.583642373 11782910,709,2012-11-04 15:25:20,51.46648407,-0.467777252,-9.0,16.5,0. +076671196,0.068239666,4.101445938,28.85420529 11782911,704,2012-11-04 15:25:20,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782912,685,2012-11-04 15:25:20,51.4774704,-0.46125412,-9.5,44.5,0.02 +9427142,0.030852625,1.642374213,21.37011001 11782913,721,2012-11-04 15:25:21,51.47083282,-0.478762537,-7.0,0.0,0.0 +28066262,0.029270458,2.781087133,24.122891683 11782914,709,2012-11-04 15:25:21,51.46648407,-0.467605591,-9.25,16.5,0 +.076671196,0.068239666,4.101445938,28.85420529 11782915,685,2012-11-04 15:25:21,51.4774704,-0.461597443,-9.5,41.75,0. +029427142,0.030852625,1.642374213,21.37011001 11782916,704,2012-11-04 15:25:21,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782917,721,2012-11-04 15:25:22,51.47083282,-0.478762537,-7.0,0.0,0.0 +2987206,0.02987206,2.826355438,23.766313359 11782918,685,2012-11-04 15:25:22,51.4774704,-0.461940765,-9.5,39.5,0.0 +30140059,0.030852625,1.655320817,21.18239479 11782919,704,2012-11-04 15:25:22,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782920,709,2012-11-04 15:25:22,51.46648407,-0.467433929,-9.25,16.75, +0.076671196,0.068945601,4.122771027,28.71316579 11782921,721,2012-11-04 15:25:23,51.47083282,-0.478762537,-7.0,0.0,0.0 +3167489,0.031074276,2.878496092,23.370307854 11782922,709,2012-11-04 15:25:23,51.46648407,-0.467433929,-9.0,17.0,0. +077369954,0.068945601,4.145555193,28.59425937 11782923,685,2012-11-04 15:25:23,51.4774704,-0.462284088,-9.75,37.25,0 +.030140059,0.030852625,1.655320817,21.18239479 11782924,704,2012-11-04 15:25:23,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782925,721,2012-11-04 15:25:24,51.47083282,-0.478762537,-7.0,0.0,0.0 +3167489,0.03167489,2.891194602,23.261540959 11782926,685,2012-11-04 15:25:24,51.4774704,-0.46245575,-9.75,35.5,0.0 +29427142,0.031564842,1.655488701,21.19210642 11782927,704,2012-11-04 15:25:24,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782928,709,2012-11-04 15:25:24,51.46648407,-0.467262268,-9.25,17.5,0 +.077369954,0.068239666,4.124230104,28.73529887 11782929,721,2012-11-04 15:25:25,51.47083282,-0.478762537,-7.0,0.0,0.0 +3167489,0.032875134,2.897547209,23.208410778 11782930,704,2012-11-04 15:25:25,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782931,685,2012-11-04 15:25:25,51.4774704,-0.462799072,-9.5,34.5,0.0 +29427142,0.031564842,1.655488701,21.19210642 11782932,709,2012-11-04 15:25:25,51.46648407,-0.467090607,-9.25,17.75, +0.077369954,0.068239666,4.124230104,28.73529887 11782933,721,2012-11-04 15:25:26,51.47083282,-0.478762537,-7.0,0.0,0.0 +3167489,0.033474765,2.904146185,23.161377706 11782934,704,2012-11-04 15:25:26,51.46665573,-0.442199707,-9.0,0.75,0. +033609102,0.033609102,2.074123446,51.09631094 11782935,685,2012-11-04 15:25:26,51.4774704,-0.462970734,-9.5,33.75,0. +029427142,0.031564842,1.655488701,21.19210642 11782936,709,2012-11-04 15:25:26,51.46648407,-0.467090607,-9.25,18.0,0 +.077369954,0.068239666,4.124230104,28.73529887 11782937,721,2012-11-04 15:25:27,51.47083282,-0.478762537,-7.0,0.0,0.0 +3167489,0.033474765,2.890677441,23.248813256 11782938,709,2012-11-04 15:25:27,51.46648407,-0.466918945,-9.0,18.25,0 +.077369954,0.068945601,4.145555193,28.59425937 11782939,685,2012-11-04 15:25:27,51.4774704,-0.463142395,-9.5,32.25,0. +029427142,0.031564842,1.655488701,21.19210642
        can you give a bit hint, especially the # at the end, :-)!
        1. my @headers = split ' ', do{ my $raw = <DATA>; $raw =~ tr[",][ ]; $raw }; #"
          • Read the header line into $raw;
          • Get rid of the quotes and commas and replace with spaces;
          • return the modified line to split and split on whitespace;
          • The #" at the end is just a comment; it balances up the earlier ", and prevents it from screwing up the syntax highlighting in my editor.
        2. $line =~ tr[",][ ]; #"

          Same as above. (Though it was wrong for the rest of the lines, a C&P error correct below.)

        3. push @{ $planes{ $aid } }, \%row;

          push a reference to the hash constructed from teh most recent line into the array indexed by the aircraft id.

        4. next unless @{ $planes{ $aid } } > 1;

          If we don't have 2 or more lines (in the array) for this aircraft, we cannot calculate a bearing yet, so skip to the next line.

        A couple of corrections and a couple of tweaks:

        #! perl -slw use strict; use Data::Dump qw[ pp ]; my @headers = split ' ', do{ my $raw = <DATA>; $raw =~ tr[",][ ]; $raw + }; #" my %planes; while( my $line = <DATA> ) { my @fields = split ',', $line; my %row; @row{ @headers } = @fields; my $rowid = $row{ rowid }; my $aid = $row{ aircraft_id }; push @{ $planes{ $aid } }, \%row; next unless @{ $planes{ $aid } } > 1; my $lat1 = $planes{ $aid }[ -2 ]{ latitude }; my $lat2 = $planes{ $aid }[ -1 ]{ latitude }; # print "$lat1 $lat2"; my $dlat = $lat2 - $lat1; my $lon1 = $planes{ $aid }[ -2 ]{ longitude }; my $lon2 = $planes{ $aid }[ -1 ]{ longitude }; # print "$lon1 $lon2"; my $dlon = $lon2 - $lon1; my $y = sin( $dlon /180 ) * cos( $lat2 /180 ); my $x = cos( $lat1 /180 ) * sin( $lat2 /180 ) - sin( $lat1 /180 ) +* cos( $lat2 /180 ) * cos( $dlon /180 ); printf "Aircraft: %d heading: %6.3f\n", $aid, atan2( $y, $x ) * 57 +.2957795; # <STDIN>; } __DATA__

        With the extended dataset (still no course changes!), produces:

        C:\test>junk53 Aircraft: 685 heading: -90.000 Aircraft: 721 heading: 0.000 Aircraft: 704 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 709 heading: 0.000 Aircraft: 721 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 709 heading: 90.000 Aircraft: 704 heading: 0.000 Aircraft: 721 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 704 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 721 heading: 0.000 Aircraft: 709 heading: 0.000 Aircraft: 704 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 721 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 704 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 721 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 685 heading: -90.000 Aircraft: 704 heading: 0.000 Aircraft: 721 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 704 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 721 heading: 0.000 Aircraft: 709 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 704 heading: 0.000 Aircraft: 721 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 704 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 721 heading: 0.000 Aircraft: 704 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 709 heading: 90.000 Aircraft: 721 heading: 0.000 Aircraft: 704 heading: 0.000 Aircraft: 685 heading: -90.000 Aircraft: 709 heading: 0.000 Aircraft: 721 heading: 0.000 Aircraft: 709 heading: 90.000 Aircraft: 685 heading: -90.000

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re^3: Calculate aircraft headings from GPS data for google kml display
by boftx (Deacon) on Jun 09, 2014 at 23:25 UTC

    Can you prove a small sample of the dataset that illustrates the structure?

    DOH!

    You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.

      Hi boftx, the data sample is in my original post, let me knwo if it's not clear enough !

        Without looking at the calcs in detail, I would say that you should convert the data (if size is not prohibitive) to a hash where each key is the aircraft ID and the value is an array hashes containing the other data points from each row with that aircraft ID.

        You can then easily loop over the aircraft IDs. Assuming the data rows are already sorted by time it should be a relatively simple matter to step through the array items at that point calculating the deltas.

        You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.