Along the lines of what aquarium suggested, here is a small program to create a database approach using DBD::SQLite. Its been a while since I've seen database coding and there are some things that I wasn't able to do like setting a dual primary key on one of the tables. Plus, if you declare a type to be INTEGER PRIMARY KEY, in SQLite it will autoincrement that column for each INSERT INTO statement. But I haven't got that worked out either

At the end of the code are queries for each of the 2 tables (showing all their contents).
Chris 

#!/usr/bin/perl
use strict;
use warnings;
use DBI;

my @data;
chomp(my @sets = split "\t", <DATA>);
<DATA>;

while (<DATA>) {
    my @xys = split;
    
    for my $dataset (0..@xys/2-1) {
        push @{ $data[$dataset] }, [splice @xys, 0, 2];
    }
}

my $dbh = DBI->connect("dbi:SQLite:dbname=datapoints_01.lite","","") o
+r die $!;

$dbh->do(qq{ CREATE TABLE data_sets
    (id INTEGER,
     name TEXT)
});

my $sth = $dbh->prepare(q{INSERT INTO data_sets VALUES(?, ?)}) or die 
+$dbh->errstr;

my $id;
for my $name (@sets) {
    $sth->execute(++$id, $name) or die $dbh->errstr;
}

$dbh->do(qq{ CREATE TABLE data_pts
    (set_id INTEGER,
    id INTEGER,
     x INTEGER,
     y INTEGER)
});

$sth = $dbh->prepare(q{INSERT INTO data_pts VALUES(?, ?, ?, ?)})
    or die $dbh->errstr;

for my $set_id (0..$#data) {
    for my $point_id (0..$#{ $data[$set_id] }) {
        $sth->execute($set_id+1, $point_id+1, @{ $data[$set_id][$point
+_id] })
            or die $dbh->errstr;
    }
}

$sth->finish;
$dbh->disconnect or die $dbh->errstr;

__DATA__
Data Set 1    Data Set 2    Data Set 3 
X units    Y units    X units    Y units    X units    Y units 
1    2    10    20    100    200
3    4    30    40    300    400
5    6    50    60    500    600
__END__
C:\perlp>sqlite3 datapoints_01.lite
SQLite version 3.3.8
Enter ".help" for instructions
sqlite> select * from data_sets;
1|Data Set 1
2|Data Set 2
3|Data Set 3

sqlite> select * from data_pts;
1|1|1|2
1|2|3|4
1|3|5|6
2|1|10|20
2|2|30|40
2|3|50|60
3|1|100|200
3|2|300|400
3|3|500|600
sqlite>

[download]

In reply to Re: Help munging tabular data by Cristoforo
in thread Help munging tabular data by c4onastick

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.