..full outer join not implemented in DBD::Sqlite....damn
if i go for rdbms it'll take me ages to just load up the tables, as they have varying number of columns and a lot of them (max 47 columns in table). The data is utf-8 with chinese characters in some columns. i'm just matching on first column, which is a plain old long number.
trying to do the full join now in perl. it's already very slow....and this is just on the 1000 line subsets of the files!