in reply to storing some values (of an array) to a DB: how does Perl talk to MySQL?
What I am going to suggest here is a bit "sacrilegious" in that I am suggesting that the OP rethink: "i want to store the output in a MySQL-DB", at least as the "next step". For the other readers, I do know something of the application and the data since I wrote part of the code for the Hessen schools. So this isn't a "wild guess".
I think that there is a step before "make an SQL DB" that makes sense.
The data set is going to be all schools in each Bundesland within Germany. Hessen for example had 2,040 schools. A Bundesland is like an American state, but there are fewer of them (16 vs 50).
I would make a single CSV ("Comma Separated Value") file. The first line of this file are the "column headers". I would add "Bundesland" as one of those fields.
The format is called CSV, but the field separator can be anything that you want - doesn't have to be a comma ','. Some of the DB dumps that I work with use '|', the pipe instead of comma. This avoids the commas within quotes and can make the parsing a lot easier. Although Perl does have a very good csv parser module, if I am the one making the .csv file, I make it as easy as possible. For example, I work with one DB dump that comes a 75MB .zip file and I just de-compress and split each line on '|' - works great.
The data set is going to be small enough for it all to fit into memory at once even on a "wimpy" machine. A single table instead of 16 tables (per Bundesland) probably will be just fine for this "next step".
The obvious way to do something useful with this file is to import it into Excel. Excel understands this '|' separated format very well. The Excel version that I have is 10 years old, but even it has some pretty good data table query functions. There used to be an Excel limit of 65K lines per sheet, but that limit is gone now and I don't think this data set will reach that older limit in the first place.
So, what I am suggesting is another "baby step" that can produce a usable DB tomorrow while the learning about SQL and MySQL is happening over the next weeks or months.
I am an advocate of "incremental development" especially when there are multiple new technologies to be learned. Get to something usable as fast as possible, use it, learn and then make a new version.
|
|---|