in reply to Re^2: MedlineParser: to parse and load MEDLINE into a RDBMS
in thread MedlineParser: to parse and load MEDLINE into a RDBMS

I know this is not necessarily a good idea for performance reasons, both in loading and querying, but ... is it possible to automatically translate DTD descriptions into SQL DDL and corresponding code to parse the XML and load the data?

How do you know (or what makes you think) that parsing a DTD is "not ... a good idea for performance reasons ..."? I doubt that using this sort of facility would have any noticeable impact on run-time performance, and it could certainly be a major boost to programmer performance (and would a good way to reduce code that is too bulky and ad-hoc).

There appear to be at least a couple modules on CPAN for converting DTD's into perl-internal objects or data structures: XML::DTDParser, XML::Smart::DTD. (I haven't used either of them myself, but a brief look at the docs makes me think the first one might be more suitable; I expect there are others.)

As for converting a perl object or data structure into an SQL DDL (or going directly from DTD to DDL), I haven't searched CPAN for that (maybe you could try it), but it seems like it could be less of a cut-and-dried sort of task; there might be different ways of specifying a table, or designing a set of relational tables, based on a given DTD, depending on what the SQL users want to do with the data.

(The same could be said for deriving different perl data structures from a DTD, but since people have already posted solutions for this on the CPAN, it might be worth trying what they've come up with.)

  • Comment on Re^3: MedlineParser: to parse and load MEDLINE into a RDBMS

Replies are listed 'Best First'.
Re^4: MedlineParser: to parse and load MEDLINE into a RDBMS
by haricothoriz (Novice) on Jun 26, 2005 at 19:11 UTC
    I know this is not necessarily a good idea for performance reasons, both in loading and querying, but ... is it possible to automatically translate DTD descriptions into SQL DDL and corresponding code to parse the XML and load the data?
    How do you know (or what makes you think) that parsing a DTD is "not ... a good idea for performance reasons ..."? I doubt that using this sort of facility would have any noticeable impact on run-time performance, and it could certainly be a major boost to programmer performance (and would a good way to reduce code that is too bulky and ad-hoc).
    You're right; I don't know for sure, and I agree about the savings in programmer time (frankly what I was interested in). The medline database has 15,000,000 big fat records, and I was guessing that a hand-teaked, denormalized table layout would be more optimal for querying and maybe data loading. But I don't know yet. Thanks for pointing out the CPAN modules. I was just wondering if I had overlooked a module that does exactly what I am interested in.