perl_help26 has asked for the wisdom of the Perl Monks concerning the following question:

Suppose I have a complex sql statement STRING such as:

"INSERT INTO TABLE1 ( PARAM1, PARAM2, PARAM3, PARAM4 , PARAM5, PAR +AM6 ,PARAM7) values (SELECT nextval('SEQ_TABLE1') , 9613000001 , 1 ,null , PARAM +8 , 1234 , localtimestamp FROM TABLE2 , TABLE3 WHERE TABLE2.PARAM1= TABLE3.PARAM2 AND TABLE2.PARAM3 = 1)"

I would like to parse it somehow to get the parameters mappings such as : PARAM1=>SELECT nextval('SEQ_TABLE1')
PARAM2=>9613000001 etc...
Is there a good parser module ?? I didn't find a module that really pulls it off. If not, could we somehow parse it on the SQL level? Thanksss..

Replies are listed 'Best First'.
Re: Parse complex SQL statement
by roboticus (Chancellor) on Jun 11, 2014 at 15:54 UTC

    perl_help26:

    I've had used SQL::Parser in the past. It worked well for my needs.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      Actually, I tried SQL::Parser. It really gets lost and gives me undef values for all parameters ( my guess is because it has nested queries )... Any other suggestion:S?

      $VAR1 = { 'table_names' => [ 'table1' ], 'values' => [ [ { 'value' => 'SEQ_TABLE1', 'type' => 'string', 'fullorg' => 'SEQ_TABLE1' } ] ], 'column_aliases' => {}, 'org_col_names' => [ 'PARAM1', 'PARAM2', 'PARAM3', 'PARAM4', 'PARAM5', 'PARAM6', 'PARAM7' ], 'column_defs' => [ { 'fullorg' => 'PARAM1', 'type' => 'column', 'value' => 'param1' }, { 'fullorg' => 'PARAM2', 'type' => 'column', 'value' => 'param2' }, { 'fullorg' => 'PARAM3', 'type' => 'column', 'value' => 'param3' }, { 'type' => 'column', 'fullorg' => 'PARAM4', 'value' => 'param4' }, { 'value' => 'param5', 'fullorg' => 'PARAM5', 'type' => 'column' }, { 'type' => 'column', 'fullorg' => 'PARAM6', 'value' => 'param6' }, { 'value' => 'param7', 'type' => 'column', 'fullorg' => 'PARAM7' } ], 'dialect' => 'ANSI', 'org_table_names' => [ 'TABLE1' ], 'command' => 'INSERT', 'original_string' => 'INSERT INTO TABLE1 ( PARAM1, PARAM2, + PARAM3, PARAM4 , PARAM5, PARAM6 ,PARAM7)values (SELECT nextval(\'SEQ_TABLE1\') , 9613000001 , 1 , , PARAM8 , 1234 + , localtimestamp FROM TABLE2 , TABLE3 WHERE TABLE2.PARAM8= TABLE3.PARAM2 AND SETTING = 1)', 'ORG_NAME' => { 'param5' => undef, 'param3' => undef, 'param2' => undef, 'param7' => undef, 'param1' => undef, 'param6' => undef, 'param4' => undef }, 'join' => {} };
Re: Parse complex SQL statement
by kennethk (Abbot) on Jun 11, 2014 at 15:57 UTC

    A CPAN search for parse sql yields a myriad of results, including SQL::Parser. I've never used the module, but it looks like it has a very long history and its most recent commits are only a few weeks old. Did you try this one? How didn't it work for you?


    #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

      As I already mentioned: I tried it but it's giving me undef values for all parameters ! maybe because of nested queries? Thanksss

        One possibility for why the parser is failing is I think your posted SQL is invalid: unless I'm reading it wrong, you have 7 parameters to insert but 8 terms because you have a repeated comma. I also note no nested queries in the query provided, so it's also possible that I can't diagnose the issue because you haven't posted the correct target.

        If this is an internal utility, you could accomplish your goal with something like

        m/insert\s+into\s+(\S+)\s*\(\s*([^)]+?)\s*\)\s* (?:values\s*\(\s*([^)]+?)\s*\) | select\s*(.+?)\s+(?:where|order) ) /ix
        but this has a number of potential issues: nested queries will parse incorrectly, it fails to consider quoted names and string literals, ... Which is why I usually suggest canned code.

        Update: Or misparse by me; the regular expression should read

        m/insert\s+into\s+(\S+)\s*\(\s*([^)]+?)\s*\)\s* (?:values\s*\(\s*([^)]+?)\s*\) | select\s*(.+?)\s+(?:from\b) ) /ix
        Of course, your posted SQL doesn't actually correspond to the SQL posted in the original_string field in Re^2: Parse complex SQL statement.

        #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.