http://qs1969.pair.com?node_id=439530

smeenz has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, O wise ones.

I'm writing an application to work with a Cisco Call Manager.. it provides an SOAP (XML over HTTP) interface, which it calls AXL.

Basically, I send it a properly formatted request, and it sends me a properly formatted reply, which may contain an error code, or the result I'm after.

For getXXXX requests, I'm just creating the request manually, because it's static and only one line long, and I'm using XML::Twig to parse the reply, and that all seems good.

The problem comes when I want to send a dynamic and larger request packet, to create a new phone on the call manager. In this case, the data exists in a hash, in the right structure, but I'm dammed if I can figure out how to get Twig to take an existing hash, and produce XML output.

I've tried doing it with XML::Simple, but that creates output that, which technically okay, is not formatted in a way that the AXL service likes:

With XML::Simple, I can produce this sort of thing:

<newPhone name='bob' value1='blah' value2='blah2'/>

But what I need is this:

<newPhone> <name>bob</name> <value1>blah</value1> <value2>blah2</value2> </newPhone>
I know that Twig can print() it okay, but the only way I can figure to get data into Twig in the first place is to parse an existing XML document, and if I could produce that, well then I wouldn't have a problem. My fallback position is to write my own routine to process a hash, similar to what the Data Dumper does.

Replies are listed 'Best First'.
Re: Outputting a hash as XML
by Tanktalus (Canon) on Mar 15, 2005 at 03:11 UTC

    XML::Twig is too flexible, and too verbose. But it's really quite cool once you get it working ... You didn't give your original hash structure, so I pretended what it was.

    use strict; use XML::Twig; my $hash = { name => 'bob', value1 => 'blah', value2 => 'blah2', }; my $twig = XML::Twig->new(pretty_print => 'indented'); my $elt = create_element(newPhone => $hash); $twig->set_root($elt); $twig->print(); sub create_element { my $gi = shift; my $data = shift; my $t = XML::Twig::Elt->new($gi); if (ref $data) { while (my ($k,$v) = each(%$data)) { create_element($k, $v)->paste(last_child => $t); } } else { $t->set_text($data); } $t; }
    Saved as "x":
    $ perl x <newPhone> <value1>blah</value1> <name>bob</name> <value2>blah2</value2> </newPhone>
    Hope that helps.

      XML::Twig is too flexible, and too verbose.

      Hey, did you try using the DOM? Now that would be what I'd call verbose!

      Seriously, if you have any idea about improvements, please send them my way, I'd be glad to include them (although I don't really understand the "too flexible" bit, I would think that flexible is a quality, see Perl...).

        I suppose I would put the XML::Twig::Elt docs into XML::Twig::Elt itself (or, if you have the code for XML::Twig::Elt inside XML/Twig.pm, put the pod into XML/Twig/Elt.pod). The docs are so long, it's hard to get one's head around it all. Maybe even an XML/Twig/Tutorial.pod would be nice (is there one and I'm not aware of it? If so - please put that in XML::Twig's pod! :->). The tutorial itself could just be a cookbook TOC - and there could be XML/Twig/Tutorial/NewTwig.pod (starting without any XML), XML/Twig/Tutorial/Load.pod (loading from string/file), XML/Twig/Tutorial/Handlers.pod, etc.

        Too flexible:

        $ perldoc XML::Twig | grep -i same.*as | wc -l 51
        You have a lot of options which duplicate each other. This simply helps obscure the unique options. I was trying to remember how to paste a twig inside another twig. Except that I didn't know it was called paste ;-). It took me probably 45-60 seconds of scanning the docs to find it. I found many ways to set the tag (gi, set_gi, tag, set_tag, ... did I miss any?), before getting paste. I also found (maybe this is just me) that paste was counterintuitive. I was expecting to tell the parent object what to insert, rather than telling the child object what its new parent is. So the first time I ran the above code, I had the create_element($k, $v) mixed up with the $t on the paste line. Of course, it didn't work. And, of course, you can't change this without breaking existing code.

        Don't get me wrong - XML::Twig is great. It has made me look good in my job (which is always important). And the fact that my successor on that project is creating XML by simply printing it out is, I'm sure, going to come back and bite him in the arse. (It's not quite the same physical project - just the same idea.) It just provides, in my experience, a significant learning curve. That said, when I chose to use it, I did look at as many of the XML parsers and handlers as I could on CPAN before choosing XML::Twig. And choose it I did. Because, as you sort of implied, it's the most perlish solution, IMO.

      I found the create_element function really useful, have added a slight modification to the above function to handle keys that are in turn array references
      sub create_element { my $gi = shift; my $data = shift; my $node = XML::Twig::Elt->new($gi); if (ref $data eq "HASH") { while (my ($key, $value) = each(%$data)) { if (ref $value eq "ARRAY") { for my $i (@$value) { createXmlTree($key, $i)->paste(last_child => $node); } } else { createXmlTree($key, $value)->paste(last_child => $node); } } } else { $node->set_text($data); } $node; };
      Thanks Joseph
Re: Outputting a hash as XML
by JediWizard (Deacon) on Mar 15, 2005 at 03:15 UTC

    Is there any reason you are assembling and parsing the xml by hand as opposed to using SOAP::Lite? I have always had success using this module, especialy when passing around complex data structures via soap.


    A truely compassionate attitude towards other does not change, even if they behave negatively or hurt you

    —His Holiness, The Dalai Lama

Re: Outputting a hash as XML
by rg0now (Chaplain) on Mar 15, 2005 at 10:46 UTC
    I think what you want to use here is the NoAttr option of XML::Simple. This inhibits the generated XML to contain attributes and all hash key/values will be represented as nested elements instead.
    use warnings; use strict; use XML::Simple; my $hash = { name => 'bob', value1 => 'blah', value2 => 'blah2', }; my $xs = new XML::Simple; my $xml = $xs->XMLout($hash, NoAttr => 1, RootName=>'newPhone', ); print $xml; __END__ <newPhone> <name>bob</name> <value1>blah</value1> <value2>blah2</value2> </newPhone>
    The sad truth is that XML::Simple options can somewhat paradoxically get really convoluted, but is is really useful for tasks of similar kind...

    rg0now

Re: Outputting a hash as XML
by m-rau (Scribe) on Mar 15, 2005 at 09:09 UTC
    A couple of weeks ago I checked out XML::Smart. It works perfectly to transform a hash into a valid XML document.
Re: Outputting a hash as XML
by beauregard (Monk) on Mar 15, 2005 at 03:40 UTC
    XML::Dumper might be closer to what you want, although probably not perfect.

    c.

Re: Outputting a hash as XML
by mirod (Canon) on Mar 15, 2005 at 10:38 UTC

    First, there is probably a way to get XML::Simple to output what you want, but anyway, here is the easiest way I found with XML::Twig: the first new creates the root element, then the map creates its children, each created by the second new.

    #!/usr/bin/perl -w use strict; use XML::Twig; my %hash= ( name => 'bob', value1 => 'blah', value2 => 'blah2', ); XML::Twig::Elt->new( newPhone => map { XML::Twig::Elt->new( $_ => $has +h{$_}) } sort keys %hash) ->print( 'indented' );
Re: Outputting a hash as XML
by rev_1318 (Chaplain) on Mar 15, 2005 at 10:50 UTC
    Have a look at the NoAttr option of XML::Simple.

    Paul

Re: Outputting a hash as XML
by smeenz (Sexton) on Mar 15, 2005 at 22:17 UTC
    Oh dear.. I've run into a new problem now. It seems that the XML parser at the other end is quite picky about the order the tags appear in. In other words, in the following two examples, only the second is valid (it expects to see name before description)

    <newPhone> <description>Test AXL addPhone</description> <name>SEP0028A9B0B21</name> <product>Cisco 7970</product> <model>Cisco 7970</model> </newPhone> <newPhone> <name>SEP0028A9B0B21</name> <description>Test AXL addPhone</description> <product>Cisco 7970</product> <model>Cisco 7970</model> </newPhone>

    Is this expected ? It seems to be 'the wrong thing' to me.

      To answer your question: it seems short-sighted to me. But not disallowed by the XML spec. XML explicitly allows ordering to matter (think of an XHTML doc - very important that the ndoes are in the right order), but for data transfer between programs, I'd not use it unless there are duplicate elements (e.g., more than one "model" tag).

      As to solving it rather than griping about it, without a huge amount of thought applied here, the way I would likely approach this is to have an XML template file, say:

      <newPhone> <name></name> <description></description> <product></product> <model></model> </newPhone>
      Then I would have XML::Twig load it, and then insert the data. I am not sure that there really is a generic solution here, so here's my offering using my favourite XML editor package ;-)
      use strict; use XML::Twig; my $hash = { name => 'bob', product => 'blah', model => 'blah model', description => 'too much money? gimme!', }; my $twig = XML::Twig->new(pretty_print => 'indented'); ## mirod pointed out that I overlooked that parse can ## indeed handle file handles. Woops. :-) $twig->parse(\*DATA); my $elt = create_elements($twig->root(), $hash); $twig->print(); sub create_elements { my $parent = shift; my $data = shift; while (my ($k,$v) = each(%$data)) { # get_xpath returns a list. We should only every find 1 # node, so extract that. my @els = $parent->get_xpath($k); my $el = $els[0]; if ($el) { $el->set_text($v); } else { $el = XML::Twig::Elt($k => $v); $el->paste(last_child => $parent); } } } __END__ <newPhone> <name></name> <description></description> <product></product> <model></model> </newPhone>

      Update: Simplified my parse call, thanks mirod!

      Yes, the order of elements is significant in XML. Some applications are tolerant of different element ordering, but others are picky. It seems your application is picky, so XML::Simple is definitely not the right tool for the job.

        right.. so.. I either kludge this by storing a static list so I know what order to output the tags in, or I read parse the DTD every time to be completely correct, but with the drawback of the additional overhead that would create.

        I think I'll just go for a static list indicating the order the tags need to appear.

Re: Outputting a hash as XML
by smeenz (Sexton) on Mar 15, 2005 at 21:34 UTC
    Thanks everyone for your comments.. I wasn't aware anyone had replied - I thought I had my account configured to notify me and hadn't got any notification.. I'll have to look into that.

    The NoAttr thing might be the simplest solution, although I also like the idea of building a twig, because I think XML::Twig is more likely to produce exactly what I need than XML::Simple will be.

    SOAP::Lite... well.. it might be lite, but it seems so complicated.. I'm not familiar with SOAP at all, and the docs for S:L look like they assume you already know everything about it. Maybe I'll change my code over to that at the next revision..for the moment, I've got a very limited amount of time to get this thing working, so I have to make compromises in terms of elegance in design versus having something work.

Re: Outputting a hash as XML
by holli (Abbot) on Mar 16, 2005 at 21:06 UTC
    just use print();

    I mean you have a data-structure and want it to become XML. So create a function that will produce XML. It may be extra typing than using a module like XML::Twig, but it's the most reliable way. If you want to be smart, make your data-structure a class and add a toXML-method to it.



    holli, /regexed monk/

      I have to wonder if mirod would take offense at implying that XML::Twig is less reliable than other choices ;-)

      For something this small, XML::Twig may be overkill. But for the project we're working on at work, "just us[ing] print()" is actually less reliable. Finding and quashing close-tag errors in a multi-MB XML file is painful. Using a module, such as XML::Twig, helps to avoid these errors.

      And writing a "toXML" function is exactly the same problem that the OP had - just not objectified. For these, I, personally, would still want to rely on an XML module to handle things for me.

      Just my opinion... :-)

      In this case you have to make sure you escape properly & and < at least (plus " in attribute values if you have attributes and use " to quote them). Writing XML is a lot simpler than parsing it, but I still would hesitate to recommend using print to a newcomer. There is enough broken XML floating around, no need to risk adding some more.

      Oh, and of course you need to make sure that the encoding of your data is OK (pure ascii or utf-8) or you need to add an XML declaration with the proper encoding. And most modules won't help you much there.