It feels like "Pass a cloned or (externally) shared XML::Writer object to your threads and (externally) serialise use of it through locking." would be an option. Can you describe how you would do it?

Still not enough info. I don't need to see the details of your content,nor even the real field names. But some idea of the structure. For example, would something like this be acceptable as output:

<?xml version="1.0" encoding="UTF-8"?> <operations> <sites-list2>website 1</sites-list1> <product>product 1</product> <sites-list1>website 1</sites-list1> <product>product 2</product> <sites-list1>website 2</sites-list1> </operations>

Or this (or some other essentially random ordering of it?):

<?xml version="1.0" encoding="UTF-8"?> <operations> <website-1-1>url</website-1-1> <product-2> <sites-list1> </product-1> <detail-2>stuff</detail-2> <website-2-1>url</website-2-1> <detail-1>stuff</detail-1> <detail-1>stuff</detail-1> <sites-list2> <website-2-2>url</website-2-2> </sites-list1> <products> <product-1> </product-2> </product> <website-1-2>url</website-1-2> </sites-list1> <detail-2>stuff</detail-2> </operations>

That's shuffled rather than interleaved, but it makes my point. Depending upon the structure of the data from different sources, the locking requirements vary. If each of the data sources only produced a single line--one tag (pair) and lots of attributes--and it didn't matter how the lines from the 3 sources where interleaved, then you would use a different solution than if the data sources produce deeply nested structures of tags for each row.

If, as I suspect, you want all the rows from each data source grouped together (possibly under a top-level tag for that data sources), and your " No, the ordering of data is irrelevant" means that you don't care which order the 3 blocks are in the file, you'd be better creating the output from each data source separately (perhaps in ram files using three separate instances of XML::Writer), and then combining them into an output file as 3 large chunks.

But if that is the case, I would seriously consider using the -xml flag on the mysql or mysqldump commands.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
RIP an inspiration; A true Folk's Guy

In reply to Re^9: Multithreaded xml writing using XML::Writer by BrowserUk
in thread Multithreaded xml writing using XML::Writer by DreamT

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.