Re: updating big XML files

I propose some DB too. Here is my notes:

- eXist, open source XML database (http://exist.sourceforge.net/).
Used for huge documents sometimes, XQuery implemented, but, afaik, it is not transactional.

- proprietary solution (which I am implementing for some reasons), XML data in relational database.
Transactions + concurrency are big plus. Need for implementing DOM-like editing functions. Difficult (I have decided that it is not required for my project) implementation of XPath or XQuery. Slow parsing, inefficient data representation, but fast updating.

- stream processing.
It need not nor DB neither huge RAM. You can write proprietary handlers to process XML doc as stream of events, or you can use something like as http://stx.sourceforge.net/

Update: It is very interesting for me. If you want, give me more informations and I will propose some solutions or I can write some code too.

Comment on Re: updating big XML files

Replies are listed 'Best First'.
Re^2: updating big XML files by dHarry (Abbot) on Jul 21, 2008 at 13:43 UTC
Thanks for the comments! I know eXist I worked with it same years ago (in a Java context). I have done mapping of XML to relational and back myself and it can be tricky! FYI: I have a good document on the topic. I think the best way forward is a RDBMS. It was probably a mistake in the first place to do it in XML. Especially since the files get relatively big and updates are necessary. Transactions are not an issue the application is stand alone.	[reply]
Re^3: updating big XML files by pajout (Curate) on Jul 21, 2008 at 22:47 UTC
Another point of view: If the data structure is well defined and it will not change (or it will change when new release occurs), sql database can fit the data structure very well. But, if the character of the data structure is, for instance, "we know that the tree have following base structure, and some nodes could be enhanced by some data from time to time, some layer keeping generic xml structure could be better than never-ending story of changing definitions of sql tables. In this case, you can choose eXist or implement xml layer in RDBMS.	[reply]