Re^2: Why can code be so slow?

I am using XML files because everything needs to be universally exportable to Flash and certain other applications where XML would be the easiest/universal solution.

For meta information of a file I use inode.xml ; for settings for that current folder I use default.xml file etc.. I made the system that way I would never read more than 2 XML files maximally ; except when reading a full directory where I'd normally extract the metadata from the files I'd extract that info from XML files.

That way I can also cross-export this info quick-and-easy to any other applications.. atleast.. I thougth so ;) My XML usage is sure not meant as database but rather as easy export/format for other systems working together with the data.

Comment on Re^2: Why can code be so slow?

Replies are listed 'Best First'.
Re^3: Why can code be so slow? by dragonchild (Archbishop) on May 02, 2007 at 15:58 UTC
In other words, you're hand-coding any searches you might need to do and iterating through datastructures that you create every time using an XML parser. And you're wondering why your system is slow? If you profile, I'll bet that the problem you're running into is in one or both of these places: Your XML parser is fast, but your usage isn't. 10-to-1 you're using the Tree option of XML::Parser, which is the slowest option. Once you have this data in memory, you're not working with them correctly. Algorithm choice is the second largest factor in runtime performance (after database (mis)usage). Also, if you're reading larger XML files (say, for your default.xml), then perl has to allocate RAM for the data structures. Depending on your OS, this could be up to 10% of your runtime. The proper solution, in case you're wondering, is to use some sort of database and export to XML as needed. So, use either something like MySQL (if you want a RDBMS and are comfortable with it) or a DBM solution could be DBM::Deep or BerkleyDB. In other words, solve the problem at hand first, then extend the solution. Baby steps. My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]

Replies are listed 'Best First'.

Re^3: Why can code be so slow?
by dragonchild (Archbishop) on May 02, 2007 at 15:58 UTC

Your XML parser is fast, but your usage isn't. 10-to-1 you're using the Tree option of XML::Parser, which is the slowest option.
Once you have this data in memory, you're not working with them correctly. Algorithm choice is the second largest factor in runtime performance (after database (mis)usage).

Also, if you're reading larger XML files (say, for your default.xml), then perl has to allocate RAM for the data structures. Depending on your OS, this could be up to 10% of your runtime.

The proper solution, in case you're wondering, is to use some sort of database and export to XML as needed. So, use either something like MySQL (if you want a RDBMS and are comfortable with it) or a DBM solution could be DBM::Deep or BerkleyDB.

In other words, solve the problem at hand first, then extend the solution. Baby steps.

My criteria for good software:

Does it work?
Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

[reply]