Re^2: Testing database updates.

Thanks for your advice.

The system I decided on was to modify the script I am re-factoring to take the path of an SQLite database on the command line. I then wrote a wrapper .t test script that creates an empty DB with just the schema before each test run. After each test run the DB is dumped to text and compared with the expected output.

My personal take is that I'd much rather start out looking at everything and explicitly choose what I can disregard -- probably through post processing the dumps -- than to prejudge the issues by preselecting what to compare.

I am post processing the dumps as well. I discovered almost immediately that one table contains a timestamp collum that defaults to the current time, so I had to replace that time stamp with a fixed string. I am also post processing the text output to suppress differences in newline chars and trailing whitespace.

I dare say that I will soon have to pass the XML output through a tidy program to remove non syntactic differences in spacing & newlines, or just load both the actual and expected output with XMLin (from XML::Simple), and pass the passed data to is_deeply()

Comment on Re^2: Testing database updates. Select or Download Code

Replies are listed 'Best First'.
Re^3: Testing database updates. by BrowserUk (Patriarch) on Apr 19, 2012 at 22:26 UTC
I dare say that I will soon have to pass the XML output XML? Did you choose that format? Looking at the help from my copy of sqllite3 XML isn't even an option: `.mode MODE ?TABLE? Set output mode where MODE is one of: csv Comma-separated values column Left-aligned columns. (See .width) html HTML <table> code insert SQL insert statements for TABLE line One value per line list Values delimited by .separator strin +g tabs Tab-separated values tcl TCL list elements` [download] Unless you chose HTML? Or did they add an XML option recently? Choosing XML over the ,dump SQL format (IMO) is a big mistake. The SQL format is so simple to manipulate and compare: sqlite> .dump onegb PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE onegb ( alpha varchar, no varchar, hex varchar, bin varch +ar ); INSERT INTO "onegb" VALUES('nvbqurkn','05812224','42295f70','"Mƒp®Iÿ&# +9618;SpæÕ¦├ƒ±¦c²NBÍ¹sÙ-¼┐ºâ5ó<Ú="'); INSERT INTO "onegb" VALUES('xzpviekv','72456229','fd245120','"ª¿.&#957 +1;¡ù©=àô²Ð╣j‗Èl╝°¾©▓º§~zo=¬fIí┬Z"'); INSERT INTO "onegb" VALUES('qrkhwqwl','43135665','316f7a36','"æe&#9559 +;´Í└╬┴╚╔¸Éwf±Û6lòêqc¸▄iZñƒ┤ +}E⌂¤┤"'); ... [download] (Okay. That's not the most complex schema in the world; but one advantage of sqlite for this kind of thing is that it actually uses very few different datatypes.) Whereas comparing XML is a nightmare. Less so I suppose if you don't have to worry about the attributes/nested sub-items equivalence -- I assume that however you are generating the XML, it at least produce consistent XML. I think you are making hard work for yourself using XML, but as I said above somewhere, it is your choice. Good luck :) With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l] [select]
Re^4: Testing database updates. by chrestomanci (Priest) on Apr 20, 2012 at 08:09 UTC
I think you miss understood me. I am not using XML as a dump format for my database. I am using the default export format of SQL insert statements. The reason I need to parse XML, is because the script I am re-factoring emits results in the form of XML and text files as well as inserts into a database. I need to make sure the file output is functionally identical as well as the database output. In terms of comparing the XML, I think using XMLin will be sufficient, as most of the consumers of the XML files are other perl scripts (that I don't control), so if the before and after XML is identical as far as XML::Simple is concerned, then it should be acceptable to those consuming perl scripts.	[reply]