in reply to Re^7: threads::shared seems to kill performance (Workaround).
in thread threads::shared seems to kill performance

I wonder (have not tested) if the ADS is being created, forcing creation of the "normal" file, and at the end, the ADS is being attempted to be deleted, but failing, leaving the data for the next run. ISTR needing to delete the main file (or perhaps some special tool besides unlink) to remove the ADS.

--MidLifeXis

Replies are listed 'Best First'.
Re^9: threads::shared seems to kill performance (Workaround).
by BrowserUk (Patriarch) on Jul 22, 2013 at 15:38 UTC

    An ADS cannot exist without a main file; and unlinking the main file will always delete the ADS; so I'm not sure what you are getting at here?

    I asked the question on the sqlite mailing list; but besides getting the obvious reply -- "if the data persists it isn't an in-memory db"; Well d'uh! -- total silence. Probably there is noone with any windows knowledge on the list.

    My suspicion is:

    • When creating an in-memory db; they use directly mapped virtual address space (memmapped files) to allow the file to grow beyond available memory and be paged on and off disk.

      To do that the need to create a file to 'back' the VAS.

    • On *nix; they probably create the backing file and then immediately unlink it (whilst still holding an open handle to it) so that it disappears when the process ends.

      This mechanism of self=managing temporary files isn't available of Windows, so they've opted to use an alternate datastream. (Perhaps as an attempt to prevent any spooled data being assessible; though that would be forlorn; more.exe can find it.)

      And they forgot to delete the file afterwards.

    Another possibility is that using threads and multiple handles to an in-memory db in conjunction with DBI's broken threading semantics is preventing the clean-up code being invoked.

    I'm not sure where else to go looking for knowledgeable sqlite+windows devs?

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      Aaah, ok. Misunderstanding on my part. I read you to say that unless you delete the main file, the data persists.

      --MidLifeXis

[reply]
        I read you to say that unless you delete the main file, the data persists.

        I did :) (I'm guessing were at cross-purposes here?)

        The problem is; *I* shouldn't have to delete a file, (any file), when I've asked for an in-memory db.

        If they have a need to create a file for an in-memory db -- that was a surprise -- they should delete it afterwards.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom