in reply to Re^5: threads::shared seems to kill performance (Workaround).
in thread threads::shared seems to kill performance

Is file:memdb making use of an alternate data stream? That would could explain the behavior that you are seeing. It would be interesting to explore the file named file to see if this was the case.

Update: Some other informational links (no assurance of quality on these):

--MidLifeXis

Replies are listed 'Best First'.
Re^7: threads::shared seems to kill performance (Workaround).
by BrowserUk (Patriarch) on Jul 19, 2013 at 18:50 UTC

    Great call! 

    C:\test>dir file
19/07/2013  19:49                 0 file
C:\test>streams file

Streams v1.56 - Enumerate alternate NTFS data streams
Copyright (C) 1999-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

C:\test\file:
   :memdb2?mode=memory&cache=shared:$DATA       347136

    [download]

    Doesn't explain why its not being deleted as (I think) it should.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

      I wonder (have not tested) if the ADS is being created, forcing creation of the "normal" file, and at the end, the ADS is being attempted to be deleted, but failing, leaving the data for the next run. ISTR needing to delete the main file (or perhaps some special tool besides unlink) to remove the ADS.

      --MidLifeXis

[reply]
[d/l]

        An ADS cannot exist without a main file; and unlinking the main file will always delete the ADS; so I'm not sure what you are getting at here?

        I asked the question on the sqlite mailing list; but besides getting the obvious reply -- "if the data persists it isn't an in-memory db"; Well d'uh! -- total silence. Probably there is noone with any windows knowledge on the list.

        My suspicion is:

        • When creating an in-memory db; they use directly mapped virtual address space (memmapped files) to allow the file to grow beyond available memory and be paged on and off disk.

          To do that the need to create a file to 'back' the VAS.

        • On *nix; they probably create the backing file and then immediately unlink it (whilst still holding an open handle to it) so that it disappears when the process ends.

          This mechanism of self=managing temporary files isn't available of Windows, so they've opted to use an alternate datastream. (Perhaps as an attempt to prevent any spooled data being assessible; though that would be forlorn; more.exe can find it.)

          And they forgot to delete the file afterwards.

        Another possibility is that using threads and multiple handles to an in-memory db in conjunction with DBI's broken threading semantics is preventing the clean-up code being invoked.

        I'm not sure where else to go looking for knowledgeable sqlite+windows devs?

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom