in reply to Re: Storable - File empties itself?
in thread Storable - File empties itself?

>> Firstly it is not using locking
Yeah, that's right. Might this be an issue? I've never worked with this Module before.

>> secondly it performs no error checking (eg. the return values from retrieve and particularly store are never tested)
On the "Userlist" we check if the Perl-Storage File exists. Then we get all userids from the CMS and we check if they exists in the Perl-Storage. If yes show it's data otherwise we display a fixed string. 
if ( $storage->{$id} ) {
  # show userid + timestamp
} else {
  # show userid + "never logged in"
}

[download]
On the "Useraction" we only check if the Perl-Storage file exists. Since this kind of system is only adding/overwriting data and not deleting anything we would add an entry to the Perl-Storage if it's existing otherwise overwrite it.

I just read in the documentation "The routine returns undef for I/O problems or other internal error, a true value otherwise. Serious errors are propagated as a die exception." so I'll add proper error checking now.

>>it would be good to know which version of Storable your environment is using
Apache-Overview states Storable Version 2.56

Replies are listed 'Best First'.
Re^3: Storable - File empties itself?
by hippo (Archbishop) on Apr 13, 2023 at 10:35 UTC
    >> Firstly it is not using locking Yeah, that's right. Might this be an issue? I've never worked with this Module before.

    I rarely use Storable directly myself but yes it might well be an issue. I don't know the details of how Storable performs its I/O deep down but any time you might have two processes performing simultaneous I/O on the same file there is the possibility of catastrophic corruption. The very fact that the module offers functions like lock_store and lock_retrieve suggests that their use might be necessary on occasion.

    just read in the documentation "The routine returns undef for I/O problems or other internal error, a true value otherwise. Serious errors are propagated as a die exception." so I'll add proper error checking now.

    That seems like a worthwile action for sure.

    Apache-Overview states Storable Version 2.56

    I don't see that version listed anywhere but the 3.x releases started coming out in 2016 so you have a version which is quite old. It is at least worth bearing that in mind when reading newer documentation.

    🦛

[reply]
[d/l]
[select]
      Unfortunately we're still running Solaris on this specific Server which makes everything about Perl-Modules a nightmare. This is the reason we're so behind on versions. So far, thank you for all your thoughts.
[reply]

        > Unfortunately we're still running Solaris on this specific Server which makes everything about Perl-Modules a nightmare

        Does this mean you're using the Solaris-supplied version of Perl rather than building your own? I strongly agree with Fletch on this one:

        If you're doing anything serious with Perl you DO NOT want to use the OS' perl as that way lies much pain. Doing so couples you tightly to the OS' upgrade schedule for both the language and (if you're using its package manager for them) CPAN modules.

        -- Fletch in Re: Prefer Pure Perl Core Modules

        See also: Re: Replicate Perl setup (Building and Installing Perl References)

[reply]

        AFAICT lock_store has been available since Storable 1.0.2 (with a potentially relevant bugfix in Storable 2.0).

[reply]
[d/l]
Re^3: Storable - File empties itself?
by afoken (Chancellor) on Apr 13, 2023 at 20:12 UTC

    If more than one process at a time may decide to rewrite the file you use for Storable data, you either need locks, or you need to delegate the problem (e.g. to a relational database).

    For locking, see Re: Update config file parameters and flock. Note that locking can be messy if NFS is involved.

    Often it is easier not to re-invent the wheel. Just don't use files at all, put your data into a relational database. File locking, concurrent reading and writing are solved problems for relational databases. You just need to connect to the database. That's what DBI is for.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
      I just posted this: Re: Storable - File empties itself?

      Yeah you might be right. I start checking if I can confirm my thoughts. Reworking things for now is an issue based on time. I'm almost on my own in 2 projects right now and yeah ... I go small steps and check if its two processes using 1 file first and see what I'm gonna do after.

[reply]

        A 'minimal' code change solution would be to use DBI and DBD::SQLite (both install fine on Solaris btw), since you seem keen to avoid storing this in the database your CMS uses.

[reply]
Re^3: Storable - File empties itself?
by NERDVANA (Priest) on Apr 14, 2023 at 18:07 UTC
    >> Firstly it is not using locking > Yeah, that's right. Might this be an issue? I've never worked with this Module before.

    It's not really a question about this module. You have a file, and (on every page request?) you read the file, and write completely new data overtop the old file. Assuming this happens in a web controller, all you need is two users to simultaneously load a page and their controllers will be simultaneously overwriting the file. Only one update will win, and if the perfect timing happens of one controller clearing the file to begin writing it while the other controller has just started reading it, you end up with an empty file.

    There are dozens of database-in-a-file options to choose from here (and DB_File is a core perl module), but since it looks like all you want are a user ID and a timestamp, you could even just create a directory and "touch" an empty file per user id.

[reply]

In Section Seekers of Perl Wisdom