Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm seeing a very weird problem in a large Catalyst-based app I have to maintain.

The logging system is set up on the application level, using Log::Dispatch::File (FileRotate, specifically); every Controller simply sets a different logfile as necessary (or disables logging entirely). The setup is exactly the same for every Controller: they inherit the same begin.

Exactly one Controller has very frequent errors: 'Caught exception in My::Controller:ThisParticularController->begin "Cannot write to '/path/to/logs/ThisParticularController.log': Input/output error at /path/to/Log/Dispach/File.pm line 101."' However, this controller also very frequently does successfully write to this logfile. No other Controller ever reports this error. This Controller is probably the busiest one, but it's not a vast difference. There are no differences in permissions for this file, and it's in the same filesystem as all the other logs.

I simply can't see any difference with how logging is handled for this Controller vs. any of the others, and I don't know what "Input/output error" actually means. What can I do to try to figure this out?

Replies are listed 'Best First'.
Re: Diagnosing "Input/output error"
by hv (Prior) on Aug 03, 2022 at 02:02 UTC

    Assuming you are using the latest version, the relevant line from Log::Dispatch::File is printing $! after getting an error from open(), so it is a C-level errno. I was not familiar with that particular error string, so I checked which one it was like this:

    perl -E 'for (1 .. 100) { $! = $_; say "$_: $!" }' | grep Input
5: Input/output error

    [download]

    A bit of (OS-specific) googling finds that that error is EIO, and I was able to confirm that with perl -MPOSIX=EIO -E 'say EIO'; the following answer on stackoverflow then gives a clue to the sort of things that might be relevant:

    In linux EIO means, that there was made an attempt to read/write to stream which is currently unavailable. This could happen because of physical error or when orphaned process (whose parent has died) attempts to get stdio from parent process, or when stream is closed.

    However none of those options make sense for the file-open happening here.

    You might get a bit further if you can trace a failure case. I'm guessing that the $mode is ">>": for me, using strace perl -E '$|=1; say "before"; open(my $fh, ">>", "/tmp/zz"); say "after"' 2>&1 | perl -nle 'print if /write.*before/ .. /write.*after/' suggests that the syscalls for a successful open might look something like:

    openat(AT_FDCWD, "/tmp/zz", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3
lseek(3, 0, SEEK_END)                   = 0
ioctl(3, TCGETS, 0x7ffdbea9a8b0)        = -1 ENOTTY (Inappropriate ioc
+tl for device)
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0

    [download]

    So if the error happens reliably enough that it makes sense to attempt, tracing just those calls (or the equivalent set on your system) would hopefully give a further clue - but here on Ubuntu none of those calls list EIO as a possible error. (I also checked perl source code in case it might be setting it directly, but that does not appear to happen.)

    Hope this helps.

[reply]
[d/l]
[select]
Re: Diagnosing "Input/output error"
by Fletch (Bishop) on Aug 03, 2022 at 02:02 UTC

    As a last resort (because doing it may involve needing root privs and/or catching a misbehaving needle run in the haystack of your normal traffic) you may have to resort to using strace (or your platform's equivalent if available) and see what's happening at the system call level. Watching what's happening "around" when open(2) gets called can provide helpful clues if you've ruled out "normal" things like permissions or running as different uids or . . .

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

[reply]
[d/l]
[select]
Re: Diagnosing "Input/output error"
by BillKSmith (Monsignor) on Aug 02, 2022 at 19:18 UTC
    As unlikely as it may be, I would not dismiss the possibility of hardware failure. At one time, I had a FORTRAN program that worked on seven of eight "identical" machines. Of course I had to fix the program to work on all eight. Years later, a hardware tech found and fixed the underlying failure.
    Bill
[reply]
Re: Diagnosing "Input/output error"
by rizzo (Curate) on Aug 02, 2022 at 21:40 UTC

    Could it be possible that a lock on the file causes this error?

[reply]

Back to Seekers of Perl Wisdom