leocharre has asked for the wisdom of the Perl Monks concerning the following question:

I want to accept user uploads via the web. This is on https, apache, linux.

Should I first accept the file into an "isolation area" and inspect it before I let it go where I want it to? Is that too paranoid

2006-02-28 Retitled by planetscape, as per Monastery guidelines
Original title: 'accepting user files online'

Replies are listed 'Best First'.
Re: (OT) accepting user files online
by Fletch (Bishop) on Feb 27, 2006 at 20:50 UTC

    When dealing with web security you can never be too paranoid (that's what THEY want you to think; but perhaps I've said too much . . .).

    That said, it really depends on what kind of files they are and what you're doing with them. Image files of some sort (JPGs, PNGs), probably not an issue. HTML that could possibly contain malicious Javascript that you're going to display to other users, that you might go over more carefully.

    Update: Oop, good point about the PNG buffer overflow; good thing I weaseled and said "probably". :)

      Image files of some sort (JPGs, PNGs), probably not an issue.

      Because, of course, we know that there are never occasions where a buffer overflow in an image processing library affects common client web browsers. That being said, I'm not sure how exactly one checks for stuff like this when the underlying libraries one wishes to use may contain unknown buffer overflows or other exploit-enabling coding errors themselves.

      --
      @/=map{[/./g]}qw/.h_nJ Xapou cets krht ele_ r_ra/; map{y/X_/\n /;print}map{pop@$_}@/for@/
Re: (OT) accepting user files online
by blue_cowdawg (Monsignor) on Feb 27, 2006 at 21:31 UTC
        I want to accept user uploads via the web. This is on https, apache, linux.

    Dear Fellow Monk,
    The first question that comes to mind is "what kind of files?" and the second that comes to mind is "for what purpose?"

    In other words, analyze the requirement thoroughly

    Next you want to assess the "threat level" and what you are trying to protect. This is not always just the server you are doing this work on either. If you are for instance having folks upload resumes to your server then you have a responsiblity to protect those resumes and the private data contained therein.

    Another example of a responsibility (liability?) you are taking on with something like this is if you are having folks upload program files you need to protect against copyright infringement, virii, trojans, et. al. not only for the health of your server but to protect other users of your web site.

    With that in mind, yes of course, stage the uploaded files into a "quarantene" until you are completely satisfied that they present minimal threat. (Notice I didn't say "represent any threat")

    Another step I would take is to log all uploads with information regarding where the files came from. Preferably you want to use some sort of login authentication before allowing an individual to upload anything so you can possibly tie an upload to an individual for accountability sake.

    Just a few thoughts that come to mind.


    Peter L. Berghold -- Unix Professional
    Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg
Re: (OT) accepting user files online
by inman (Curate) on Feb 28, 2006 at 08:45 UTC
    It's not paranoid at all. It's perfectly sensible. In addition to techno nasties mentioned earlier, your website may be open to people posting adverts, porn, links to undesirable material etc. At one level, this is a pain. At another level, the material may be illegal.

    It all very much depends on what you are trying to accept and the scale of your problem. If you are running a small newsgroup, you can isolate content before letting it go or provide a mechanism for users to report content that they don't like. You can then operate an auto-exclusion rule when three people object to the content. If you are setting up the next version of Flickr then you will need to add some technology into the mix.

    You can automate the approval of most text content if you are able to compare it to examine it. The problem is that humans are endlessly inventive when it comes to spellings of Ciali5 etc.