in reply to Re: Getting File Type using Regular Expressions
in thread Getting File Type using Regular Expressions

All we have in any situation is context and convention. Intuition won't solve everything, and computational completeness won't solve everything. Perhaps the byte sequence "P100s of Samsung are really cool phones" is a perfectly well-formed 6x6 pixel GIF file. You can only guess at the intent, and more data gives a better guess. That's why they call them 'heuristics.'

That said, malicious users will attack any such heuristic assumptions to their favor. Britney.jpg.exe If your upload code expects web-intended images and only wants to accept web-intended images, it benefits the system to expect that any available heuristic passes muster. If it's not .jpg, toss it. If it's not JPEG magic, toss it. If the ImageMagic tool says the pixel dimensions are over 10000 in either dimension, toss it.

--
[ e d @ h a l l e y . c c ]

Replies are listed 'Best First'.
Re: Getting File Type using Regular Expressions
by Abigail-II (Bishop) on Apr 21, 2004 at 16:38 UTC
    Perhaps the byte sequence "P100s of Samsung are really cool phones" is a perfectly well-formed 6x6 pixel GIF file.
    No, it isn't. I didn't pick my example lightly. It's not a valid GIF or Netpbm PBM file. But even if it would be valid, it strengthens my argument, that looking at the first couple of bytes to guess the type of content is as bug-ridden as just looking at the file name.
    That said, malicious users will attack any such heuristic assumptions to their favor. Britney.jpg.exe If your upload code expects web-intended images and only wants to accept web-intended images, it benefits the system to expect that any available heuristic passes muster.
    Interesting that you bring up the web. See, the people who initially designed HTTP got it right. Documents delivered over HTTP are tagged with their type. No inspection of a fragment of the data is required. No implied type derived from the URL. Instead, a clear separation of information (the content) and the meta-information (the content type). Simple. Logical. Not error prone. Isn't that what MacOS does with its resource and data forks?

    Abigail

[reply]
      The HTTP protocol offers a MIME type. Where does the server get it? In most configurations, it guesses it. Download a RedHat Package Manager (.rpm) file, and you get application/x-realplayer-media (.rpm) mime type. Oops. Why? Because it used an imperfect heuristic scheme to specify the mime type.

      Neither the Windows filesystem nor the Unix filesystem offers typing metadata. Resource forks and Windows .rc/.res resource files offer typing metadata inside their encoded contents.

      It's just another example when you and I are in "violent agreement" over some topic. Metadata and type-determination methods are weak, and usually poorly implemented. In my case, I just suggested that two heuristics are better than one, instead of ranting that it's all futile unless you have God's Gift to Omniscient Filesystems.

      --
      [ e d @ h a l l e y . c c ]

[reply]

In Section Seekers of Perl Wisdom