perleager has asked for the wisdom of the Perl Monks concerning the following question:

Hey,

I'm building a web site for this investment firm. Now the content for the website is stored in a mysql database, this is because it allows content editors to edit the content web- based.

Now all my files are going to have a .pl extentsion (i.e: index.pl), what does this do to my search engine rankings? I'm not sure if this is a 100% perl question but j/w since the html is going to be generated by perl, would search engines still include me? Or should I rather use a language like PHP to carry out the html generation?

Thanks,

Anthony

20040220 Edit by castaway: Changed title from 'Perl or PHP Web Site?'

  • Comment on Are Perl-based CGI scripts seen by search engines?

Replies are listed 'Best First'.
Re: Are Perl-based CGI scripts seen by search engines?
by pfaut (Priest) on Feb 20, 2004 at 00:40 UTC

    I've had trouble accessing .pl files on some of my sites when using Internet Explorer from a machine that had perl installed on it. The browser would occassionally try to execute whatever was returned by the CGI program because it thinks .pl means a perl program so it tries to run it. When I run into this, I rename them to .cgi. This allows me to figure out what the real problem is without the browser getting in the way (Windows doesn't know what a .cgi file is). All I have to do is put a little directive in my .htaccess file to tell the web server that .cgi files should be processed with perl.

    If you wanted to, you could modify your own web server's configuration files to change the file extension mappings. You could write your web site in perl using files with .php extensions so long as the web server thinks .php files should be processed with perl. You could also write your web site in php with .pl extensions so long as you tell your web server to process .pl files with the php interpreter. It really doesn't matter what the extension is so long as the web server knows how to map it to the right processor.

    I'll agree with the first reply in that none of this should matter at all to search engines unless they are trying to guess what language your site is written in.

    90% of every Perl application is already written. ⇒
    dragonchild
[reply]
      OT Rant of course. IE chokes on .pl files when Perl is installed on a target machine!? That's insane! I cannot believe that Microsoft still refuses to make IE respect the Content-Type header. Why do people still bother to use this piece of trash?? /rant
      Lobster Aliens Are attacking the world!
[reply]
Re: Are Perl-based CGI scripts seen by search engines?
by injunjoel (Priest) on Feb 20, 2004 at 02:40 UTC
    Greetings all,
    First I would suggest not concerning yourself with search engine placement. It is my opinion that this is a antiquated mentality left over from the dot com boom when everyone thought the world wide web would solve all of their company's marketing problems. It doesn't! "if you build it..." does not mean they will come. Companies need to understand that simply having a website will not make people want to visit it and they need a supplementary marketing plan to drive traffic to their site. Tools dont make money, using them to work more efficiently does, and you are building a tool for them. Unfortunately this is rarely understood by those in charge and so aside from a press release concerning the 'launch date' of the new site most of the marketing falls on the developer and his perceived ability to shoot sites to the top of any search at will. Do yourself a favor and concentrate your efforts on making a robust, scalable and user friendly application, because if they want people to come back to the site it needs to work.
    Not that I have any strong feelings on the subject...
    That being said, the extension on your files will not discourage nor encourage a search engine from listing your site. A quick search of Google on most anything turns up URL's that point to .php, .pl, .asp, .pdf along with html files. I have read somewhere that the ranking and placement within search results, at least for google, is based on how many other sites refer to yours, a sort of XP system for web pages if you will. Many sites also allow you to submit your site for inclusion in their database. If the client is still worried throw some meta-tags on your pages to make them happy and tell them to leave you alone.
    Now Perl vs. PHP for your project?
    My answer is use the one that you are most comfortable with since it sounds like you are writting a simple content manager. If however you or your client envision extensive expansion of this application into other protocols aside from HTTP you might want to develop it in Perl from the start. Perl is better suited for system type tasks and has been expanded to include CGI & Web related stuff, where PHP was developed with CGI & Web stuff in mind and has been extended into other areas (I think the name came from 'Personal Home Page' early on and has been redefined as 'Hypertext Pre-Processor' though Im not sure how they get PHP from that).
    HTH
[reply]
Re: Are Perl-based CGI scripts seen by search engines?
by OhReally (Monk) on Feb 20, 2004 at 11:31 UTC
    Unlike other comments I do believe that search engines may take page type into account. There was a long period for example where SEs(Search Engines) would not index URLs with parameters in them at all. Google was one of the first ones I noticed actually following links such as these. I have noticed that people who administer discussion forums complain that many of their "guests" are actually SE spiders. I felt my rankings rose when I switched from pages with parameters to more normal "looking" .shtml pages.

    Search Engine Optimisation is a whole topic of its own so I will not ramble forever and instead will point you to Webmaster World where there is an amazing accumulation of knowledge from people who are keep up with the day to day details of search engine ranking algorithms. I hope you will find much interesting reading there - I know I did.

    Of course, you don't need to have all your perl cgi scripts end with .pl there is more than one way to do this ranging from simply renaming them to using mod_rewrite to make your CGI scripts look like static page URLs. Mod_rewrite is especially handy if you have a set of scripts in place already and don't want to change them all.
[reply]
Re: Are Perl-based CGI scripts seen by search engines?
by ctilmes (Vicar) on Feb 20, 2004 at 00:29 UTC
    You can use Perl or PHP and still name your files however you please if it matters to you.

    FWIW, I don't think it makes a bit of difference to search engines.

[reply]
Re: Are Perl-based CGI scripts seen by search engines?
by asarih (Hermit) on Feb 20, 2004 at 00:33 UTC
    Designing web sites around a search engine ranking is a pure lunacy if you ask me. Google is a fad now, but it used to be Alta Vista. Are you going to redesign everything if Inktomi becomes popular?

    The choice of language is irrelevant.

[reply]
Re: Are Perl-based CGI scripts seen by search engines?
by thraxil (Prior) on Feb 20, 2004 at 16:50 UTC

    as others have said, it doesn't matter what your pages are written in. all clients should care about is the Content-Type of the results.

    that said, it is highly recommended that you make use of technology like mod_rewrite to eliminate filename extensions from urls. this will help you when, a few years down the line, you decide to rewrite your site in a different language. if you have to change urls, you'll break other sites' links (and lose PageRank with google).

    search engines are also usually a little skittish about indexing content that looks dynamically generated. this is based not on filename extensions but by parameters in the url (ie if there is a '?' with anything after it). eg, google will follow links to a page with parameters, but won't follow links from that page to other dynamic pages. this is to avoid getting the spider caught in an infinite loop (imagine if a site appended a random number to every url.) again, using something like mod_rewrite to change your urls from 'product.pl?product_id=1234' to something more like '/products/1234/' will make it much easier to index.

    anders pearson

[reply]

Back to Seekers of Perl Wisdom