Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: Searching the monastery with duckduckgo leads to ugly results

by Bod (Parson)
on Nov 01, 2021 at 19:40 UTC ( [id://11138311]=note: print w/replies, xml ) Need Help??


in reply to Searching the monastery with duckduckgo leads to ugly results

I use Google for searching and regularly it returns the bare version of pages for Perl Monks

Replies are listed 'Best First'.
Re^2: Searching the monastery with duckduckgo leads to ugly results (patch)
by LanX (Saint) on Nov 01, 2021 at 20:36 UTC
    OK I provided a patch to the "bare general container" and added a noindex meta-tag in the head. °

    After application, this should be effective within days. If successful this could be extended to other page paths like mobile/ and displaytypes like print

    The gods are still free to adjust the main robots.txt then, to additionally reduce the load by spiders. :)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

    °) applied 2021-11-02

      > noindex meta-tag in the head

      unfortunately this didn't have the expected outcome.

      bare is still shown in the search results, only these pages are not cached by the search engines anymore.

      We need to adjust the robots.txt, and this can only be done by a god.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

      according to DDG's website they are getting most of their results from Bing, which in turn is respecting noindex and robots.txt

      Search results look indeed similar (lot's of bare ATM)

      -> https://www.bing.com/search?q=schwartzian+transform+perlmonks

      So solving the problem for Bing should solve it for DDG too.

      The top result was cashed by bing in 2021-10-05, so updating may take a while.

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11138311]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (2)
As of 2024-04-26 03:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found