You have not shown how you generate the value of $viewState, which is pertinent here. However it won't make any difference since you define the value of $param in single quotes, meaning $viewState won't be interpolated anyway.

You also are using the most inconvenient way of building and making your request. Consider using at least HTTP::Request::Common, if not something simpler such as HTTP::Tiny.

Sometimes sites are just so anti-scraping that you can't get there with LWP. This seems to be one of them. I don't think your problem is your proxy. I couldn't retrieve the page via LWP either. I am able to get search results using cURL with the same args: try running this from your command line and see what you get:

curl -L 'https://patentscope.wipo.int/search/en/search.jsf' -H 'Host: +patentscope.wipo.int' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel M +ac OS X 10.11; rv:53.0) Gecko/20100101 Firefox/53.0' -H 'Accept: text +/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Acce +pt-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://patent +scope.wipo.int/search/en/search.jsf' -H 'Content-Type: application/x- +www-form-urlencoded' -H 'Cookie: JSESSIONID=CD31D066DF8710F9FE9B5C9C5 +397A977.wapp1nC' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Req +uests: 1' --data 'simpleSearchSearchForm=simpleSearchSearchForm&simpl +eSearchSearchForm%3Aj_idt379=FP&simpleSearchSearchForm%3AfpSearch=bru +shless+motor&simpleSearchSearchForm%3AcommandSimpleFPSearch=Search&si +mpleSearchSearchForm%3Aj_idt447=workaround&javax.faces.ViewState=7923 +733300114075152%3A8489940171963107341' | grep 'Tokyo Parts'

I don't know how long the tokens will be good for. That command will probably stop working after some time.

You may have to move up to WWW::Mechanize (which handles cookies) or even Selenium::Remote::Driver (which handles JavaScript).

Hope this helps!

update:: add mention of Mech, thanks Corion...


The way forward always starts with a minimal test.

In reply to Re: Authen::Negotiate Wep-Programming by 1nickt
in thread Authorization Negotiate Wep-Programming by amitsq

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.