Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I'm trying to screen scrape a site that is built in ASP.NET environment. I send the correct URL in POST method and get the following error page: "Validation of viewstate MAC failed. If this application is hosted by a Web Farm or cluster, ensure that <machineKey> configuration specifies the same validationKey and validation algorithm. AutoGenerate cannot be used in a cluster." I know that sending the request in GET method will solve this problem, but its to long for a GET request (about 12,400 bytes). Any ideas on how to overcome the VIEWSTATE validation error? Thank you very much.

Replies are listed 'Best First'.
Re: Validation of viewstate MAC failed
by Marshall (Canon) on Nov 26, 2009 at 12:07 UTC
    I know that sending the request in GET method will solve this problem

    No, I don't think so. It has been awhile since I accessed an MS ASP.NET website, but basically when you get the web page, it will contain a "viewstate". This is a very long string that represents what you are doing on the website, like what page you are looking at now, who are, etc. You need to use POST, not GET. When you send your POST, you need to "barf back" verbatim the latest "viewstate" that you received. When you get a reply page, you will get a new "viewstate" string. When you send the next request, you need to post this latest "viewstate".

    The basic idea of a "viewstate" is that the client keeps track of what is going on instead of the server. You as the client are acting as distributed storage for the server. For you, this string is absolutely meaningless. Just get it, save it and use it for the next request.

    Update:

    I just looked at a typical "viewstate". It is 182 lines (formatted like below, of course there are no line breaks in this string) of gibberish. You don't have to decode this, but you do have to barf it back to the server when you send a request. And you will get a new viewstate in the resulting page.

    viewstate is: "/wEPDwUKMTM0NjAxNTIwMA9kFgQCAQ9kFhACAQ8PFgIeBFRleHRlZGQCAw8PFgQfAAUHU +2l nbiBJbh4LTmF2aWdhdGVVcmwFPC9Mb2dpbi5hc3B4P1JFRkVSUkVSPS9teVVTR0JDL01lb +WJ lcnMvTWVtYmVyc0RpcmVjdG9yeS5hc3B4P2RkAgUPFgIfAAWUIDx1bCBjbGFzcz0nbmF2J +yB pZD0nbmF2MSc+PGxpPjxhIGNsYXNzPSdtZW51X3RpdGxlJyBpZD0nbGVlZF9idG4nIGhyZ +WY 9Jy9MRUVEJz5MRUVEPC9hPjx1bCBjbGFzcz0nc21lbnUnPjxsaT48YSBocmVmPSdodHRwO +i8 vd3d3LnVzZ2JjLm9yZy9EaXNwbGF5UGFnZS5hc3B4P0NNU1BhZ2VJRD0xOTg4Jz5JbnRyb +zw vYT48L2xpPjxsaT48YSBocmVmPSdodHRwOi8vd3d3LnVzZ2JjLm9yZy9EaXNwbGF5UGFnZ +S5 ... total of 182 lines like this .... "
      Hello,
      Thank you,
      I am taking the VIEWSTATE and the EVENTVALIDATION parameters to build the URL. When I copy_paste this very long URL to the Mozilla FireFox window (meaning that the URL is sent in GET method), it leads to the correct results page. But I can't send it in GET method from the command line in my server, using the basic Perl modules. When I try to do so I get the error:
      Warning::(ml_lwp_wrapper): short write
      I don't think I can change the GET request length limitation of the basic Perl modules.
      So I'm trying to understand if I can overcome the VIEWSTATE validation error that I get, when I send the long URL in POST method.
        Not quite sure what you are doing. "But I can't send it in GET method from the command line in my server" A VIEWSTATE can be HUGE, even 5KB or 10KB! Your command line buffer is probably a lot smaller than that! Maybe 1KB? Anyway I figure that VIEWSTATE is just too long to pass around as a command line parameter..it could just get truncated independent of what Perl can or cannot do. Past that, the VIEWSTATE is in general a dynamic thing that changes. I've not seen something that passes this around as a command line parm, hence my confusion. Yes, Post method can send a humongous thing.

        I would suggest that you post some code. Make the example as simple as possible to illustrate the behavior.

Re: Validation of viewstate MAC failed
by Your Mother (Archbishop) on Nov 26, 2009 at 06:57 UTC

    The first result on Google -- Searching on "Validation of viewstate MAC failed" with the quotes -- suggests:

    Remove the action attribute, or change it to post to the right page if you want to have viewstate enabled. -If broken it is, fix it you should