Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Wiki size monitor.

by davido (Cardinal)
on Nov 20, 2004 at 09:16 UTC ( [id://409240]=sourcecode: print w/replies, xml ) Need Help??
Category: PerlMonks Related Scripts
Author/Contact Info davido
Description:

Really only useful to the gods, though possibly of interest to all Cabal: This is a quick way to check to see if any of the wikis are needing rotation.

My first time logging into the Monastery with WWW::Mechanize as myself (instead of just scraping anonymously), and my first time fetching Monastery pages via the xml displaytype. More straightforward than I expected. XML++.

I hope it comes in handy.

Update: Implemented demerphq's suggestion.


use strict;
use warnings;

use WWW::Mechanize;
use XML::Simple;

my $user   = '*****';
my $passwd = '*******';

my %wiki_ids = (    'Cabal'     => 322009,
                    'PMDev'     => 106868,
                    'PMDevToDo' => 222493,
                    'Port'      => 110265,
                    'Editors'   => 74283 ,
                    'SDC'       => 174180,
                    'SDC Plan'  => 240586,
                    'SCD ToDo'  => 237008,
                    'QAEditors' => 133122    );

my $max_wiki_size = 50_000;

my $agent = WWW::Mechanize->new();

$agent->env_proxy();

print "\n--- Wikis ---\n";

# Check each wiki's size.

my $login = "op=login;user=$user;passwd=$passwd;expires=+10y;";

foreach my $wiki ( sort keys %wiki_ids ) {
    $agent->get(  
          'http://www.perlmonks.org/index.pl?'
        . $login
        . "node_id=$wiki_ids{$wiki};"
        . 'displaytype=xml'
        . ';xmlstyle=clean'
    );
    $login = '';
    printf "%-10s (id=%7d): Size= ", $wiki, $wiki_ids{$wiki};
    if ( $agent->success() ) {
        my $hr_xml = XMLin( $agent->content() );
        my $size = length $hr_xml->{data}{field}{content};
        print $size,
              ( $size >= $max_wiki_size ) ? ". Rotate.\n" : ".\n";
    } else {
        print "Unable to fetch.\n"
    }
    sleep 2;
}
Replies are listed 'Best First'.
Re: Wiki size monitor.
by demerphq (Chancellor) on Nov 20, 2004 at 09:56 UTC

    IMO you should add 'xmlstyle=clean' or possibly 'xmlstyle=flat' to the argument list. The former will prevent the data in the node having newlines appended to the end, and the latter will do that and also use a different "flat" key layout that IMO makes things a little more logical.

    ---
    demerphq

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://409240]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2024-04-25 12:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found