Perl MongoDB Results and Version (batch

maikelnight has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, i have some code that works pretty fine with MongoDB v0.705.0.0 :

use Data::Dumper;
use DateTime::Format::Strptime;
use POSIX qw(strftime);
use MongoDB;
use Data::Structure::Util qw( unbless );
use strict;
use warnings;

my $mongoclient = MongoDB::MongoClient->new(
    host => '127.0.0.1', port => 27017
    );

my $db = $mongoclient->get_database('database');

my $collect = $db->get_collection('collection')->aggregate([
    {'$group' => {
        '_id' => {_path => '$path' , _ip => '$IP', _time => '$TIME'},
        '_count' => { '$sum' => 1},
        '_docs' => { '$push' => '$_id' }
     }    
    },
    { '$match' => {
        '_count' => { '$gt' => 1}
    }}
]);
my $mongo_aggregate = unbless $collect;
my @out = (@$mongo_aggregate);
print Dumper @out;
[download]

I receive a few thousand results what is expected and checked in database....so far so good...On another system with MongoDB v1.2.2 i receive only 101 results:

use Data::Dumper;
use DateTime::Format::Strptime;
use POSIX qw(strftime);
use MongoDB;
use Data::Structure::Util qw( unbless );
use strict;
use warnings;

my $mongoclient = MongoDB::MongoClient->new(
    host => '127.0.0.1', port => 27017
    );

my $db = $mongoclient->get_database('database');

my $collect = $db->get_collection('collection')->aggregate([
    {'$group' => {
        '_id' => {_path => '$path' , _ip => '$IP', _time => '$TIME'},
        '_count' => { '$sum' => 1},
        '_docs' => { '$push' => '$_id' }
     }    
    },
    { '$match' => {
        '_count' => { '$gt' => 1}
    }}
]);
my $mongo_aggregate = unbless $collect->{'_docs'};
my @out = (@$mongo_aggregate);
print Dumper @out;
[download]

If i dump my $collect i found a hint that says: '_batch_size' => 101 I believe thats the point where im stucking. I dont know how to solve the issue nor i can fix that with documentation (as im advanced beginner). Can someone please shed some light on me, please. Thanks, regards,

Comment on Perl MongoDB Results and Version (batch_size??) Select or Download Code

Replies are listed 'Best First'.
Re: Perl MongoDB Results and Version (batch_size??) by poj (Abbot) on Sep 22, 2018 at 07:01 UTC
'_batch_size' => 101 Yes, see default size. You can set it in the options document after the pipeline array. `my $all_collect = $collect->aggregate([ {'$group' => { '_id' => {_path => '$path' , _ip => '$IP', _time => '$TIME'}, '_count' => { '$sum' => 1}, '_docs' => { '$push' => '$_id' } } }, { '$match' => { '_count' => { '$gt' => 1} }} ], { 'cursor' => { 'batchSize' => 10000 } } );` [download] poj	[reply] [d/l]
Re^2: Perl MongoDB Results and Version (batch_size??) by maikelnight (Sexton) on Sep 22, 2018 at 13:09 UTC
awesome mate, thanks a lot! that solved the issue! big ups!	[reply]
Re: Perl MongoDB Results and Version (batch_size??) by bangor (Monk) on Sep 21, 2018 at 22:15 UTC
The docs say that the aggregate() method returns a MongoDB::QueryResult object, so have a look at that - you could try `my @out; while ( my $doc = $collect->next ) { push(@out,$doc) }` [download]	[reply] [d/l]