Ah! I have very little neural net experience and didn't realize the learning process was not strictly deterministic but had a random element. I see that now by running xor and xor_minimal several times sequentially: each run had a different number of learning epochs but produced the same result.

There's something that troubles me, though. The learning process seems to occasionally go on infinitely. For example, my last xor_minimal run went for 67,481 epochs before I decided to stop it. I ran xor about 8 times, all with epochs under 256, but the last time I ran it I let it go for a few minutes up to epoch 37892 before I killed it -- it got stuck on an error value of 5.30893529204799. Is it possible that the learning process even for small nets will never show an error level less than .001 depending on the random initialization?

I was awed by the implications of the ex_add example in the Mesh package. If it's possible to "teach" a system to add, subtract, or do other basic math by example with reasonable accuracy, then I have pretty high expectations for my data. So, right now, I'm trying to prove to myself that I can "teach" a net to do basic operations -- things that I can verify independently and easily. When I set it loose on my data, after the initial tweaking and verifying, it's going to get more and more expensive (in terms of manual labor) for me to verify all of the results so I'd like some confidence that I understand what it's doing up front.

Your suggested modification, removing atanh as the errorfunction, seems to work on Linux. It's on Epoch 400 with an error around .35, which has steadily been dropping from the initial error level of about 200. I'll let it run a little longer.

With that said, I've taken your xor example and modified it with the ex_add example from the Mesh package. Here's the resulting code: 

use AI::NNFlex::Backprop;
use AI::NNFlex::Dataset;

my $network = AI::NNFlex::Backprop->new(
                learningrate=>.2,
                bias=>1,
                fahlmanconstant=>0.1,
                momentum=>0.6,
                round=>1);



$network->add_layer(    nodes=>2,
            activationfunction=>"tanh");


$network->add_layer(    nodes=>2,
            activationfunction=>"tanh");

$network->add_layer(    nodes=>1,
            activationfunction=>"linear");


$network->init();

# Taken from Mesh ex_add.pl
my $dataset = AI::NNFlex::Dataset->new([
[ 1,   1   ], [ 2    ],
[ 1,   2   ], [ 3    ],
[ 2,   2   ], [ 4    ],
[ 20,  20  ], [ 40   ],
[ 50,  50  ], [ 100  ],
[ 60,  40  ], [ 100  ],
[ 100, 100 ], [ 200  ],
[ 150, 150 ], [ 300  ],
[ 500, 500 ], [ 1000 ],
[ 10,  10  ], [ 20   ],
[ 15,  15  ], [ 30   ],
[ 12,  8   ], [ 20   ],
]);

my $err = 10;
# Stop after 4096 epochs -- don't want to wait more than that
for ( my $i = 0; ($err > 0.001) && ($i < 4096); $i++ ) {
    $err = $dataset->learn($network);
    print "Epoch = $i error = $err\n";
}

foreach (@{$dataset->run($network)})
{
    foreach (@$_){print $_}
    print "\n";    
}

print "this should be 1 - ".@{$network->run([0,1])}."\n";

# foreach my $a ( 1..10 ) {
#     foreach my $b ( 1..10 ) {
#     my($ans) = $a+$b;
#     my($nnans) = @{$network->run([$a,$b])};
#     print "[$a] [$b] ans=$ans but nnans=$nnans\n" unless $ans == $nn
+ans;
#     }
# }

[download]

In reply to Re^2: NNflex problems (win32) by tlpriest
in thread NNflex problems (win32) by tlpriest

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.