this post is a follow up of Module design for loadable external modules containing data but, after I squeezed my mind to spot the core of the problem and after finding some valid approach, I think it is worth a new Meditation.

A dark corner where the problem raise

In the past when I found myself, facing an error or an unexpected behaviour, thinking: "it is perl or it is me?" it was always my fault. This time maybe not. Or maybe this is a problem in the way I imagine new things and perl does not make it easy as usual. What I mean is: modules are intended as an abstraction of behaviours and scripts are generally the incarnation of these behaviours in the real life. A module can be loaded and can export subs or methods. A module can be tested because generally it does nothing but exporting abstract behaviours. Only in few rare cases a module exposes data and if so it is just some bare package variable intended to modify its internal behaviour like $Data::Dumper::Indent and even this simple use can be accomplished in other, nowadays preferred, ways like providing parameters to the constructor and providing accessors for these kind of things.

What happens if I need a new design pattern? I still need an abstraction, obviously, but I also need a serie of incarnations to be loaded indipendently upon request. These are not plugins that extend the main module with new functionalities: they are different incarnations of a mother abstraction.

In the Perl::Teacher example I need the abstraction of Perl::Teacher::Lesson but then I need the incarnation of Perl::Teacher::Lesson::first_lesson and Perl::Teacher::Lesson::second_lesson and so on.

Another project of mine ( my mad and fun Game::Term :) is stucked exactly for the very same reason. Infact I started to code ignoring the above problem and I have designed it to have incarnations (game scenarios in this case) as standalone scripts and this approach lead me to shell out when changing the current scenario, messing the whole thing (shelling out it is always a bad thing and behaves very differently on different platforms like linux and windows).

A note about data: in the current post and in the previous one with data I intend not static data but a possible longish serie of perl data as others objects, anonymous subroutines, mixed with some (few) more static fragments as texts and questions.

Abstraction/Incarnation and OO roles

I was suggested to use a role for this. Honestly I'm not a big fan of OO perl frameworks, or better saying it, I will be a big fan when I will have the need to use all the feautures they provide.
I understand a role as a trasversal behaviour applicable to different kinds of objects, traversing the simple schema of father-children inheritance. The classic example of the breakble role can be consumed by very different classes of objects like bones, cars, cookies.. But I have many incarnations to one and only one abstraction. So no transversal roles to compose. Well.. I can transform my Perl::Teacher::Lesson into a role, let say Perl::Teacher::Teachable but I cannot see any advantage over a simple inheritance.

A new design pattern?

What I imagined is an orchestrator, a super-object (in this case of the Perl::Teacher class, but the same is valid for the Game::Term project) instantiated inside a perlteacher.pl program shipped within the main distribution.

This super-object will be able to do many things related to all teaching activities (reading configuration file, interact with the user..) and its Perl::Teacher distribution will include the abstraction of what a course, a lesson, a talk, a question and a test are. But the main activity will be to load a course and its lessons in sequence.

Real courses are just containers of real lessons and are shipped separetly from the main distribution. Here is the new design pattern I see.

If you look at solution I provide below, you will see I prefere to use the constructor of the incarnation module to ship the meat to the super object. This pattern is vaguely similar to routes in modern web frameworks: fatty subroutines of behaviours and data. Well... in web programming we were told to separate the logic from the presentation and this sounds sane. I think my case is a bit different because I have mainly perl data (objects of other classes like Perl::Teacher::Test or Perl::Teacher::Question filling a lesson) in my incarnations. There is not static data to serve (as templates of html in the case of web programming) and for this reason I dislike the idea to have external, yaml or json, files containing the data to be served. Infact to use static external files I have to strictly describe their format and I loose the flexibility of a perl module (for example a lesson can provide a special kind of test defining a custom sub or loading another module).

Here I present some sketch of my approaches, for sake of semplicity not in seprated .pm files but I think you will get an idea.

Possible approaches

First option: use the constructor of the incarnation module

This is my preferred one.
The abstraction module ( My::Lesson in this example ) defines methods usable by its children. Children (incarnation modules) use the new constructor to fill in the object with all the data it needs. Filling the object is done using methods provided by the abstraction module My::Lesson

use strict;
use warnings;

# ABSTRACTION
package My::Lesson;
    
    sub new{
        my $class = shift;
        return bless { steps => [] }, $class;
    }
    
    sub add_step{
        my $self = shift;
        push @{ $self->{ steps }}, @_;
    }



# INCARNATION
package My::Lesson::Example;
    our @ISA = ( 'My::Lesson' );
    sub new{
        my $class = shift;
        my $self = $class->SUPER::new;
        $self->add_step('one','two');
        
        return $self;
    }

# USAGE
# Please note that this is only an example of usage. The real one will
+ be done by an object of the Perl::Teacher class, like in
#
# $teacher->current_lesson( My::Lesson::Example->new )
#
# or something similar
package main;

my $lesson = My::Lesson::Example->new;
[download]

Second option: using a bare EXPORT in the incarnation module to provide data

The abstraction module does not need to provide methods to its children. The abstraction module becomes an incarnation loading a data structure from My::Lesson::Example

# INCARNATION
package My::Lesson::Example;

    require Exporter;
    our @ISA = qw( Exporter );
    our @EXPORT = qw( @steps );
    
    my @steps = ( 'three', 'four' );


# ABSTRACTION
package My::Lesson;

    use Module::Load 'autoload';
    # or CPAN Module::Runtime
    
    sub new{
        my $class = shift;
        my %opts = @_;
        
        # here must be some logic to load the appropriate incarnation 
+module.
        # paying attention to the @steps array (?)
        #
        # autoload from Module::Load should be the right tool (not tes
+ted)
        # autoload $opts{lesson};
        warn "autoloading $opts{lesson}";
        
        return bless { steps => [ @steps ] }, $class;
    }
    
# USAGE    
package main;

my $lesson = My::Lesson->new( lesson => 'My::Lesson::Example' );
[download]

Note: first and second approach can be mixed letting the incarnation module authors to use the interface they prefere. Something like this should be enough:

# ABSTRACTION
package My::Lesson;

    use Module::Load 'autoload';
    
    sub new{
        my $class = shift;
        my %opts = @_;
        
        if ( $class->isa( 'My::Lesson' ) ){
            return bless { steps => [] }, $class;
        }
        elsif ( $class->isa( 'Exporter' ) ){
            # autoload $opts{lesson};
            warn "autoloading $opts{lesson}";
            return bless { steps => [ @steps ] }, $class;
        }
        else{
            die "incarnation method not recognized for class [$class]!
+";
        }        
    }
[download]

third option: static data

File::ShareDir::Install allows you to install read-only data files from a distribution and File::ShareDir will be able to retrieve them. For reasons explained above I dont intend to use this solution for the Perl::Teacher project, but can be a viable solution, for example, to provide static maps to my scenarios in the Game-Term project. Thanks to kcott to show these modules. Anyway I dont see any advice against using __DATA__ in a module (if it is not a enormous amount of data): it will be accessible from within the module itself or from outside specifying the package name as described in Special Literals (taking care to close the filehandle when finished).

Conclusions

It seems very weird to me that this design problem does not occured before. As always there is the possibility that it is me :)
I tend to imagine very complex designs with a super-object able to rule and load a plethora of incarnations. In a ideal world my perlteacher.pl will consists of a mere use Perl::Teacher; my $teacher = Perl::Teacher->new(); $teacher->run;

The ability to switch from an incarnation to another, at runtime, in Perl is only available through modules. So I need an easy way to provide this mechanism and a clear interface to propose to incarnation modules authors, if any.

I think my preferred solution, providing the meat in the incarnation module constructor, it is not so nice as interface for eventual authors but is the more perlish one I found at the moment.

Any comment, suggestion, inspiration will be welcome.

Thanks for reading.

L*

Learning difficult topics like programming languages, human languages, or the thought processes of the opposite gender can be challenging. Over time, too much challenge can be discouraging and can end the effort before success is achieved.

In my own learning efforts I have found great success in mapping out the core topics for my next level. Doing so lets me focus on what I need right now and avoid stress about not learning things beyond my current level goal. In Perl, for example, I should really know the syntax and uses of hash references before I dig into object oriented web frameworks. While the idea seems simple, and it is, it does not seem common. For me it has helped in everything from programming languages to things like human languages, shooting sports, writing fiction, and searching for new jobs. Pretty much every aspect of my life has been helped by making a good plan for the next step and ignoring everything else. Except for the "opposite gender" bit, I'm still working on that one...

Here's my learning plan for becoming a well rounded Perl Apprentice. The goal is to have the basics down solidly so that I can begin my journeymanship. Until these topics are firmly embedded in my head and skills, I do not need to be trying to do Expert Perl stuff.

Data types and sigils

• scalar, list, aray, hash, subrouting, references, objects
• Variable assignment and Scope
• Context
• Autovivication

Control Structures

• if, for, foreach, until, while, ...

Pattern Matching

• Just the basics

Files

• reading and writing
• tests and operations
• Directories

Modules

• Loading and using Standard Library modules
• Using CPAN

POD

• Documenting code

Testing

• Writing and using basic tests
• Debugging

CLI

• Input and output

Process Management

Wow! I am truly humbled and indebted to all of you who took the time to help me out on my earlier questions. This was my first post here so I was a bit hesitant as I only joined a few weeks ago and wanted to ensure I did my homework before posting. Apologies if I was not able to respond to everyone’s replies but I feel that I must digest everything you’ve provided.

(I'm posting this here because I was getting embarrassed by how long my other thread was getting.)

I’m still a novice programmer and even more of an apprentice when it comes to Perl. Reading through the documentation, my head is starting to spin. My question originated because I was trying to come up with my own subroutine to determine permutations of elements in a array and I was getting lost in how to structure the subroutine using recursion (rest assured, questions on that topic are forthcoming!) But it appears to me that I still have more of the fundamentals to learn.

If I’m honest, some of the shine of Perl is starting to fade but the quality and quantity of help I have received here are motivating me to continue!

I don’t know and it doesn’t really matter. Some may ask why I ask. Because of by chance I «discovered» this. Very nice. See also FFI::Platypus.

  link: https://github.com/Baseband-processor/Air-Wireless
[download]

Recently I have released a new perl library: RTE::Dump

if anyone wants to test or contribute to the project let me know :P

Regards

Edoardo Mantovani, 2020

https://github.com/Baseband-processor/Air-Lorcon2

If interested in participating everyone can write an e-mail to: Baseband@cpan.org

Also suggestions, opinions and feedback are well accepted!

Thanks for every help from this site!

Edoardo Mantovani, 2020

There are streaming modules on CPAN that support gzip and Bzip2 and many more based on PerlIO::via, all using syntax like:

  # compress
  open  my $fh, ">:via(XXZ)", "test.xxz";
  print $fh $text;

  # decompress
  open  my $fh, "<:via(XXZ)", "test.xxz";
  while (<$fh>) {
      ...
      }
[download]

where XXZ is the required PerlIO::via::XXZ (gzip, Bzip2, ...) and .xxz is the extension that goes with it (.gz, .bz2, ...).

I needed this for xz-(de)compression, but it wasn't there (yet), so I stole from existing modules and whipped up PerlIO::via::xz, which works for what I want.

BUT, it fails in a scope where $/ = undef;, and I have no tuits to investigate.

This has not (yet) been put to CPAN, and to be honest, I don't feel like it (yet).

If anyone thinks this is a good thing to have and wants a kickstart in getting something ready to distribute, please contact me: you're welcome to grab it a use it to make it *your* (first) distribution.

Introduction

This is the follow up of Perl Automateaching -- part 1: brainstorming so read it first to have an idea of my intentions even if the pseudocode presented there is not what I currently plan.

I have choosen the name for this project and it will be Perl::Teacher as it is clear and explicative.

This post is a mere proof of concept about Perl teaching and, yes! it can be done! I'd like to be billionaire to hire super skilled perl geeks to develop my idea... but let's say they are all busy at the moment :) so the pupil (discipulus in Latin) will squeeze his brain and will dress teacher dresses. Contributors are welcome!

In the final form Perl::Teacher will be document oriented, ie: it will analyze perl programs wrote by the pupil in physical files. But in the current proof of concepts various student's attempts are hardcoded into the below program contained in scalars from $work_01 to $work_n and with a $solution_code

Also the final form of Perl::Teacher will be a bit interactive presenting and reviewing assignements and telling small lessons, but for the moment nothing of this is done.

So running the below program you will see a serie of attempts to satisfy the assignemnt and results of tests applied to provided code fragments.

Modify the $debug variable to 1 or 2 to see much more messages.

Proof of concept

Here my efforts up now (Ignore the warning you'll receive: Having more than one /x regexp modifier is deprecated at .../perl5.24-64b/perl/site/lib/Perl/Critic/Policy/ValuesAndExpressions/RequireInterpolationOfMetachars.pm line 110. beacuse it is a problem of Perl::Critic itself: see resolved issue on github)

Implementation (current)

As you can see there is a lot PPI stuff but not exclusively. Tests are execuded in order from 001 to 00n and if a test fails the current mini program is rejected.

Each test can contain different steps, the first one being the optional run that simply sends the current code to a sub: this preliminary, optional test passes if the sub returns 1 and fails otherwise. Here it is used only to check if the program compiles ( see below for future ideas ).

The second step of a test is select_child_of and it expects a PPI class name and a serie of subtests. Each PPI element of the specified PPI class, for example PPI::Statement::Variable (a variable declaration) will be processed to see if they contains PPI elemnts which satisfy all subtests. The first PPI element passing all subtests is returned by select_child_of and becomes a candidate for further inspections.

Infact if evaluate_to is also specified, the current PPI element is, take a deep breath, keep calm, string evaluated to see if it holds the wanted value. And hic sunt leones or here are dragons because eval only returns the last statement value. Search the code above for the string dirty trick to see my workaround. For me it is a genial solution, but wait, I'm the guy who string eval'ed entire CPAN.. :) so improvements are warmly welcome.

This form of testing is a proof of concepts: is not the final form of the testing framework needed by Perl::Teacher

When a miniprogram passes all tests it is evaluated by Perl::Critic to give more hints to the student. Eventual policy violations will not make the program to be marked as wrong, but are just presented as suggestions.

A note about flexibilty: looking carefully at the assignement you will notice that @letters and $result are constraints. Not the same for the intermediate scalars containing the first element and the last one.

Implementation (future)

module design

In my idea the course creator will publish Perl::Teacher::Course::EN::BasicVariables or Perl::Teacher::Course::IT::RegexIntroduzione all being child of the main Perl::Teacher::Course class. These courses have to be pluggable to the $teacher object ( Module::Pluggable probably but I have to investigate it further)

Each course will contain a serie of lessons published a sub modules, as in Perl::Teacher::Course::EN::BasicVariables::01_strings , ..::02_lists etc.

Yes I know: very long names.. but this will ensure a clarity of intent and of usage, in my opinion.

( update October 14 2020 see the related question Module design for loadable external modules containing data )

lessons

Each lesson will contain an ordered serie of optional elements: zero one or more assignement, multiple test elements possibly interleaved by one or more discourse and direct question.

So a possible flow can be:

Suggestions on module design are warmly welcome, but i want to keep it as simple as possible, not spawning objects for everything.

tests

Tests presented in the above code are too semplicistics to cover each teaching activity. I need beside positive tests also negative ones for example to prevent the use of modules, or all modules but one, to prevent external program execution and so on. Theese tests will be quite on success and will emit messages only on failure: "dont do this!".

I can use Test::Script to add tests about correct overall syntax check, behaviour of STDOUT and STDERR given different arguments and so on.

Then Perl::Teacher will provide its own tests like ones presented above: evaluate_to ( evaluate_at is probably a better name as it eval the code at a certain line), is dirty but it seems to me a viable option not so risky given the super small and controlled environment. I also plan a method named evaluate_subs which will grab al subs to test them.

I have to mix all this features in a clean and easy to use interface. Suggetions are welcome.

student interaction

During a lesson the student must have the possibility to review the current assignement, to receive hints and be pointed to relevant documentation. Part of this is roughly done in the presented code using hints and docs embedded in tests. Can be and must be improved.

I like to add a TIMTOWTDI discourse at the end of each lesson showing more ways to accomplish, even if not in the very same way, the assignement.

Every output, comprensive of examined code, errors and hints, emitted during 03_array_manipulation must be saved into a 03_array_manipulation.history file so that the student can review the whole lesson including errors commited and pitfalls and the solution alongside different approaches to the same problem. Passing the time this becomes a good source of knoweledge.

further ideas

Testing standalone scripts is an idea haunting me since years. Modulino is an approach. I can be mad enough to take the original PDOM of a given program, then save all subs and use PPI method prune to delete them from the PDOM, then wrap the rest into a new main_original_program sub, add it to a new PDOM along with all previously saved subs. Then I could do the obtained file and test it nicely. A lot of cut 'n paste and probably error prone, but can be a path to explore.

I'd like also my Perl::Teacher to be as much possible input/output agnostic: implement a way to interact with the console leaving open the possibility to be used by a web interface too: how to do this?

I'd like to ear your opinions about this project, sugesstions on module design and implementation of its parts, comments to the above proof concepts and everything you want to share.

L*

I've had a long ride of negativity over the past few years, and I haven't coped very well.

I want to thank all the Perlers (Monks and external). My life was saved.

My tears can't begin to describe appreciation I have for the outpouring of care you've shown, once again.

So today for the third episode of this diary, and after Diary of a Zydeco experiment - E02 - History, I would like to share a nice error message that I encountered while working on the modeling of my project. It can look strange to be happy about encountering an error message, but this one made my life really easier. It is still not about Zydeco directly but about one of the stable technology it relies on. Please don't look too deeply at the examples, they are mostly sarcastic but not very well structured on the OO side for now.

So we have this role, that summarize very naively what a Buyer can do:

    # lib/Wildlife/Behavior/Buyer.zydeco.pm
    role Buyer {
      requires money;
      method acquire ( Num $price ) {
        say "I bought !";
      }

      method sale ( Num $price ) {
        say "I sold !";
      }
    }
[download]

    # lib/Place/Behavior/Exhibit.zydeco.pm
    role Exhibit {
      has exhibition ( type => ArrayRef );
      has artist ( type => ArrayRef );
      has artwork ( type => ArrayRef )
      method display {
        say "Shoooow";
      }
    }
[download]

    # lib/Art.pm
    package Art {

      use Zydeco;

      class Place {
        has space;
        include Place::Behavior::Exhibit;
        include Wildlife::Behavior::Buyer;

        class Gallery with Exhibit, Buyer {
          has artwork (  type => ArrayRef );
          has artist (  type => ArrayRef );
          has event ( type => ArrayRef );
          has owner;
          has public;
        }
      }
    }
[download]

    # t/gallery.t
    use v5.16;
    use Test::More;
    use Art;

    my $gallery = Art->new_gallery(
      space => 1000,
      exhibitions => [ "Foo", "Bar" ],
      owner => "Arty Person",
      money => 10_000_000 );

    ok $gallery->does('Art::Exhibit'), 'Gallery does role Exhibit';
    ok $gallery->exhibitions, 'Gallery got an exhibitions attribute';
    ok $gallery->owner, 'Gallery got an owner';

    can_ok $gallery, 'acquire';
    can_ok $gallery, 'sale';
[download]

    smonff@padi:~/projects/Art-World$ prove -lv t/15_gallery.t
    t/15_gallery.t .. Can't apply Art::Buyer to Art::Gallery - missing
+ money at /home/smonff/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl
+/5.32.0/Moo/Role.pm line 307.
    BEGIN failed--compilation aborted at (eval 269) line 1.
     at /home/smonff/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.
+32.0/B/Hooks/EndOfScope/XS.pm line 26.
    Compilation failed in require at t/15_gallery.t line 3.
    BEGIN failed--compilation aborted at t/15_gallery.t line 3.
    Dubious, test returned 2 (wstat 512, 0x200)
    No subtests run

    Test Summary Report
    -------------------
    t/15_gallery.t (Wstat: 512 Tests: 0 Failed: 0)
      Non-zero exit status: 2
      Parse errors: No plan found in TAP output
    Files=1, Tests=0,  2 wallclock secs ( 0.03 usr  0.00 sys +  2.00 c
+usr  0.10 csys =  2.13 CPU)
    Result: FAIL
[download]

The error message is launched by a croak call in the _check_requires() method of Role::Tiny.

    croak "Can't apply ${name} to ${to} - missing ".join(', ', @requir
+es_fail);
[download]

Once we add the right attribut to the Gallery class, everything goes well:

    class Gallery with Exhibit, Buyer {

      has artwork (  type => ArrayRef );
      has artist (  type => ArrayRef );
      has event ( type => ArrayRef );
      has owner;
      has public;
      has money;
      ...
  }
[download]

  smonff@padi:~/projects/Art-World$ prove -lv t/15_gallery.t
  t/15_gallery.t ..
  ok 1 - use Art;
  ok 2 - Gallery does role Exhibit
  ok 3 - Gallery got an exhibition attribute
  ok 4 - Gallery got an owner
  ok 5 - Art::Gallery->can('acquire')
  ok 6 - Art::Gallery->can('serve')
  ok 7 - Art::Gallery->can('sale')
  1..7
  ok
  All tests successful.
  Files=1, Tests=7,  2 wallclock secs ( 0.03 usr  0.00 sys +  2.30 cus
+r  0.16 csys =  2.49 CPU)
  Result: PASS
[download]

What I found amazing is how the error message makes sense. It just tells what should be done to fix the problem: adding a money attribute to the Gallery class. But more than that, it have a deeper meaning, that is exactly the point og this project. I mean you wouldn't have an art gallery with zero money would you? This is what I call efficient and reliable object oriented programming thanks Zydeco making a great use of the stable technologies it is built on (like Role::Tiny).

So far, my overall use of Zydeco is very satisfactory during the application modeling phase. Some could say that I could draw some class diagrams and not coding, but the Zydeco use is so easy and non-verbose that it really make possible to focus ont the modeling and not on the coding: focusing on listing attributes, methods, roles and their relationship, not on the implementation.

Next episode is mostly news from the brand new 2021 year...