Be careful with opening files from CGI parameters. collection=../../../../../../../home/aleena/passwords would reveal a bit more than you intended ("passwords.txt" in your home directory).

Thank you for bringing this to my attention. If I ever store passwords somewhere, I will make sure to keep them out of the $collections_dir. I would probably put them in some deep dark corner of my directory structure with a name that does not look anything like the word 'password' and use something more secure than a plain text file.

I could also add / and . to the encode entities to make sure that the string that the cgi param returns will not recurse. Adding those would make your string return the following.

../../../../..
+/../../home/aleena/passwords
[download]

I will give that serious thought. Again, thank you.

Update: Forward slashes will be html encoded.

my $collection       = $cgi->param('collection') ? encode_entities($cg
+i->param('collection'),'/<>"') : undef;
[download]

My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.8.8 on web host.

Version control is a non-issue, I do not use it.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!

Lady Aleena

If it were me, I'd use the list of valid collections that you already have (assuming I understand your data correctly). Something like:

# Note: Still has underscsores:
my @collections = map { $_ =~ s/\.txt//; $_ }
                  grep {/^\p{uppercase}/}
                  @collections_list;

# $collection guaranteed to match one of your file names.
my $collection_raw = $cgi->param('collection');
my ($collection)   = grep { $_ eq $collection_raw } @collections;
$collection      //= 'Deafult';
# If your file names might have <>", you can still do this:
$collection        = encode_entities($collection);

# Without underscores
my @collection_labels = map { $_ =~ s/_/ /g; $_ } @collections;
[download]

Generally, it is safest to do something like this before using $collections for anything.

Good Day,
    Dean

How is that a problem? You seem to still be storing one file per page, just with a .txt suffix instead of a .pl suffix, unless I misunderstand. Another option is to ensure that the list of valid collections is stored in some recognizable way at each index, and have the menu builder parse the indexes.

Here is the difference:

./Collections            # is where scripts live and is where base_men
+u looks for menu items
./files/data/Collections # is where data lives and is not where base_m
+enu looks for menu items
[download]

Taking the data outside of individual scripts means I need to figure out how I am going to rewrite base_menu to find items to add to my site menu that will work for all the index files (see below).

./Collections/index.pl
./Fandom/index.pl
./Fandom/Crossovers/index.pl
./Fandom/Crossovers/Big_fake_companies/index.pl
./Fandom/Crossovers/Westphall_crossovers/index.pl
./Fandom/Fictional_family_trees/index.pl
./Fandom/The_Riftwar/index.pl
./Fandom/Xanth/index.pl
./Miscellany/index.pl
./Miscellany/Geeky_thoughts/index.pl
./Miscellany/Political_opinions/index.pl
./Movies/index.pl
./Role_playing/index.pl
./Role_playing/Locations/Afma/index.pl
./Role_playing/Locations/index.pl
./Role_playing/Magic_items/index.pl
./Role_playing/Miscellany/index.pl
./Role_playing/Monsters/index.pl
./Role_playing/Player_characters/index.pl
./Role_playing/Reference_tables/index.pl
./Writing/index.pl
./Writing/Erotic_fiction/index.pl
./Writing/Fan_fiction/index.pl
./Writing/Poetry/index.pl
[download]

The data for those indexes will be in the following.

./files/data/Collections/
./files/data/Fandom/
./files/data/Fandom/Crossovers/
./files/data/Fandom/Crossovers/Big_fake_companies/
./files/data/Fandom/Crossovers/Westphall_crossovers/
./files/data/Fandom/Fictional_family_trees/
./files/data/Fandom/The_Riftwar/
./files/data/Fandom/Xanth/
./files/data/Miscellany/
./files/data/Miscellany/Geeky_thoughts/
./files/data/Miscellany/Political_opinions/
./files/data/Movies/
./files/data/Role_playing/
./files/data/Role_playing/Locations/
./files/data/Role_playing/Locations/Afma/
./files/data/Role_playing/Magic_items/
./files/data/Role_playing/Miscellany/
./files/data/Role_playing/Monsters/
./files/data/Role_playing/Player_characters/
./files/data/Role_playing/Reference_tables/
./files/data/Writing/
./files/data/Writing/Erotic_fiction/
./files/data/Writing/Fan_fiction/
./files/data/Writing/Poetry/
[download]

This is what I am grappling with in my head.

My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.8.8 on web host.

Version control is a non-issue, I do not use it.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!

Lady Aleena

For each index.pl in the equivalent of find . -name index.pl, try ($dir = $indexfile) =~ s!^[.]/!./files/data/! and glob $dir.'*.txt' (obviously untested) as a starting point, if the files in those directories correspond to menu items.

use lib '../files/lib';
[download]

For example, index.pl becomes:

use strict;
use warnings;
package site::index;

our $DATA = q{Welcome to Lady Aleena's B<collections>, which is lists 
+of A<novels|href="Fiction.pl">, A<books|href="Non-fiction.pl">, A<mus
+ic|href="Music_and_comedy.pl">, A<movies|href="Movies.pl">, A<tie-ins
+|href="Tie-ins.pl">, and A<programs|href="Programs.pl"> I am willing 
+to admit I own or use. The list of movies here is just those movies I
+ own and should not to be confused with my more general interest in A
+<movies|href="../Movies">. Tie-ins are books and music connected to m
+ovies or television series. I also share my A<fandom|href="../Fandom"
+> elsewhere.
Here is a key for the notations after each title with the exception of
+ programs.};
1;
[download]

use strict;
use warnings;
package site::collection::Fiction;

our $DATA = q{This is my B<fiction collection> of SPAN<hardcovers|^har
+dcovers^>, SPAN<trade paperbacks|^trades^>, and SPAN<mass market pape
+rbacks|^massmarkets^>.}
1;
[download]

Then you'd register these modules in your main driver, and based on whatever it is that you're dispatching on you could use the appropriate full package name to get the content via $site::index::DATA, $site::collection::Fiction::DATA, etc. I suggest this because it seems you like this style for its maintenance ease, but not for it's Perl sophistication. The solution above keeps the maintenance overhead similar to now, but is more sophisticated Perl by allowing you to eliminate the repeated calling code and implement some dispatching. And you can even keep the same .pl file names and directory structure provided you require them (which you'd have to anyway.

I looked at FindBin and can not tell what purpose it serves. It seems I would still need to know the relative path to my lib, so why add this to it?

And so you know, the __DATA__ I put in the original post was just first paragraphs. Some __DATA__ fields are hundreds of lines long. So the modules you are suggesting would be tidier with __DATA__ fields and exporting those fields as a glob (faux file handle). I am also not sure that use statements could be put into loops.

Read more... (438 Bytes)

However, I may be misunderstanding what you are suggesting. The text still needs extensive munging any way I do it.

My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.8.8 on web host.

Version control is a non-issue, I do not use it.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!

Lady Aleena

No.

Keep all your files.

But keep your data and your code separate.

Remove the code from your scripts because it doesn't belong. Remove everything before __DATA__.

Rename your .pl templates to .lat or .dat

Collapse any other page specific code needed into subroutines inside a module.

This is all that should be in your master script

use lib ...;
use MyStuff;
MyStuff::run_cgi( '/path/to/root/or/config/if/nonstandard' );
[download]

Eventually get inspired by https://metacpan.org/release/Template-TT2Site/tt2site and make the dynamically generated static parts of your site truly static

I intend to keep all the text in the __DATA__ fields, but not in their current directories. I am not sure what the difference is between me leaving the files in place and renaming them or moving the text to a new directory with a similar new filename.

./Collections/Fiction.pl # to
./Collections/Fiction.txt # or
./files/text/Collections/Fiction.txt
[download]

Leaving them in place would mean adding another grep to exclude all .txt files in the directories in base_menu along with initial lower case files. I'm having difficulty rewriting base_menu for the new structure I am contemplating.

I think .dat would expand to "data", so what would .lat expand to be? I am curious.

My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.8.8 on web host.

Version control is a non-issue, I do not use it.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!

Lady Aleena

Update: I am done mostly with writing the indexes. There are a few parts of the code that I am still mulling over. The following is a basic index script.

I have 5 page-scripts that do not use story, so they are still independent scripts for now until I can figure out how to get them into the menu automatically if I put their code into their indexes. I mentioned in the OP that I had to rewrite my base_menu subroutine. Lines 16-28 took me several hours over a couple of days to get right.

After our CB convo, I decide to try to follow my own advice, this is what I got for Programs.pl:

#!/usr/bin/perl
use strict;
use warnings FATAL => qw( all );

package my::site::Programs;

sub get_data {
    my $self = shift;    # not used but passed with '->' notation
    return q{This is a list of B<programs> that I am using or have use
+d. I can not account for I<all> the software we have had and used ove
+r the years. Some of it was so bad, we blanked it out of our heads. T
+his list does not include a full list of hardware drivers either. So 
+much software, so little time or in this case patience.};
}

1; # not sure if this is actually needed here, but would be if you mov
+ed this to a .pm that is "use"d or "require"d

# replicate the functionality of the original Programs.pl
package main; 

use CGI::Carp qw(fatalsToBrowser);

use lib '../files/lib';
use Base::Page qw(page story);
use Util::StoryMagic qw(program_magic);

my $magic = program_magic;
page( 'code' => sub { story(my::site::Programs->get_data(), { 'line ma
+gic' => $magic }) }); # note you'll have to change how you handle wha
+t was passed as "*DATA"
#
# NOTE, since you're now not using __DATA__ (via *DATA handle), you ca
+n make the "main" content
# minimal by passing in just the code reference:
#
# package main;
# use lib '../files/lib';
# Base::Page->page( code => \&my::site::Program::get_data, story => q{
+Util::StoryMagic} );
#
[download]

The goal here was to show what I meant by making the catagory files into modules and to drop __DATA__. This offers no change to how things work now, but by converting all of your .pl files to a module internally, this sets you up for some things:

1. incrementally put your data into module-like libraries
2. move them to functional libraries by turning the above into a modulino - allows you to  use require q{../file/Programs.pl}, e.g.,
3. once they work as they do now, but can be used us libraries as modulinos, you are free to experiment with Dancer2 or Mojo since they will fundamentally differ from your existing code being router/dispatch based and not file based (e.g., require content modules in the route handlers)
4. then you can decide how to proceed, but this allows you to retain the functionality of the site now while exploring a more modern approach

Then eventually if you made them full libraries, you'd pull them into to the framework like:

# in framework route handing
require q{path/to/Programs.pl};
my $data = my::site::Programs->get_data();
#...now do something with $data
[download]