Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical


by Bod (Curate)
on Nov 15, 2020 at 00:48 UTC ( #11123653=user: print w/replies, xml ) Need Help??

Long time amateur coder since growing up with a ZX Spectrum and BBC Micro...

Introduced to Perl in the early 1990's which quickly became the language of choice. Built many websites and backend applications using Perl including the sites for my property business:
Lets Delight - company site
Lets Stay - booking site
Also a few simple TK based desktop apps to speed things up.

Guilty of only learning what I need to get the job done - a recipe for propagating bad practice and difficult to maintain code...difficult for me so good luck to anyone else!

Now (Nov 2020) decided to improve my coding skills although I'm not really sure what "improve" means in this context. It seems Perl and best practice have come along way since I last checked in and my programming is approach is stuck in the last decade.

Onwards and upwards...

20th October 2021 - added to Saint in our Book 😀

Find me on LinkedIn

CPAN Release


Posts by Bod
Quoting hash keys in Seekers of Perl Wisdom
5 direct replies — Read more / Contribute
by Bod
on Nov 08, 2021 at 19:11

    This follows on from Indirect Object Syntax but it is a different question so it gets its own place!

    In doing some of the reading that came out from Indirect Object Syntax, especially this on SO from ikegami, I have noticed that often (but not always) hash keys are not quoted literals but barewords. Given the potential for confusion that has been shown from Indirect Object Syntax, it seems that this is another place where confusion could arise.

    I have noticed this before but never really thought anything of it.

    my $value = $hash{ key };
    I never write it like that. I always quote it unless the key is non-constant.
    my $value = $hash{'key'}; my $value = $hash{ $key_value }; my $value = $hash{"st_$id"};

    Is there a difference between $hash{ key } and $hash{'key'}?

Indirect Object Syntax in Seekers of Perl Wisdom
5 direct replies — Read more / Contribute
by Bod
on Nov 06, 2021 at 16:04

    Over on on another thread, I got my wrist slapped* by kcott for using Indirect Object Syntax. I have read the linked documentation and its warnings and I am not sure I totally understand. So I am hoping some wise Monks will help with clarification.

    The offending code I posted was:

    my @bounds = new GD::Image->stringFT($colour, "Image/outline.ttf", 90, + 0.18, 0, 0, $watermark_text);
    As I read the documentation, the problem is that the Perl interpreter has difficulty knowing whether I mean:
    @bounds = &new(GD::Image->stringFT(...));
    my $gd = new GD::Image; @bounds = $gd->stringFT(...);
    (&new used to make it clear it is a subroutine!)
    and because Perl's interpreter could potentially get this wrong, so could any human trying to understand the code.

    Is that about right???
    Or is there more too it than that?

    A secondary question that follows on...
    The documentation says:
    To parse this code, Perl uses a heuristic based on what package names it has seen, what subroutines exist in the current package, what barewords it has previously seen, and other input. Needless to say, heuristics can produce very surprising results!

    Does this mean that a constant piece of code, such as a module, could behave very differently depending on context? For example, if the same module were utilised in two different scripts? And what about between different versions of Perl. Could code behave differently depending on the version of Perl?

    * Just to be clear, it was a very welcome wrist slapping from kcott - I am here to learn and hopefully help others. I am always very grateful when my mistakes are pointed out as it allows me to improve my skills.

Procedural vs OOP modules in Seekers of Perl Wisdom
7 direct replies — Read more / Contribute
by Bod
on Oct 28, 2021 at 18:52

    When writing a new module, how do you decide whether to write it as an OOP module or a procedural module?
    In the modern era of Perl, is there any reason to create a new procedural module or should they all be OOP?

AWS EC2 and Perl in Seekers of Perl Wisdom
4 direct replies — Read more / Contribute
by Bod
on Oct 20, 2021 at 09:00

    Many will know that I need to upgrade from the current shared is overdue!

    A dedicated server would, of course, be ideal but I'm not sure I can justify the cost - I'm not totally convinced that I can't either! Plus, there is the server admin that I certainly don't have the time for or the skills for. The latter is solvable but not the time issue in the short-term.

    So, I thought about an AWS EC2 instance for all our webs services. This seems an ideal solution for a number of reasons:

    1. Once setup, take an image to restore if I mess things up
    2. Whatever version of Perl we want, we can install
    3. For testing, we take an image of production, spin up another EC2 instance and use that for testing/developing thus avoiding having a test setup and potential security holes
    4. No need to guess about the specs we need - AWS automagically scales as needed
    5. If one part of the business grows significantly, its easy to spin up another EC2 instance just for that
    6. Not as much server admin needed as a dedicated server but still some

    In some ways this seems too good to be true...
    Is this a plan worth looking further into or have a overlooked a flaw somewhere?

    We have databases of significant size - the biggest has 138 tables but only takes up 285Mb storage space. Plus we use Perl for everything! The shared hosting runs Apache but I have nothing to do with that other than some simple additions to the webroot .htaccess file.

my within brackets in Seekers of Perl Wisdom
6 direct replies — Read more / Contribute
by Bod
on Oct 14, 2021 at 13:19

    I've come across this bit of code within an existing legacy script - not written by me

    (my $pname,my $sname)=split / +/,$rname,2;

    I'm wondering if it is functionally different to writing

    my ($pname, $sname) = split / +/, $rname, 2;
    I've tested the two against each other and they seem to behave exactly the same. But, is there some subtle difference my quick test has not uncovered?

Errors uncaught by CGI::Carp in Seekers of Perl Wisdom
2 direct replies — Read more / Contribute
by Bod
on Oct 13, 2021 at 16:46

    This is a follow on from the second part of the question I asked in CGI::Carp fatalsToFile
    The collective wisdom of The Monastery solved the first issue but not the seems they are unrelated!

    I am getting 500 errors intermittently from most, if not all of the pages of a website. The errors seems to be clumped together over time. In other words, once an error happens it seems likely that it will happen again quite soon. Whereas at other times we go quite a while without errors.

    Every script has use CGI::Carp qw(fatalsToBrowser); at the start of the script.
    The bit I am most confused about is that my understanding is that, with fatalsToBrowser the only two ways to get a 500 error is that the script has the wrong permissions or that the script doesn't return valid HTTP headers. As this is an intermittent problem, it is neither of these.

    This is the code of the modules that are common :

    package Site::HTML; use strict; use warnings; use DBI; #use DBD::mysql; use Exporter; use Template; use MIME::Lite; use Digest::SHA qw(sha256_base64); use cPanelUserConfig; use Site::Variables; use Bod::CRM; our @ISA = qw(Exporter); our @EXPORT = qw(%data %file %cookie $dbh $current_user); our $site = $Site::Variables::site; our $template = Template->new( INCLUDE_PATH => $Site::Variables::template_path, PRE_CHOMP => 0, POST_CHOMP => 1, ); our (%data, %file, %cookie, $dbh); our $current_user = 0; my $crm = Bod::CRM->new($Site::Variables::env_crm_env); # Untaint ENV{PATH} for MIME::Lite my $path = $ENV{'PATH'}; $ENV{'PATH'} = undef; foreach my $p(split /:/, $path) { if ($p =~ m!^(/(usr|bin).*)!) { $ENV{'PATH'} .= ':' if $ENV{'PATH'}; $ENV{'PATH'} .= $1; } }
    package Site::Wayfinder; use strict; use warnings; use cPanelUserConfig; use Digest::SHA qw(sha512_base64); use Facebook::Graph; use GD; use Bod::CRM; use Site::HTML; use Site::Point; my $crm = Bod::CRM->new(Site::HTML->crm_env); sub new { my $class = shift; my %attrs = @_; $attrs{'error'} = ''; $attrs{'fb'} = Facebook::Graph->new( app_id => '12345678', secret => 'abcdefghijklmnop', postback => "https://$ENV{'HTTP_HOST'}/?command=authori +ze_facebook", ); return bless \%attrs, $class; }
    I have commented out use DBD::mysql; because on one occasion I got some output from CGI::Carp that showed an error on that line. I have not been able to replicate this error. Not specifying a DBD driver forces DBI::DBD to use the driver in it's config which is setup to connect to the MariaDB database.

    My only hunch is that, because errors seem more concentrated at certain times, it is connected with the number of concurrent users. This in turn makes me wonder if is is Facebook::Graph that is the problem as it is deprecated. We only use Facebook for authentication. Users can create an account using Facebook rather than creating a password or can sign into an existing account using Facebook provided the email addresses are the same. This works under normal circumstances.

    It isn't just web browsers that are getting the error. Facebook is complaining that our privacy policy page is giving errors and Stripe gets occasion webhook errors.

    I am unable to replicate the problem on our test site - it only seems to happen on the live production site. Therefore I am limited on how much I can test or what I can tweak!

    Any ideas what I can do to try and get some understanding of what is going on?

    UPDATE: - to test my hunch that it is to do with multiple users, I have tried to cross referencer 500 errors with the data collected by Google Analytics and Hotjar. But the errors are not written to the Apache error log so this is proving difficult. I have started having Google Analytics open when I access the site so that I can look at the real time data when I next see an error.

CGI::Carp fatalsToFile in Seekers of Perl Wisdom
4 direct replies — Read more / Contribute
by Bod
on Oct 05, 2021 at 13:14

    I'm looking for some debugging help please
    Ideally I would like something like a fatalsToFile equivalent to the CGI::Carp fatalsToBrowser.

    I have what appears to be the same issue on two different websites. They are both on the same shared hosting but otherwise totally independent.

    The most critical issue is with a script that gets booking information from OTAs, Online Travel Agents (Airbnb,, etc) using their API to provide an ICS file. Our booking platform checks this information as well as our own booking database to show customers our availability. Our script caches the ICS file for 30 minutes. This script has an API that our booking platform calls to get a simple text file which lists the property number and either 'available' or 'booked'. When a customer commits to book, the API is called again but with a force parameter to bypass the cache.

    Except the API returns a 500 error when called from the booking platform. But I cannot replicate it when calling the script using a web browser.

    Here's the code we had working. I know there are some bad practices in there but this is legacy code and will get improved in due course. The important thing is that is was working fine until recently and I am not aware of anything that could have caused the change:

    if (!defined @avail) { foreach (split /\n/, &get("https://$ENV{'HTTP_HOST'}/cgi-bin/booki$data{'st'}&ed=$data{'ed'}&force=$data{'force' +}", $availability);) { chomp; my ($k, $v) = split/,/; $avail[$k] = $v; } }
    The &get is from LWP::Simple.
    To try and understand what was going on, I changed the code to this:
    my $availability = LWP::Simple::get("https://$ENV{'HTTP_HOST'}/cgi-bin +/$data{'st'}&ed=$data{'ed'}&force=$data{' +force'}"); print "->$availability<-\n"; if (!defined @avail) { foreach (split /\n/, $availability) { chomp; my ($k, $v) = split/,/; $avail[$k] = $v; } }
    and found that $availability was empty.
    my $availability; my $response = LWP::Simple::getstore("https://$ENV{'HTTP_HOST'}/cgi-bi +n/$data{'st'}&ed=$data{'ed'}&force=$data{ +'force'}", $availability); print qq[("https://$ENV{'HTTP_HOST'}/cgi-bin/ +st=$data{'st'}&ed=$data{'ed'}&force=$data{'force'}")\n]; print "->$availability<-\n"; print "$response\n";
    This shows that the response code is 500.
    Copying the printed URL into a web browser gives the text output that I would expect.

    I have checked the error logs provided to me by cPanel and nothing shows up at all.

    Any ideas why a script would return a 500 error when called by LWP::Simple get but not when called by Chrome?
    There is nothing in the script that checks the user agent and it was previously working.

    I have a similar, but intermittent problem on another site. This time the script is called from the browser using the fetch API. The script returns a JSON object. Again, the script returns a 500 error some of the time but works as predicted most of the time (unlike above which fails every time). Again I am at a loss how to debug this as I cannot catch the errors to find what or where they are.

    This is the Javascript from the Template file that calls the Perl API which generates the error:

    fetch('/api/', { method: 'POST', body: 'a=[% token %]&u=[% user %]&q=getWays&' + request, }).then(response => response.json()) .then(data => { document.getElementById('mapmsg').innerHTML = ''; if (data[0][0].status == 'success') { plotWays(data); } else { document.getElementById('timeoutbox').style.display='block +'; } }) .catch(error => { console.log(error.message); });

    If I had any hair, I would be pulling it out...

Similarities of Perl and Python? in Meditations
8 direct replies — Read more / Contribute
by Bod
on Aug 30, 2021 at 06:07

    Learned and wise Monks...

    Within my business we use the usual front-end web technologies of HTML/CSS/native Javascript/AJAX and the back-end is entirely Perl. With a very small amount of Java for two Android apps.

    Here in the UK there is currently a government scheme to get young people into work. The Kickstart Scheme pays the young person minimum wage for 25 hours per week for 6 months. We have applied to take on two young people under the scheme. Most applicants have played with React at most but I am interviewing a graduate tomorrow. On his CV he has C, Python and R which has been done as part of his Physics degree plus Python and Shell Scripts on a Raspberry Pi as a hobby.

    I know absolutely nothing about R.
    From my very limited knowledge of Python, I believe it is broadly in the same group of languages as Perl. Therefore, the skills this applicant already has in Python should be relatively easily transferrable to Perl
    - is that a fair assessment???

    Any suggestions for me bearing in mind the applicant has had no real workplace experience?

Creating an abstract in Seekers of Perl Wisdom
4 direct replies — Read more / Contribute
by Bod
on Aug 09, 2021 at 16:14

    In a few places I have code that takes a significant piece of text, such as a knowledgebase article, and splits a small section from the beginning as an abstract. The code I generally use came from an answer to Splitting long text for Template

    But now, for a different application, I want to create an abstract of text which may contain HTML tags. The problem comes with not wanting to split an HTML tag in two. I want either all of it or none of it. So this is the code I am using...

    sub abstract { my $text = shift; if (length $text > 200 and $text =~ /^(.{0,200}\b)(.*)$/s) { $text = "$1..."; } # Check we have not split an HTML tag my $lt = $text =~ tr/<//; my $gt = $text =~ tr/>//; if ($lt != $gt) { my ($keep, $strip) = $text =~ /(.*)<(.*)/; $text = "$keep..."; } return $text; }
    It does exactly what I want.

    However, I cannot help thinking that the code could be more succinct...
    Can you suggest a better way to do it?

Insecure CPAN module in taint mode in Seekers of Perl Wisdom
3 direct replies — Read more / Contribute
by Bod
on Jul 06, 2021 at 15:03

    Still trying to get to grips with taint mode...

    What is the best course of action when a CPAN module throws an error under taint mode?
    I am getting: Insecure $ENV{PATH} while running with -T switch at /usr/local/share/perl5/MIME/ line 2697, <DATA> line 1000.

    This appears to be a documented bug in MIME::Lite. However, as this module is widely used, I assume there is a way to use it with taint mode.

    Is there a way to call it differently?
    Or should taint mode be turned off just for this module if that is even possible?
    Or do I have to use a different module?
    Or is MIME::Lite actually OK and it is my code that is is wrong?

    I also suspect that MIME::Lite isn't the only taint incompatible module on CPAN. So some general advice on dealing with this situation would be welcome.

    This is the code I am using to call MIME::Lite:

    sub email { my ($self, $disp, $id, $vars) = @_; my $cid = $crm->get($id); return undef unless $cid; return 0 unless $cid->{'email'} and $cid->{'fname'}; my $message; $template->process("email/$", $vars, \$message); $vars->{'from'} ||= 'xxx<'; $vars->{'subject'} ||= 'xxx'; $vars->{'fname'} = $cid->{'fname'}; my $mail = MIME::Lite->new( From => $vars->{'from'}, To => $cid->{'fname'} . ' ' . $cid->{'sname'} . '<' . $ci +d->{'email'} . '>', Subject => $vars->{'subject'}, Type => 'text/html', Data => $message, ); return $mail->send; }

Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (5)
As of 2021-12-06 17:56 GMT
Find Nodes?
    Voting Booth?
    R or B?

    Results (33 votes). Check out past polls.