lis128 has asked for the wisdom of the Perl Monks concerning the following question:

hello fellow monks

From long time i've been developing little Perl programs to do some stuff. As i do often (ab)use other REST apis to scrap data i need, i am familiar with Curl blessings, i know how to perform simple GET or trick certain endpoint to POST him given payload to receive wanted answer. Until now it almost always happened in console, where requested informations were thrown out to STDOUT. Simple stuff

But my programs always lacked a crossplatform, human friendly web interface. Now i wanted to change this and write my own API (simple DB selects, some system() executions) and send AND receive response in browser
So i started Apache, made it capable of running CGI scripts and wrote some simple "parse url arguments and return json" procedures. it works, cool.
Problem arose when i tried to use same set of tools to run my "frontend". My "webui" consists of single form with textarea and submit button. Action is targetted against my api. So i fill textarea, press submit and... i get raw json response in my browser.
Question is:

How do i separate "api" from "frontend" and made them pass data between each other in order to
- have nice form
- have api to do actual job
- receive data from api in json format, process it and return output to user in "web format", like a table?
i deliberately am trying to avoid javascript and other node.js-junk, as i don't want to have anything in common with JS ;p

my first idea was to include some sort of www-client (like LWP) in my "fronted" to do request on my behalf, capture "api's" output and process it, but it seems like big overkill for me to have Apache running CGI script which "embeds" www client which requests data from other apache instance.
My other idea (utilizing forms) was to submit to the same page (my fronted) and check wheter request was caused by form submission (then do www-client magic) or just by entering site "for the first time to fill the form".
Both solutions feels clunky to me, so here i am, seeking a wisdom ;)

My frontend doesn't have to be pretty, "responsive" and "modern". I just want to be able to click my buttons using browser in phone, tablet and computer, simpler the better in my case
Also separation is important as i want to develop only my "backend" features, while focusing as little as possible on frontend. Seriously, form with button and optional checkboxes is something which will satisfy me as this point

Replies are listed 'Best First'.
Re: a simple web front end
by hippo (Archbishop) on Oct 27, 2020 at 22:09 UTC
    but it seems like big overkill for me to have Apache running CGI script which "embeds" www client which requests data from other apache instance.

    You say overkill, I say classic application stack. For the past quarter century or so we've been working to a greater or lesser extent with this setup where the user/client/browser interacts with the web server(s), behind which is the application server(s), behind which is the data source(s). There are other ways for sure but this is by far the most common basic setup I have seen across many installations. You already have the backend sorted by the sound of it so adding the lightweight front end should not be "overkill".

    🦛

[reply]

      Thank you, probably i needed assertion like yours from someone way more experienced than me.
      So - from your classic app stack experience - is having frontend posting to itself, reloading page and act accordingly on postdata, an OK approach? Something like

      print $q -> header();
if ($q -> param) {
   # we do have something to process, lets move on
   do_process($q -> param); 
} else {
   # this is clean entry, just display options
   show_page_with_form($q);
}
# despite of previous step's outcome, show footer with copylefts and e
+nd document gracefully
show_footer($q);
print $q -> end_html;
1

      [download]
[reply]
[d/l]

        Yes, that's fine. Combining a form plus its handler in one script/URL is not atypical and it gives an easy way for you redisplay the form in the handler if the user's inputs are in some way invalid.

        One more thing I would recommend is to look at a templating system. This is another thing that you might consider overkill just now but will pay off handsomely in the long run. I tend to use Template and other folks swear by HTML::Template and there are a host more to choose from too. The point is to separate the rendering of the output from the logic of the application and to allow for easy re-use of that output in terms of common page layout, error handling, etc.

        Good luck with your project!

        🦛

[reply]
Re: a simple web front end
by haukex (Archbishop) on Oct 28, 2020 at 21:07 UTC
    So i fill textarea, press submit and... i get raw json response in my browser. Question is: How do i separate "api" from "frontend" and made them pass data between each other in order to - have nice form

    Note you can seprate your API from the frontend purely on the Perl level. Mojolicious includes content negotiation, as I show in the following example (run it e.g. via morbo script.pl). Note how POST and GET just work, including http://127.0.0.1:3000/myform?foo=helloworld&format=json and http://127.0.0.1:3000/myform.txt?foo=blahblah.

    #!/usr/bin/env perl
use Mojolicious::Lite -signatures;

any '/' => sub ($c) { $c->redirect_to('/myform.html') };

sub my_api_do_thing ($arg) { # could/should be in a module!
    return $arg =~ s/[aeiou]//gr;
}

any '/myform' => sub ($c) {
    if ( defined $c->param('foo') ) { # if the form was submitted
        $c->stash( returnval => my_api_do_thing( $c->param('foo') ) );
    }
    $c->respond_to(
        json => { json => { retval => $c->stash('returnval') } },
        txt  => { text => $c->stash('returnval') },
        html => { template => 'myform' },
    );
};

app->start;
__DATA__

@@ myform.html.ep
% layout 'main', title => 'Hello, World!';
<div>
% if ( stash 'returnval' ) {
    <div>
    Return value: <%= stash 'returnval' %>
    </div>
% }
%= form_for myform => ( method=>'post' ) => begin
<div>
    %= label_for foo => 'Foo'
    %= text_field foo=>'Foobar?'
</div>
<div>
    %= radio_button format => 'html', id=>'format_html', checked => un
+def
    %= label_for format_html => 'HTML'
    %= radio_button format => 'json', id=>'format_json'
    %= label_for format_json => 'JSON'
</div>
<div>
    %= submit_button
</div>
%= end
</div>

@@ layouts/main.html.ep
<!DOCTYPE html>
<html>
<head><title><%= title %></title></head>
<body>
%= content
</body>
</html>

    [download]
[reply]
[d/l]
[select]
Re: a simple web front end
by tobyink (Canon) on Oct 27, 2020 at 21:42 UTC

    Take a look at some of the MVC frameworks on CPAN. Dancer2 is probably a good one to start with. If you've been mostly using CGI scripts, your first few steps will likely be a steep learning experience, but it will be worth it.

    toby döt ink
[reply]

      i've heard already about Dancer, but all examples were self-contained, i.e Dancer started its own webserver and my goal was to take advantage of Apache's VirtualHosts in order to not tinker too much about threading, forking, handling whole protocol layer "by hand" and so on.
      Also, using webserver's name-based virtualhosts makes my "application" easliy scalable: for development purposes i can have it running on single machine, running single instance of apache on single 80 port, and separate front from back only when usage scenario will require it.
      However, i must admit that didn't took too much time to check Dancer's capabiliites.
      But if you said that it's wort to give a try, i at least check if Dancer's app can be embedded into Apache's CGI/PSGI

[reply]

        Dancer2::Manual::Deployment shows how you can run Dancer2 apps via CGI, FastCGI, or PSGI. CGI, I really wouldn't recommend because it will perform horribly, but the other options should work fine. Another possibility it to run it standalone and use Apache mod_proxy to act as a frontend for it.

        toby döt ink
[reply]
Re: a simple web front end
by bliako (Abbot) on Oct 30, 2020 at 08:59 UTC

    As haukex's code implies, you can program the backend to send json, text, excel, word-doc or html or pdf or ... to the client according to client's needs. It means you have a separate backend handler sub for each output format the client requests. The art is to not get lost in all those formats when you decide to make a change in the raw data (e.g. DB schema), it must be reflected in all the output formats: html, pdf and text etc., as transparently as possible.

    hippo mentioned templates, which are not for HTML only btw (you can create TEXT templates with the same module just as easily). Using templates can be one way to make your process as painless as possible. You must create a template for each of your output formats and also create subs to convert a perl hash with the data you want to render to each of the formats by loading the template file and filling in the gaps, so-to-speak. Adding a new field to DB and sending it back to the client still means editing/adjusting each template file but this is more "tidy" IMO and you should not need to modify any perl code except db retrieval. Something like:

    my $data = retrieve_from_db($params);
if( $params->{need_html} ){
  my $html = perl2html($HTML_TMPL_FILE, $data);
  render(html => $html);
} elsif( $params->{need_json} ){
  my $json = perl2json($JSON_TMPL_FILE, $data);
  render(json => $json);
} # ...

    [download]

    You are thinking of creating an intermediate layer to act as a translator from raw-data (perl-hash in my example, json in your setup) to HTML using LWP. In essense that's what I am doing above but I do it within the app and not using LWP at all. If you must go with separate layers then consider creating separate apps which send requests to each other within the same server, residing on different ports. Come to think of it, a DB is just one of these apps which can even reside in physically separate server. An advantage of using different apps is that you can distribute your load to different physical servers and isolate "sensitive" apps behind internal firewalls.

    Alternatively, you have the choice of "templating" in the client using javascript. Essentially your client accepts only json and your javascript will take that, convert it to HTML or PDF or EXCEL (I guess there must be some libraries for the latters???) and inject the output dynamically into the client's browser's current page. HTML injecting is quite simple to do now-a-days.

    bw, bliako

[reply]
[d/l]

Back to Seekers of Perl Wisdom