in reply to Too difficult for me...

From your description, it sounds to me as if you have an XML representation of the Abstract Syntax Tree to execute your code, but it's hard to tell. You posted a wall of text, but no code. I'm not sure what part of it relates to the problem, but you have this one sentence:

It's just not fast enough....

Without showing us some code, we can't help you there except with general points. Do less work - don't work on the textual representation of your program, but preparse it into a data structure. Cache expressions instead of re-evaluating them every time. Create static output as files instead of creating the output with every page load.

Replies are listed 'Best First'.
Re^2: Too difficult for me...
by simonodell (Acolyte) on Jun 19, 2011 at 07:46 UTC
    You wouldn't like my code....

    My code doesn't need fixing... it needs replacing! the parser I have written implements the syntax and provides a working platform, but it is pre-alpha at best.

    I just read the link you gave me and it seems similar, but the syntax I have is not quite the same as XML, as it has two extra types of tag delimiters ( ) and [ ]

    I'm really shooting in the dark as to how to explain what I have, so please bear with my ignorance (not stupidity, at least I hope not).

    perhaps if I gave you some more examples of the syntax itself you could understand what I am getting at... like I said the code I have which runs it ought to be deleted, and the hard drive it is stored upon melted down!

    anyway...

    Upon receiving a request, the existing system looks up a file in the appropriate folder called body.aXML (aXML is what I call it but I am aware that name is taken, another reason for a rethink) 
    
listing of an example body.aXML

<html>
  <head>
    <title><conf>site_title</conf></title>
  </head>
  <body>
     <use>main</use>
  </body>
</html>

listing of main.aXML

<div id="comments">

  <table>
    (sql mode="mask")
      <query>
        SELECT * FROM comments;
      </query>
      <mask>
        <tr>
          <td><d>comment</d></td>
          <td>[link action="showuser" 
                    userid="<d>userid</d>"
                  ]<d>username</d>[/link]</td>
          <td><d>timestamp</d></td>
        </tr>
      </mask>
    (/sql)
  </table>
</div>

    [download]
    In most cases the body.aXML just contains a use tag which calls up a template, which then calls main.aXML but for simplicity I didn't bother putting that detail into the example. The example gives an output of comments from the SQL database, the system itself takes care of connecting to the database etc, leaving the designer free to simply plug together the existing modules using the simple syntax... this massively streamlines the development of new systems, as the modules are reusable and can be stacked together in multiple different abstract ways. for another example, here is a version of main.aXML which takes a query data argument telling it how many results to show; 
    
<div id="comments">

  <table>
    (sql mode="mask")
      <query>
        SELECT * FROM comments LIMIT (qd)limit(/qd);
      </query>
      <mask>
        <tr>
          <td><d>comment</d></td>
          <td>[link action="showuser" 
                    userid="<d>userid</d>"
                  ]<d>username</d>[/link]</td>
          <td><d>timestamp</d></td>
        </tr>
      </mask>
    (/sql)
  </table>
</div>

    [download]
    Where (qd)limit(/qd) would refer to a value passed in the query data key => value hash, under the key named "limit".
[reply]
[d/l]
[select]

      I can only recommend to you to write a compiler that takes your input language and outputs (for example) Perl:

      <div id="comments">

  <table>
    (sql mode="mask")
      <query>
        SELECT * FROM comments LIMIT (qd)limit(/qd);
      </query>
      <mask>
        <tr>
          <td><d>comment</d></td>
          <td>[link action="showuser" 
                    userid="<d>userid</d>"
                  ]<d>username</d>[/link]</td>
          <td><d>timestamp</d></td>
        </tr>
      </mask>
    (/sql)
  </table>
</div>

      [download]
      ... could become 
      my $output;
$output .= <<HTML;
<div id="comments">

  <table>
HTML
my $limit = get_qd('limit'); # whatever "qd" is supposed to be
my $results = fetch_sql( mode => "mask", query => <<SQL);
        SELECT * FROM comments LIMIT (qd)limit(/qd);
SQL

for my $row (@$results) {
    $output .= <<HTML;
      <mask>
        <tr>
          <td>
HTML

$output .= $row->{comment};
$output .= <<HTML;
</td>
          <td>
HTML
$output .= link( action => 'showuser', userid => $row->{userid}, text 
+=> <<HTML );
$row->{username}
HTML
$output .= <<HTML;
</td>
          <td>
HTML
$output .= $row->{timestamp};
$output .= <<HTML;
</td>
        </tr>
      </mask>
HTML
}; # (/sql)
$output .= <<HTML
  </table>
</div>
HTML

      [download]

      This is basically the same technique that Template uses. The remaining infrastructure of including parts of pages from other code reminds me of HTML::Mason. I think you could learn lots from looking at the respective implementations. The Everything Engine (which this site runs on) is also fairly similar, except that it doesn't try to encode database queries as HTML - it leaves plain Perl for that. Subroutine calls can be encoded as special tags, but I'm not sure that this is an overall good idea.

      Note that I'm no friend of large frameworks, because they usually work for nobody other than the author(s).

[reply]
[d/l]
[select]
        Well in effect the engine I have already scans and converts the code to perl for execution... it's just too slow because my methods are not as advanced as they could be... in all honesty I don't think I as a programmer am worthy of the discovery I am hamfistedly trying to describe and implement.

        The system is not a large framework at all, infact the core engine is less than 67KB, with about another 70KB of plugins which includes everything from input validation to database and file i/o, session management, and a whole bunch of other stuff.

        As far as I am aware, mason is 477KB just for the core engine, and catalyst 248KB.

        To answer the hidden question above, qd is a plugin which provides access to a hash containing GET and POST data.

        The source of the qd plugin is a one liner;

        $result = $qd->{$data};

        [download]
        Using the rules of this syntax, consider a request something like;

        action.pl?action=test&a=b&b=c&c=42

        [download]
        This is a silly example I know, but I can't think of any other way to express the nature of the system.

        
listing of /actions/test/body.aXML 
----------------------------------

<qd><qd><qd>a</qd></qd></qd>

        [download]
        Output : 42

        The computation starts with;

        0: <qd><qd><qd>a</qd></qd></qd>

        [download]
        the innermost qd tag runs, and in the query we find that  $qd->{'a'} = 'b'; the result 'b' is then handed off to the next tag in the expression 
        1: <qd><qd>b</qd></qd>

        [download]
        once again the plugin qd is called, only this time it is fed the result of the previous call, which was the letter 'b'. If we look up the key 'b' in the query data, we get the letter 'c'.

        the structure of the expression causes it to iterate one more time : 
        2 : <qd>c</qd>

        [download]
        and since in the query c = 42, the result of the expression

          <qd><qd><qd>a</qd></qd></qd>

        [download]
        is 42.

        The system supports unlimited levels of nesting, using any combination of the existing plugins and is extensible by writing new plugins and either adding them to the core plugins folder, the site local plugins folder, or the action local plugin folder.

        lets say you wanted to write a silly plugin to say hello... 
         
listing of "/plugins/sayhello.aXMLpi"
-------------------------------------
  
 $result = "hello $data monks";

        [download]
        now if that plugin is placed in the core plugins folder, any page on any site on the server can use the tag, 
        <sayhello>perl</sayhello>

        [download]
        and the page will contain the result "hello perl monks".

          <sayhello>java</sayhello>

        [download]
        will produce "hello java monks"... etc etc.

        if our query data contains a key value pair 'who' => 'PERL'
and we include into the document;

  <sayhello><qd>who</qd></sayhello>

        [download]
        we get "hello PERL monks".

        You all think I'm barking mad don't you??
[reply]
[d/l]
[select]
      With a more efficient rendering program, it would be possible to add in extra levels of abstraction to quickly and easily implement things such as a database abstraction layer. I could go into that in more detail but I don't want to over do it before the simplicity of the ruleset and structure becomes apparent.
[reply]
        Just had an idea.... later on I will make a short video presentation using screen capture, and hopefully seeing the thing in dynamic action will clarify it better than writing a wall of text!

        (side note...is it possible to embed a youtube video onto this forum?)
[reply]

In Section Seekers of Perl Wisdom