in reply to Re: snakes and ladders
in thread snakes and ladders

>Big O

Yup, I get that, that's why the system will spit out "hello world" in like 0.03 seconds, but take 0.22 to produce the 400 line output for the forum sections page for instance. More data = more computing = slower result.

>Efficient tokenizer with a finite state machine.

So I have to treat it almost like a de-compression algorithm? Take the input letter by letter and inflate it up into something which can then be stored and processed quickly... I think I can see how that works.


  <use><qd>action</></>

  becomes something like;

  [TOKEN name="use"]
  [TOKEN name="qd"]
  [TOKEN value="action"]
  [ENDT]
  [ENDT]

then process line by line? I can feel a stack is going to be needed. I
+ think I need to have a play with getting results from the expected t
+oken format (written by hand) so I can be sure how the system will wo
+rk post-tokenisation. (then write the tokeniser)

[download]
>beloved to other hackers

I'm not the sort of person who craves or needs adoration or respect of peers. I'm actually quite a reserved and quiet person that tends to try and blend in where possible, (believe it or not) so I won't be worrying about that very much... besides, this system isn't for high level hackers, it's for dummies like me to make work/life easier, and measured by that standard it works very well!

Replies are listed 'Best First'.
Re^3: snakes and ladders
by chromatic (Archbishop) on Aug 25, 2011 at 07:03 UTC
    ... then process line by line?

    As a proper data structure, not plain text. A data structure has, well, structure. You need that structure to identify which types of tokens you have and what they mean.

    I tend to use objects for this, but an ad hoc hash will serve as well for your experiments.

[reply]
      Something like the above? (see earlier comment posted before I saw your comment)
[reply]
        Very very like. This is what the complier folks call a "parse tree". You can, once you have the tree, "walk" it and evaluate what to do at each node. (This is what Perl does to execute a Perl program.)
[reply]
Re^3: snakes and ladders
by JavaFan (Canon) on Aug 25, 2011 at 23:12 UTC
    >Big O

    Yup, I get that, that's why the system will spit out "hello world" in like 0.03 seconds, but take 0.22 to produce the 400 line output for the forum sections page for instance. More data = more computing = slower result.

    I'm afraid you're not getting it at all.

    Complexity is not about "there's more input, so it will take longer", it's the indication how it will scale. Does it scale lineary, quadratic, logarithmic, etc.

[reply]
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re^3: snakes and ladders
by Logicus (Initiate) on Aug 25, 2011 at 05:59 UTC

    So I just did a more complex example by hand, now I need to figure out a way to iterate across such a complex data structure :/

    
# <html lang="<qd>lang</>">
#   (sql mode="mask" table="users")
#     <query>
#        SELECT * FROM users
#     </>
#     <mask>
#       Profile : [link action="profile" 
#                     username="<d>username</>"
#                 ]<d>username</>[/link]
#       <br/>
#     </>
#   (/)  
# </>




#TODO: translate above, into this :

@_ = (
        {  type => '<',
          value => 'html',
           attr => { lang => ( { type => '<', value => 'qd' },
                               { type => 'data', value => 'lang' },
                               { type => '>' } )
                   }
        },

        {  type => '(',
          value => 'sql',
           attr => { mode => 'mask', table => 'users' }
        },

             { type => '<', value => 'query' },
             { type => 'data', value => 'SELECT * FROM users' },
             { type => '>' },


             { type => '<', value => 'mask' },
             { type => 'd', value => 'Profile : ' },


                  { type => '[',
                   value => 'link',
                    attr => {
                              action => 'profile',
                            username => (
                                           { type => '<', value => 'd'
+ },
                                           { type => 'data', value => 
+'username' },
                                           { type => '>' }
                                        )
                            }
                  },
                        { type => '<', value => 'd' },
                        { type => 'data', value => 'username' },
                        { type => '>' },

                 { type => ']' },

                 { type => 'data', value => '<br/>' },

             { type => '>' },

         { type => ')' },

      { type => '>' }

     );

    [download]
    What's going to make life fun is the recursive nature of the thing, each line may have attr's which then may have muliple lines to describe them.
[reply]
[d/l]

In Section Meditations