Re^2: heuristic to detect (perl) code

yep, it's an extended version of my sigil-frequency idea... but it's a good start, THANKS! =)

But IMHO it would be necessary to roughly strip:

comments,
strings,
here-docs and
__DATA__, __END__ sections,

since adding some comments makes your example already "debatable".

BTW: I tried your Nodelethack in Re^3: CSS Show and Tell: Colored Code but it didn't work for me...:(

Do you know if CodeMagic provides an API to do code sniffing or is the logic internal?

Cheers Rolf

Comment on Re^2: heuristic to detect (perl) code

Replies are listed 'Best First'.
Re^3: heuristic to detect (perl) code by 7stud (Deacon) on Jan 20, 2013 at 09:22 UTC
1) Comments: `my $count = ($line =~ s/[#] .? \n//xms); $total += $count;` [download] 2) Strings: `$line =~ s/["'] [^'"\n] ['"]//gxms` [download] 3) use statements: `$count = ($line =~ s/use [^;]+ ;//gxms); $total += $count;` [download] 4) Here docs: `my $count = ($all_text =~ s/<<(\w+) .? \1//gxms); $total += $count;` [download] 5) __DATA__, __END__: `$total++ if $all_text =~ s/(__DATA__\|__END__) .*//xms;` [download] Although, I would argue that if __DATA__ or __END__ appear anywhere in the text, then you couldn't go wrong by delcaring then and there that the text has perl code in it.	[reply] [d/l] [select]
Re^4: heuristic to detect (perl) code by Anonymous Monk on Jan 20, 2013 at 10:25 UTC
I would have said you can't go wrong with /\Q=~\E/ or /\Q!~\E/ or /print\s\{/ or /<>/ or /<\s\$?\w+\s*>/ but that argile/frink/groovy/slate/smalltalk have =~	[reply]
Re^4: heuristic to detect (perl) code by LanX (Saint) on Jan 20, 2013 at 10:46 UTC
It's not that easy for instance: 1) comments '#' should (mostly) follow a newline or a semicolon or to be more precise the '#' shouldn't be preceded by a quote-like operator (single s, y tr, q, qq, qr or qw or whatever) 2) strings are closed by the same quote so you need to capture the opening one and check the ending with \1. 3) __DATA__ must appear at line start, OTOH the existence of DATA is already a good indicator for perlcode. I think discussing single strategies is for vain, in the end you have to test and train different criteria against a suitable big amount of perlmonk posts, to see if the code-sections are found. With bayes classifier there is a very good mathematical method to combine the probabilities of such methods. Some of the products I listed in OP use this approach, they are just not trained for perlmonks posts (where tiny code-snippets also appear in text) and have maybe a to heavy footprint to be integrated here. update For instance highlight.js has a function which returns the guessed language. Cheers Rolf	[reply]

update