Is there anything around explaining or documenting what that is, how it works, and what it gives you?

There are bits and pieces here and there, and I'll try to put them together.

STD.pm lives in the pugs repo in src/perl6/. It's a Perl 6 file, and mostly contains regexes as spec'ced in S05. I wrote an article about the techniques it uses to be modifiable (feedback welcome) that tries to motivate some of the choices in the implementation.

In the same directory as STD.pm there's a perl 5 script called gimme5 that "compiles" STD.pm to a perl 5 module named STD.pmc. That file, together with some runtime files (like Cursor.pmc) is able to parse Perl 6 files.

If you want to test it, you need perl 5.10.0 installed under /usr/local/bin, and some modules like YAML::XS and re::engine::TRE.

Once you've got the requirements installed, you can go ahead and test it:

$ svn co http://svn.pugscode.org/pugs
$ cd pugs/src/perl6
$ make
$ echo 'say "Hello, world";' > test.t
$ ./tryfile test.t
$ # let it parse itself and then the test suite:
$ make test
[download]

Last time I tried tonight it hang on one test (or maybe I wasn't patient enough), and failed one other, the rest of the > 250 test files where parsed.

There's much more to say about STD.pm, its relation to rakudo and other compilers (such as elf and pixie), its implementation details (about which I'm mostly ignorant) and history - too much to write it all down at once. If there's anything that you find particularly interesting, just ask.

(Continued update here:)

What it gives you? The best parser for Perl 6 out there, and one with the option to modify the grammar from perl 6 source code. For stuff like introducing new operators you don't have to know anything about the grammar (well, nearly nothing at least), for macros you might have to know a tiny bit more, for substantial changes you'd still be able to do it without recompiling your perl compiler.

Also, is there any substance about it in the video?

No. That video talks much about philosophy, but says little in terms of substantial progress.