guillaumerava has asked for the wisdom of the Perl Monks concerning the following question:

VT100 escape sequences looks like this: \033[1m , \033[12G ... How could I strip them from a string knowing that they always start with the ESC character (octal code 033), ends with a letter (upper or lower case), and that an undefined number of digits, commas and space are in the middle ? ( I'm not sure this description encompass *all* VT100 escape sequences though)

Replies are listed 'Best First'.
Re: how to strip VT100 escape sequences?
by Fastolfe (Vicar) on Dec 19, 2001 at 07:44 UTC

    You can always try it and see if any escape sequences get left behind. I agree that just "assuming" that this is how a VT100 escape sequence is always represented is a bad idea, but until you find authoritative information to tell you otherwise, it's pretty easy to write some code to strip what you do have out. You've very nearly stated your requirements:

    1. A literal \033 (or \e in Perl)
    2. A literal bracket [ (I added this)
    3. One or more (digits, commas, spaces)
    4. A letter

    Pitting this against perlre, we can construct this:

    /\e # ESC \[ # [ [\d,\s] # one of: digit, comma, whitespace + # at least 1 [A-Z] # letters from A-Z /ix # case-insensitive # or: /\e\[[\d,\s]+[A-Z]/i

    Put that in the left-hand-side of a substitution operator and you have yourself a functional escape code stripper.

      Well, Fastolfe, VT100 escape sequences need not have any parameter value at all, in which case the terminating letter immediately follows the left square bracket. (In this case, any numeric value is taken to be zero.) Thus you need a "*" instead of a "+".

      And I'd be really surprised if lower case letters can't end a sequence.

      And I'm pretty sure there are some sequences that end with punctuation marks.

      And I'm fairly sure that multiple parameters are separated with semicolons, not commas.

      Ah, forget it! I'm gonna fire up my old H19 terminal and leave all this DEC hair far behind....

          -- Chip Salzenberg, Free-Floating Agent of Chaos

        You appear to have remembered correctly, chip, except that apparently no escape sequence ends in a punctuation mark.

        Interestly enough, an examination of CPAN shows that there is no module that currently does this. Writers, to your keyboards!

        --
        g r i n d e r
        just another bofh

        print@_{sort keys %_},$/if%_=split//,'= & *a?b:e\f/h^h!j+n,o@o;r$s-t%t#u';
        I'm sure the original poster appreciates the tips.. Though my post wasn't so much on how to do what he was doing (strip VT100 escape sequences), but how he might figure it out for himself. I didn't do anything beyond his requirements deliberately (because I can't speak for all escape sequences).
Re: how to strip VT100 escape sequences?
by hossman (Prior) on Dec 19, 2001 at 07:52 UTC
    How could I strip them from a string knowing that they always start with the ESC character (octal code 033), ends with a letter (upper or lower case), and that an undefined number of digits, commas and space are in the middle ?

    Off the top of my head...
    s/\033[0-9, ]*[a-zA-Z]//g

    NOTE: That won't accutally replace the two examples you gave however, because your examples contain "[" which wasn't in your description.

Re: how to strip VT100 escape sequences?
by larryk (Friar) on Dec 19, 2001 at 22:20 UTC
    Non-perl way: If VT100 escape sequences are the same as the markup on man pages then you can pipe it through col -b as I do to get a plain text version of those man pages.

    e.g. man bash | col -b > bash.txt

    If not then I have completely mixed myself up and can be safely ignored!

       larryk                                          
    perl -le "s,,reverse killer,e,y,rifle,lycra,,print"
    
Re: how to strip VT100 escape sequences?
by Anonymous Monk on Oct 22, 2009 at 06:52 UTC

    Hi

    Here is one hässle which works 4 me :
    perl -pe 's/\e\[[\d,\s]*[a-zA-Z]//g; s/\e\][\d];//g; s/\r\n/\n/g; s/[ +\000-\011]//g; s/[\013-\037]//g' < in > out

    'njoy your day,

    Sami

      ...and for want of some code tags, a node was lost...

      (Even after fixing the [ and ] bits, it looks a bit off to me, but I'm not well versed on VT100 escape sequences.)

      ...roboticus

      Then there's Guide de Rosa's unescape.pl, stripping both too little and probably too much.

      I added some more stripping to better cope with ANSI/ curses esc sequences or vi sessions. Consider script's typescript of a shell session in which you invoked mc or vi. A bit less messy & more grep-able than before, but still by no means a clean session command + output protocol. See also my notes at the start of this script. I update my version each time when I run into too many esc sequences in a transcript. But it won't ever be able to do a 100% job.

      Some more information and keywords can be found e.g. in http://invisible-island.net/xterm/ctlseqs/ctlseqs.html, in this case fromt the xterm side of things.

      IIRC, weren't there length-determined arbitrary byte sequences as well (w/o end character)?

      Basically either you know both $TERM and the real terminal interacted with in the session (incl. it's initial settings and size), and reimplement all escapes for both of them (at their specific versions at that point in time even?), or you'll never be able to catch exactly 100% of the sequences being generated or intercepted by terminal or application. Finally add some line-noise for flavour, a pinch of bit-rot, exactly 7 bits of a multi byte encoding, stir & enjoy.

      Here's a simpler task to consider: Split a transscript into lines. Or just: Determine the line width of a transscript (from a non-resized terminal). Problem: still suffers the same problems as above. In the end, there's only a single known '100% perfect' use of a (timed) transcript: replaying on the very same terminal with the same initial settings and size... .

      In the end, the script command is surprisingly useless... . Some other tools also provide means for dumping a scrollback buffer, e.g. xterm or screen (which also suffers the line width guessing issue, but at least doesn't contain escapes).