How do I create a simple, perl-based text filter for Vim

dimar has asked for the wisdom of the Perl Monks concerning the following question:

Question:

Can someone provide a very simple proof-of-concept perl sample that works as a text filter for (g)Vim?

Background:

H is editing text with Vim. Vim can filter a region of text using (among other things) the built-in Vim-script language or an external program. H already has a trusty arsenal of perl scripts that can do various text-filtering operations, and H wants to use those instead of essentially re-writing them all in Vim-speak.

Problem:

H is able to get Vim filtering to work with other external programs, but not with perl. For example:

!sort           <-- this Vim command calls a binary, OK
!myfilter.pl    <-- this Vim command calls a perl script, FAIL
[download]

The root of the problem appears to be that H cannot figure out how to get the input from Vim through STDIN.

Proposed Solution:

H simply wants to see a bare-bones simple perl script that can filter text successfully from Vim (just like !sort can do) just to prove that it can indeed be done. It doesn't have to be sort, just something simple like:

#!/usr/local/perl
### myfilter.pl

my  $sTest  = '';
while(<STDIN>){$sTest .= $_;}
print "
    _IT_WORKED_
    $sTest
    _IT_WORKED_
";
[download]

NOTE: The previous code might as well be pseudocode, because it is not working. When run from Vim, the target text is not filtered, but simply blanked out. The _IT_WORKED_ tags do, however, appear.
NOTE: H has not been able to find any examples in the usual search venues.
NOTE: Alternate solution proposals are welcome, but see Constraints.

Constraints:

H requests cross-platform solution (linux and windows).
Vim :echo has("perl") returns false, and re-installing or recompiling Vim source is not an option.
H is willing to try an alternate approach (e.g., mucking with the Clipboard, temp files) as long as it is still cross-platform.
H knows that external process is less performant than native Vim, but performance loss is more acceptable than wheel-reinvention.

TIA

Update:Thanks for all the suggestions, unfortunately none of them worked on Windows, this seems to be a problem with inter-process communication between Vim and STDIN on Windows ... still looking for a solution
Update:Bingo. bmann and Re: Batch files in Win/Perl helped re-emphasize the long-known (yet forgotten) Windows bug ... pl2bat has a solution. Knee-jerk nauseated aversion to the words 'MSDOS' and 'batch' caused repressed memory syndrome...

20050219 Edit by castaway: Changed title from 'HOWTO: A simple, perl-based text filter for Vim'

Comment on How do I create a simple, perl-based text filter for Vim Select or Download Code

Replies are listed 'Best First'.
Re: How do I create a simple, perl-based text filter for Vim by CountZero (Bishop) on Feb 13, 2005 at 19:42 UTC
When browsing through the vim.org website I stumbled upon the following tip: Insert line numbers or filter thru perl Update: For those who are interested, a vim binary for Windows with Perl embedded can be found here. CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law	[reply]
Re^2: How do I create a simple, perl-based text filter for Vim by dimar (Curate) on Feb 13, 2005 at 20:36 UTC
Additionally, this one from vim.org also has embedded perl. (choose the windows self-installing exe). To verify embedded perl, issue the command `:echo has('perl')` you might try that command in case you already have perl-enabled VIM.	[reply] [d/l]
Re: How do I create a simple, perl-based text filter for Vim by gaal (Parson) on Feb 13, 2005 at 18:10 UTC
`#!/usr/bin/perl -wp $_ = "> $_";` [download] Make sure myfilter.pl is in your path and is executable.	[reply] [d/l]
Re: How do I create a simple, perl-based text filter for Vim by mr_mischief (Monsignor) on Feb 14, 2005 at 00:03 UTC
It works for me with vim version 6.1.320 and the code you posted. I saved the code into a file named 'filter', set the file to be executable, and called it with the relative path on the command line. Here's the command I used in vim: `:1,$!./filter` [download] The command ':!program' just runs an external program, it doesn't filter. You have to provide a range either at the colon prompt, by visual selection, or by motion selection. When I used just the ':!./filter' command as you suggest, it waited for me to type the input to the filter program, and to send it an end-of-file character. If you can't make the program executable, then you may need to use ':!perl program' instead. That works for me, too. This is really a vim question from what I can tell, as the program you provided works for me the same as any other. However, your node mentions Windows, and I don't have any form of vi on any of my Windows machines at the moment. There's no way for me to tell if there's something strange going on with a platform issue here. Christopher E. Stith	[reply] [d/l]
Re^2: How do I create a simple, perl-based text filter for Vim by dimar (Curate) on Feb 14, 2005 at 01:15 UTC
mr_mischief, Thanks for your reply, which confirms a nagging suspicion: (this is really a problem with Windows and STDIN, not perl or Vim). You have to provide a range either at the colon prompt, by visual selection, or by motion selection. Understood, the method used to provide the range (visual selection in the instant case) was intentionally edited out of the original post for simplicity sake. Moreover, that does not seem to be at issue since the 'sort' (binary) variant worked as expected, while the 'myfilter.pl' variant did not work (all other things held constant) ceteris paribus. If the range were not properly specified, both variants should have failed. your node mentions Windows, and I don't have any form of vi on any of my Windows machines at the moment. Windows appears to be the culprit here. There were some problems on Linux but those were traced to script permissions. Moreover, none of the other suggestions in this thread (so far) have worked on Windows ... all produce blank output where the stuff from STDIN should be ... the test tags appear, but not the 'filtered' text.	[reply]
Re^3: How do I create a simple, perl-based text filter for Vim by mr_mischief (Monsignor) on Feb 14, 2005 at 01:51 UTC
I'm not sure if it would be a Windows issue or a porting issue of vim, and I'd be interested to know if any of these suggestions work: as someone already mentioned, try using just `<>` instead of `<STDIN>`, which may work better with the C library on Windows, although I'm not sure why it would Try using read() in a loop instead of the bracket operator. Try playing with buffering. It shouldn't make a difference, of course, since you're getting both what's before the text and after it in the same print(). I'd consider it an unknown here, just because the results seem so bizarre. Try some simple debugging on the program, even though the source is working. Make sure your perl gets the variable filled when the script is run from the command line. You've probably already done this, but I can see how it'd be overlooked for such a simple project. If all else fails, remember that from within vim you can edit a second file, save it, etc. You could yank the needed text, ':new temp' to edit another file, put (paste) the text there, ':w' to write the file, ':!./filter temp' to change the temp file on disk, yank the changed version from that window, quit the new window, reselect the portion in the original file, and paste the buffer over the original section. Since vim has a macro recording function, I'd record parts of that mess as macros. That may be part of what you're trying to avoid, too, though. In any case, it's probably more of a pain than you want. Christopher E. Stith	[reply] [d/l] [select]
Re^3: How do I create a simple, perl-based text filter for Vim by bmann (Priest) on Feb 14, 2005 at 02:20 UTC
(this is really a problem with Windows and STDIN, not perl or Vim) That was my first thought too. Perl scripts often have difficulty with pipes on Windows. Have you tried wrapping the script you want to use as a filter with pl2bat? A quick test here works with Vim 6.3 on Win2K.	[reply]
Re: How do I create a simple, perl-based text filter for Vim by holli (Abbot) on Feb 13, 2005 at 18:10 UTC
I think that is because a perl script is no binary-executable, maybe the same reason why you can´t say `#!/some/script.pl` in a shebang-line. (I read a node that addresses this (by merlyn) somewhere but i cannot find it.) If that is really the case i would `!perl script.pl` or create a c-wrapper that calls the script. Update: martinvi is a big vi(m)? expert. He doesn´t post very often, but if you /msg him and point to this thread i am sure he can help. holli, /regexed monk/	[reply] [d/l] [select]
Re^2: How do I create a simple, perl-based text filter for Vim by neniro (Priest) on Feb 14, 2005 at 10:46 UTC
I've tried, based on your suggestion, :%!perl script.pl and it works fine with vim6.3 on WindowsXP. Calling it without involving the perl.exe doesn't work.	[reply]
Re: How do I create a simple, perl-based text filter for Vim by TilRMan (Friar) on Feb 13, 2005 at 18:50 UTC
Since I don't know how to use Vim, this is a guess. If the _IT_WORKED_ tags are showing up, then the Perl script is running. Perhaps Vim is passing perl the text as a tempfile instead of on standard input. Change `<STDIN>` to `<>` and see if that helps.	[reply] [d/l] [select]
Re: How do I create a simple, perl-based text filter for Vim by toma (Vicar) on Feb 14, 2005 at 05:21 UTC
If it's a mumble mumble windows problem `binmode();` is worth a try. It should work perfectly the first time! - toma	[reply] [d/l]
Re: How do I create a simple, perl-based text filter for Vim by Intrepid (Curate) on Feb 23, 2005 at 06:33 UTC
this thing, "inline-scripting", might be of interest. I just found it and have not tried it. When I stumbled upon it I remembered this node. Soren A / somian / perlspinr / Intrepid	[reply]