RegEx for users who dont know RegEx

dimar has asked for the wisdom of the Perl Monks concerning the following question:

Goal: There is a simple script that searches through flat text files based on a simple 'query string.' The query string is supplied by the user. The goal is to allow the user to supply 'wildcards' in her query string. The syntax for queries is very minimal (i.e., dumb flat search for a string with no fancy boolean operators, no filtering capabilities, no letter-case distinctions).

Question:What is the best way to process a user-supplied search 'query' such that a user is allowed to include simple wildcard characters. (eg not full-blown RegEx) For example, if the user supplies:

i*ation
[download]

The script should return all matching items in the flat file:

invitation
information
Isolation
InFlaTiOn
IATION
[download]

But the script should not return:

In our nation
it requires concentration
fiat ionizing
[download]

Is the best approach simply to take the user input and translate that into a full-blown RegEx, and then use that? Devising a RegEx that meets the requirement is no problem, but it seems like a potential red-flag to be post processing a user-supplied into a RegEx for some reason. Perhaps there is a better way to approach this.

Update -- modified example positive match that was intended to represent a false-positive match; re: The Mad Hatter

Comment on RegEx for users who dont know RegEx Select or Download Code

Replies are listed 'Best First'.
Re: RegEx for users who dont know RegEx by ikegami (Patriarch) on Dec 23, 2004 at 17:23 UTC
Something like this? `($reg_exp = $user_query) =~ s/(\W)/$1 eq '' ? "\\S" : "\\$1"/ge;` [download] It can be used as follows: `# File as an array of lines: @matching_lines = grep { /$reg_exp/ } @lines_to_search;` [download] or `# File in a scalar: @matching_lines = $file =~ /^(.${reg_exp}.)$/mg;` [download]	[reply] [d/l] [select]
Re^2: RegEx for users who dont know RegEx by The Mad Hatter (Priest) on Dec 23, 2004 at 17:36 UTC
`($reg_exp = $user_query) =~ s/(\W)/$1 eq '' ? "\\S" : "\\\$1"/ge;` [download] You have too many slashes in the second part of that ternary (results in invalid chars being replaced by a literal '\$1').	[reply] [d/l]
Re^3: RegEx for users who dont know RegEx by ikegami (Patriarch) on Dec 23, 2004 at 17:39 UTC
Thanks. Fixed. Tested.	[reply]
Re: RegEx for users who dont know RegEx by The Mad Hatter (Priest) on Dec 23, 2004 at 17:18 UTC
One solution is to strip out all characters in the user-supplied data that aren't explicitly allowed and then generate your regex based off of that. Update -- Try this sample code: `#!/usr/bin/perl use warnings; use strict; my $query = shift; die "usage: $0 query-string\n" if not $query; print "Original query: '$query'\n"; $query =~ s/[^\w\]//g; print "Safe query: '$query'\n"; $query =~ s/\/\\w\*/g; print "Parsed query: '$query'\n"; while (<DATA>) { print "match: $_" if /$query/i; } __DATA__ invitation information Isolation InFlaTiOn IATION In our nation it requires concentration at ionizing radiation` [download] Note that "at ionizing radiation" matches because the iation in radiation matches. Did you just miss that, or should it not match?	[reply] [d/l]
Re^2: RegEx for users who dont know RegEx by ikegami (Patriarch) on Dec 23, 2004 at 17:30 UTC
Why strip out the unsafe characters instead of escaping them, as I did below? Using your approach to search for "can't" will fail, for example.	[reply]
Re^3: RegEx for users who dont know RegEx by The Mad Hatter (Priest) on Dec 23, 2004 at 17:39 UTC
shrugs Stripping out unsafe characters just seems like a better idea; you know exactly what you're left with. If you want to allow single ticks or other characters, modifying the regex of allowed chars is easy. The code above is just an example, not a final product.	[reply]
Re: RegEx for users who dont know RegEx by inman (Curate) on Dec 24, 2004 at 12:29 UTC
Just quotemeta the search string then substitute any characters that you want to use as wildcards. I have used * and ? #! /usr/bin/perl -w use strict; use warnings; $/ = undef; my $text = <DATA>; my $count = 1; my $searchRegex = quotemeta $ARGV[0];; $searchRegex =~ s/\\\/\\S/g; $searchRegex =~ s/\\\?/\\S{1}/g; print "Search RegEx = $searchRegex\n"; $text =~ s/($searchRegex)\b/print $count++, " $1\n"/ige; __DATA__ can't won't This is a selection of example passages to find We received an invitation from Mark to his party He provided information on how to get to his house. He has been in the isolation ward for much of the year due to some unfortunate bladder InFlaTiOn. Let's hope he gets better! IATION [download]	[reply] [d/l]