Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^5: [OT] Re: Get unique fields from file

by Marshall (Canon)
on Jan 11, 2022 at 20:14 UTC ( #11140387=note: print w/replies, xml ) Need Help??


in reply to Re^4: [OT] Re: Get unique fields from file
in thread Get unique fields from file

Well....I don't know about the postgres foundation for SQLite...check this interesting story out from Richard Hipp: Origin of SQLite

My SQL-Foo is weak. OMG this can get complicated!
As an example, I present jaro-winkler.sql - this is so far above my SQL pay grade, I get a nose bleed just thinking about it!:

-- ------------------------------------------------------------------- +------------- -- Routine DDL -- ------------------------------------------------------------------- +------------- -- from https://androidaddicted.wordpress.com/ DELIMITER $$ CREATE DEFINER=`root`@`localhost` FUNCTION `jaro_winkler_similarity`( in1 varchar(255), in2 varchar(255) ) RETURNS float DETERMINISTIC BEGIN #finestra:= search window, curString:= scanning cursor for the origina +l string, curSub:= scanning cursor for the compared string declare finestra, curString, curSub, maxSub, trasposizioni, prefixlen, + maxPrefix int; declare char1, char2 char(1); declare common1, common2, old1, old2 varchar(255); declare trovato boolean; declare returnValue, jaro float; set maxPrefix=6; #from the original jaro - winkler algorithm set common1=""; set common2=""; set finestra=(length(in1)+length(in2)-abs(length(in1)-length(in2))) DI +V 4 + ((length(in1)+length(in2)-abs(length(in1)-length(in2)))/2) mod 2; set old1=in1; set old2=in2; #calculating common letters vectors set curString=1; while curString<=length(in1) and (curString<=(length(in2)+finestra)) d +o set curSub=curstring-finestra; if (curSub)<1 then set curSub=1; end if; set maxSub=curstring+finestra; if (maxSub)>length(in2) then set maxSub=length(in2); end if; set trovato = false; while curSub<=maxSub and trovato=false do if substr(in1,curString,1)=substr(in2,curSub,1) then set common1 = concat(common1,substr(in1,curString,1)); set in2 = concat(substr(in2,1,curSub-1),concat("0",substr(in2,curSub+1 +,length(in2)-curSub+1))); set trovato=true; end if; set curSub=curSub+1; end while; set curString=curString+1; end while; #back to the original string set in2=old2; set curString=1; while curString<=length(in2) and (curString<=(length(in1)+finestra)) d +o set curSub=curstring-finestra; if (curSub)<1 then set curSub=1; end if; set maxSub=curstring+finestra; if (maxSub)>length(in1) then set maxSub=length(in1); end if; set trovato = false; while curSub<=maxSub and trovato=false do if substr(in2,curString,1)=substr(in1,curSub,1) then set common2 = concat(common2,substr(in2,curString,1)); set in1 = concat(substr(in1,1,curSub-1),concat("0",substr(in1,curSub+1 +,length(in1)-curSub+1))); set trovato=true; end if; set curSub=curSub+1; end while; set curString=curString+1; end while; #back to the original string set in1=old1; #calculating jaro metric if length(common1)<>length(common2) then set jaro=0; elseif length(common1)=0 or length(common2)=0 then set jaro=0; else #calcolo la distanza di winkler #passo 1: calcolo le trasposizioni set trasposizioni=0; set curString=1; while curString<=length(common1) do if(substr(common1,curString,1)<>substr(common2,curString,1)) then set trasposizioni=trasposizioni+1; end if; set curString=curString+1; end while; set jaro= ( length(common1)/length(in1)+ length(common2)/length(in2)+ (length(common1)-trasposizioni/2)/length(common1) )/3; end if; #end if for jaro metric #calculating common prefix for winkler metric set prefixlen=0; while (substring(in1,prefixlen+1,1)=substring(in2,prefixlen+1,1)) and +(prefixlen<6) do set prefixlen= prefixlen+1; end while; #calculate jaro-winkler metric return jaro+(prefixlen*0.1*(1-jaro)); END $$ delimiter ;

I have a tutorial on postgres that I am slowly working my way through. SQL is a complex subject. Admin of a DB is a complex subject. I have certain tasks that I want to learn how to do in SQL. But this is very much a "work in progress". I am now able to do simple SQL tasks and I have found that the combo of Perl + SQL is super powerful. SQLite is my "go to" DB because there is no admin or installation hassle.

I like this Hipp quote: "Fortunately, I didn’t know any experts — and so I did it anyway.”

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11140387]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (2)
As of 2022-07-06 00:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?