monsignor has asked for the wisdom of the Perl Monks concerning the following question:
Greeting fellow monks. I am again trying my hand at Perl and hope that someone can point me in the right direction.
I found a good free IP Range to Country/City/ASN, CSV database:
https://ipinfo.io/pricing - (Partway down the page you can see the free db downloads)
The Database seems good, but that leaves the issue of how do do lookups. I'm wondering if anybody has any suggestions?
I would think that there should be code available for find an IPv4 address in a table of Low/High addresses.
I'm open to other free solutions, or some code that I can hack. Its just for personal use to help with spam filtering.
Any suggestions would be much appreciated.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: IP Lookup Tables
by afoken (Chancellor) on Mar 03, 2024 at 11:19 UTC | |
Just a little reminder: Guessing the user's location from the user's IP address is just that - guessing. I've explained one problem with IP address location years ago in Re: Help with Geo::IP output. And if you think that things got better over the past years, you are completely wrong. It got worse: Google tries to guess my location from my IP address. I'm still using the same internet provider, who is strictly local to Hamburg and the surrounding areas in Schleswig-Holstein and Lower Saxony. My IP address changes every 24 hours, simply because almost all providers in Germany do so (and demand an extra fee for a fixed IP address, or just completely refuse to provide private users with a fixed IP address). And so, every now and then, Google guesses that I'm somewhere in Ukraine, switches the user interface to Ukrainain with Cyrillic letters all over. And of course, because Youtube belongs to Google, all of the video ads are in Ukrainain. I don't mind video ads in foreign languages, because I simply bypass them. But switching the search engine to Ukrainian is really annoying, because most search results are Ukrainian and it is hard to switch the Google website back to English or German. My guess is that perhaps some of the Ukrainian refugees run VPN servers located in or around Hamburg for the people in the Ukraine, and so parts of my provider's IP range look like they are in the Ukraine. Or perhaps it's just that a lot of the Ukrainian refugees use my provider, and use Google to search in Ukrainain for information from the Ukraine. Combined with the regular change of IP addresses, every now and then I might get an IP address that had a very strong relation to the Ukraine for a few days. Of course, that guessing by Google is utterly nonsense. The IP address ranges used by my provider are used only in and around Hamburg. And that does not change, even if a lot of traffic is related to the Ukraine at the moment. Alexander
-- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-) | [reply] |
Re: IP Lookup Tables
by NERDVANA (Priest) on Mar 03, 2024 at 08:34 UTC | |
Examples:
Parsing that blob of json is fairly slow, so you probably want this to stay loaded in memory.
If you loaded this into a database, I think you'd get decent performance from That should follow the index straight to the record you want, then stop iterating as soon as it finds it. | [reply] [d/l] [select] |
by monsignor (Acolyte) on Mar 30, 2024 at 15:56 UTC | |
I just wanted to say a quick thank you for all the input. I did some experimenting, and for my use case I ended up with an sqlite3 database that I built myself from the csv tables. It took way too long to ingest the data every time I needed it, and I have sqlinte3 on the small box that I want to run this. Using an external call takes to long, and could potentially rate limit. I put a primary index using the first octet, and that made the search way faster. | [reply] |
Re: IP Lookup Tables
by marto (Cardinal) on Mar 03, 2024 at 08:11 UTC | |
Geo::IPinfo, The official Perl library for IPinfo(.io). See also Re: Help with Geo::IP output. | [reply] |
Re: IP Lookup Tables
by Tux (Canon) on Mar 07, 2024 at 13:04 UTC | |
What might help if you *do* have a local postgres database, is App::geoip, a tool that uses GeoIP2 data from MaxMind ... (note that you need a free MaxMind account for which you need te enter the data in ~/.config/geoip. Your own location is optional, but when available, the tool can show the distance)
Enjoy, Have FUN! H.Merijn | [reply] [d/l] |
Re: IP Lookup Tables
by cavac (Prior) on Mar 07, 2024 at 08:50 UTC | |
I pull GeoIP data into a PostgreSQL database and do the lookups from there. Here's my writeup from 2018 (the software still works fine on my system, no guarantees though): GeoIP revisited PostgreSQL is nice for this, since it supports CIDR address ranges, you can do stuff like this: SELECT country_code, city_name, latitude, longitude, radius FROM geoip WHERE ? << netblock LIMIT 1 With the proper GIST index, it works reasonably fast, too. CREATE INDEX geoip_netblock_idx ON geoip USING GIST(netblock inet_ops);
PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
| [reply] [d/l] [select] |