GeoLite Legacy databases are now discontinued

[toddmazierski]

As of today, MaxMind's GeoLite Legacy databases are discontinued:

https://support.maxmind.com/geolite-legacy-discontinuation-notice/

Two of the locations are now returning 404:

• https://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
• https://geolite.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz

You may want to consider updating the README to indicate this. Thanks!

[cjheath]

Thank you for the heads-up. Readme updated and pushed to Github. I don't think it's necessary to re-release the gem for this change.

[toddmazierski]

Many thanks, @cjheath!

[jdelStrother]

@cjheath I'm a bit confused by the update - the readme tells people they can download a Geolite2 database from https://dev.maxmind.com/geoip/geoip2/geolite2, but those aren't supported with GeoIP, right?

[cjheath]

Unfortunately correct, this gem has no support for the mmdb database format. Brain fart - I haven't touched this code in a long time. There are two options:

• implement support for the new indexing and database format
• implement a compiler from the CSV data to the legacy format.

It might sound bogus, but the latter is a much more well-defined problem.
I don't want to resort to a C extension (this gem was written to replace one!).
I don't know when I'll have time to read Maxmind's V2 code and re-implement it, or even if enough information is provided to make that possible. A contributor might offer to help, of course.

Any other options?

[jdelStrother]

I’ve not tried it yet, but https://github.com/yhirose/maxminddb is pure ruby and handles geoip2. If it’s any good, we could point users to that rather than maintaining a competing gem

[cjheath]

We could. But this gem has been in continuous use since 2005 (rubyforge days, before rubygems existed) and a lot of people still use it. maxminddb is MIT-licensed, so we could borrow the code with attribution. A quick scan of that code shows that the basic index structure and search algorithm is unchanged in any case, so it would also be possible to enhance the current implementation to support V2 databases.

[oschwald]

If you are thinking about implementing support, there is a public spec for the MaxMind DB format. There is also a relatively recent official pure Ruby implementation.

[cjheath]

Finally - it only took them 14 years to document and implement in Ruby! I don't feel too bad about letting mine die now, but my users might want something different.

[rtyler]

@cjheath I think it's worth wrapping the maxmind-db gem with the existing geoip API just to make it nice and easy for everybody to continue to without needing code changes.

I started to look into doing this, but I cannot figure out for the life of me how to make these tests run since the test Rake task doesn't quite work with files like test_file.rb

[cjheath]

There never has been an automated test for this gem, just a stub.
test_file.rb is a manual test, used for dumping a GeoIP file in IP order - it needs to be given the pathname to a data file.
I'd welcome a new implementation, but some tests are obviously needed first.

[rtyler]

@cjheath Am I understanding correctly, that you'd accept a new implementation based on maxminddb-reader that implemented the current GeoIP interfaces?

[cjheath]

Yes, I'd accept that. Perhaps the old code could be loaded to handle legacy files in case the maxminddb-reader doesn't?

[andyroberts]

I haven't tried this out yet, but it seems that there's been some progress on the conversion of geo-ip2 to legacy formats.

There's a python script https://github.com/sherpya/geolite2legacy

And a site distributing dat files generated, presumably with this script: https://www.miyuru.lk/geoiplegacy