unless’s blog

日々のちょっとした技術的なことの羅列

[Perl] [GeoIP2] MaxMind::DB::Reader::XS と MaxMind::DB::Reader での速度比較

Perl GeoIP2 benchmark docker sentos6 epel geoip geoip2

GeoIP2 とは

下記にあるようにipアドレスに紐づく地域やISP等が格納されているMaxMind社が提供するDatabase www.maxmind.com

概要

perlの公式APIのREADMEに下記のような記載があり本当に100倍も速度が違うのか気になったので速度を比較してみる

github.com

The XS implementation is approximately 100x faster than the pure Perl implementation.

環境準備

とりあえずdockerで速度を比較できる環境を準備する
ほんとこういう時にサクッと環境作れるdockerは偉大だなってつくづく思う

docker pull

sudo docker pull centos:centos6
centos6: Pulling from library/centos
~~~ (ry ~~~
Status: Downloaded newer image for centos:centos6
  • sentos6なことに特に意味はない
  • 一番慣れ親しんでるだけで今のご時世ならsentos7にすべきかなって思いながらサボった

docker run

docker run -ti -d --name geoip2 centos:centos6

perlのinstall

perl入ってなかったのでインストール

# yum info perl
Loaded plugins: fastestmirror, ovl
base                                                                                                                                                                                 | 3.7 kB     00:00
base/primary_db                                                                                                                                                                      | 4.7 MB     00:01
extras                                                                                                                                                                               | 3.4 kB     00:00
extras/primary_db                                                                                                                                                                    |  27 kB     00:00
updates                                                                                                                                                                              | 3.4 kB     00:00
updates/primary_db                                                                                                                                                                   | 3.0 MB     00:04
Available Packages
Name        : perl
Arch        : x86_64
Epoch       : 4
Version     : 5.10.1
Release     : 144.el6
Size        : 10 M
Repo        : base
Summary     : Practical Extraction and Report Language
URL         : http://www.perl.org/
License     : (GPL+ or Artistic) and (GPLv2+ or Artistic) and Copyright Only and MIT and Public Domain and UCD
Description : Perl is a high-level programming language with roots in C, sed, awk
            : and shell scripting.  Perl is good at handling processes and files,
            : and is especially good at handling text.  Perl's hallmarks are
            : practicality and efficiency.  While it is used to do a lot of
            : different things, Perl's most common applications are system
            : administration utilities and web programming.  A large proportion of
            : the CGI scripts on the web are written in Perl.  You need the perl
            : package installed on your system so that your system can handle Perl
            : scripts.
            :
            : Install this package if you want to program in Perl or enable your
            : system to handle Perl scripts.

とりあえず諸々install

# yum -y install perl gcc make patch
# curl -Lk  http://xrl.us/perlbrewinstall | bash
# echo " source ~/perl5/perlbrew/etc/bashrc " >> ~/.bashrc
source ~/perl5/perlbrew/etc/bashrc

perlbrewで使えるversionの確認

[root@c6cc159d9e64 /]# perlbrew available

   perl-5.29.7
   perl-5.28.1
   perl-5.28.0
   perl-5.26.3
   perl-5.26.2
   perl-5.26.1
   perl-5.26.0
   perl-5.24.4
   perl-5.24.3
~~~ (ry ~~~

とりあえずperl-5.28.1をinstall(結構な時間が掛かる)
この辺でなぜperlのimageを使わなかったのか激しく後悔したがここまできたのでそのまま進める

# perlbrew install perl-5.28.1
Installing /root/perl5/perlbrew/build/perl-5.28.1/perl-5.28.1 into ~/perl5/perlbrew/perls/perl-5.28.1

This could take a while. You can run the following command on another shell to track the status:

  tail -f ~/perl5/perlbrew/build.perl-5.28.1.log

perl-5.28.1 is successfully installed.

switch

# perlbrew switch perl-5.28.1

cpanm

# perlbrew install-cpanm

geoip2のinstall

database

# yum -y install wget gunzip openssl-devel
# mkdir gepip; cd gepip/
# wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz
# gunzip GeoLite2-Country.mmdb.gz
# wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
# gunzip GeoLite2-City.mmdb.gz

perl module

# mkdir extlib; cd extlib
cpanm -L . GeoIP2

試しに実行

# perl -Mlocal::lib=/root/extlib/ -MGeoIP2::Database::Reader -E 'my $reader = GeoIP2::Database::Reader->new( file => "/root/gepip/GeoLite2-City.mmdb", locales => ["en"] ); my $city = $reader->city( ip => "24\.24\.24\.24" ); my $country = $city->country(); say $country->iso_code()'
US

libmaxminddb

下記に記載の通り、libmaxminddbとMaxMind::DB::Reader::XSが必要なのでinstall github.com

epel repo の追加

# yum install epel-release

libmaxminddbのinstalll

# yum install libmaxminddb

速度比較用に別ディレクトリにGeoIP2とMaxMind::DB::Reader::XSをinstall

# mkdir extlib-xs; cd extlib-xs
cpanm -L . GeoIP2
cpanm -L . MaxMind::DB::Reader::XS

速度比較

簡易的なscriptを組んでbenchmarkを取ってみる

use strict;
use warnings;
use Benchmark qw(:all);
use GeoIP2::Database::Reader;

timethese (10000,
    {
        geoip => sub {
            my $reader = GeoIP2::Database::Reader->new( file => "/root/geoip/GeoLite2-City.mmdb", locales => ["en"] );
            my $city = $reader->city( ip => "24.24.24.24" );
            my $country = $city->country();
            $country->iso_code();
            $country->name();
        },
    }
);

実行結果

# perl -Mlocal::lib=/root/extlib benchmark.pl
Benchmark: timing 10000 iterations of geoip...
     geoip: 164 wallclock secs (141.50 usr + 21.71 sys = 163.21 CPU) @ 61.27/s (n=10000)
# perl -Mlocal::lib=/root/extlib-xs benchmark.pl
Benchmark: timing 10000 iterations of geoip...
     geoip:  6 wallclock secs ( 4.83 usr +  0.69 sys =  5.52 CPU) @ 1811.59/s (n=10000)

結論

100倍どころじゃなく速かったのではやりXSはすごいなぁって印象(当たり前だけども)