Testing out Percona and HandlerSocket with AvroBase

Testing out Percona and HandlerSocket with AvroBase

The great thing about the AvroBase interface is that it can be used with many different datastores because of its relatively low requirements on those stores. HandlerSocket is a plugin for MySQL that gives you direct access to the low level APIs that let you do the operations that AvroBase requires without going through the high-level SQL APIs and therefore skip a lot of boilerplate parsing that happens for every single lookup or scan. You can read about the HandlerSocket benchmarks here. Percona Server is just a better implementation of MySQL with its own modified InnoDB engine called XtraDB. In combination you should be able to get even higher numbers of transactions per second for individual servers in a cluster. Building it on the Mac (or any unix system) is pretty straight-forward.

cd /Users/sam/Software/Percona-Server-5.1.53/
./configure — prefix=/Users/sam/usr
make && make install

Similarly, download the HandlerSocket source via Git and then build it:

git clone https://github.com/ahiguti/HandlerSocket-Plugin-for-MySQL.git
cd HandlerSocket-Plugin-for-MySQL/
sh autogen.sh
./configure — prefix=/users/sam/usr — with-mysql-source=/Users/sam/Software/Percona-Server-5.1.53 — with-mysql-bindir=/users/sam/usr/bin — with-mysql-plugindir=/users/sam/usr/lib/mysql/plugin
make && make install

You then need to make some changes to the my.cnf file:

user = sam
port = 3406
loose_handlersocket_port = 9998
loose_handlersocket_port_wr = 9999
loose_handlersocket_threads = 16
loose_handlersocket_threads_wr = 1

Start up mysql for the first time:

export PATH=~/usr/bin:$PATH
cd /Users/sam/usr ; /Users/sam/usr/bin/mysqld_safe &

Then connect to mysql and install the plugin:

mysql -u root
install plugin handlersocket soname ‘handlersocket.so’;

For AvroBase I am using the native i/o Java client:

svn checkout http://hs4j.googlecode.com/svn/trunk/ hs4j
mvn install -Dmaven.test.skip=true

Haven’t fully put it through its paces but I expect to do some benchmarking here shortly. Because of the amount of work to deserialize objects vs the database access my guess is that it won’t make a huge different in terms of qps for a single front-end. However, it will likely let us go to a much higher number of frontends before accessing the database becomes the bottleneck.