最近两年NoSQL发展如火如荼,国内众多门户及新兴的WEB2.0网站都在利用各种开源NoSQL技术来解决实际问题。根据应用场景的不同,有各种各样对应的NoSQL产品,比如说用于列式存储及分布式计算的有Hadoop项目中的Hbase,FaceBook的Cassandra,GreenPlum;用于文档存储类的包括CouchDB、MongoDB等;高可用读写的Key Value方案更是不胜枚举,包括MemcacheDB、Tokyo Cabinet/Tyrant等等。

  由于关系型数据库自身的局限,某些应用场景下NoSQL方案确实比MySQL这种关系型数据库要快很多倍,不过,需要正确理解的是,所谓NoSQL并非No SQL,而是Not Only SQL,因为目前来看流行的NoSQL产品对于简单查询的处理比较高效,但是稍复杂的关联查询,条件过滤则无能为力了,当前想要完全废弃常规的关系数据库也不现实。因此,流行环境中多数都是MySQL+NoSQL并用。

  本文介绍的HandlerSocket相当于是种折中的方案,它做为一个MySQL的引擎插件安装之后,可以访问内存的物理存储中的数据,由于直接调用MySQL存储引擎的API,跳过了SQL解析层,直接访问数据,性能很不错,因为它是做为MySQL的插件存在,因此也不会影响正常的MySQL数据库功能,当需要基于k/v方式快速高效获取数据时,就可以选择HandlerSocket,当需要通过稍复杂的关系运算获取数据时,仍可以使用标准的SELECT方式。

  本文主要描述Linux平台HandlerSocket安装和配置,及简单的性能对比测试。

  HandlerSocket插件可至其官网下载:

https://github.com/DeNADev/HandlerSocket-Plugin-for-MySQL/downloads

  目前下载到的最新版本为:1.1.0-2

  由于HandlerSocket是基于MySQL数据库的,因此在安装HanderSocket前需要先按照常规方式部署MySQL服务,这个具体过程略,如果没有安装过MySQL的朋友可以参考下列网址:Linux环境下源码编译安装MySQL5.5

  注意安装HandlerSocket时需要MySQL的源码,对于rpm包方式安装的MySQL环境,还需要下载MySQL的源代码。

1、安装HandlerSocket插件

  源码编译方式安装非常简单,解压下载到的压缩包,执行configure即可:

    # tar xvfz DeNADev-HandlerSocket-Plugin-for-MySQL-1.1.0-2-g395fa55.tar.gz

    # cd DeNADev-HandlerSocket-Plugin-for-MySQL-395fa55/

    # ./autogen.sh

    # ./configure --with-mysql-source=/data/software/mysql-5.5.12 \

    --with-mysql-bindir=/usr/local/mysql55/bin \

    --with-mysql-plugindir=/usr/local/mysql55/lib/plugin

其中:

  • --with-mysql-source:指定MySQL源码所在目录
  • --with-mysql-bindir:指定mysql_config所在目录
  • --with-mysql-plugindir:指定MySQL插件的存储路径

执行make编译安装:

    #make && make install

完成后,mysql-plugindir目录下应有handlersocket相关文件:

    # ll /usr/local/mysql55/lib/plugin/handlersocket*

    -rw-r--r-- 1 root root 3557360 Mar 28 15:01 /usr/local/mysql55/lib/plugin/handlersocket.a

    -rwxr-xr-x 1 root root 978 Mar 28 15:01 /usr/local/mysql55/lib/plugin/handlersocket.la

    lrwxrwxrwx 1 root root 22 Mar 28 15:01 /usr/local/mysql55/lib/plugin/handlersocket.so -> handlersocket.so.0.0.0

    lrwxrwxrwx 1 root root 22 Mar 28 15:01 /usr/local/mysql55/lib/plugin/handlersocket.so.0 -> handlersocket.so.0.0.0

    -rwxr-xr-x 1 root root 1733353 Mar 28 15:01 /usr/local/mysql55/lib/plugin/handlersocket.so.0.0.0

软件的安全就完成了。接下来需要对MySQL进行配置。

2、MySQL中进行配置

编辑MySQL的初始化参数文件my.cnf,增加下列配置:

    #*********** HandlerSocket setting ***********

    loose_handlersocket_port = 9998

    # the port number to bind to (for read requests)

    loose_handlersocket_port_wr = 9999

    # the port number to bind to (for write requests)

    loose_handlersocket_threads = 16

    # the number of worker threads (for read requests)

    loose_handlersocket_threads_wr = 1

    # the number of worker threads (for write requests)

    open_files_limit = 65535

    # to allow handlersocket accept many concurrent

    # connections, make open_files_limit as large as

    # possible.

几个参数注释说的很清楚,就不一一解释用途了。

启动MySQL服务:

    # mysqld_safe --defaults-file=/data/mysqldata/3306/my.cnf &

在MySQL中安装HandlerSocket插件:

    mysql> install plugin handlersocket soname ¨handlersocket.so¨;

    Query OK, 0 rows affected (0.01 sec)

而后查看当前的线程列表,应该就能看handlersocket的17(1写16读)个线程了:

    mysql> show processlist;

    +----+-------------+-----------------+---------------+---------+------+-------------------------------------------+------------------+

    | Id | User | Host | db | Command | Time | State | Info |

    +----+-------------+-----------------+---------------+---------+------+-------------------------------------------+------------------+

    | 1 | root | localhost | NULL | Query | 0 | NULL | show processlist |

    | 2 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 3 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 4 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 5 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 6 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 7 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 8 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 9 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 10 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 11 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 12 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 13 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 14 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 15 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 16 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 17 | system user | connecting host | NULL | Connect | NULL | handlersocket: mode=rd, 0 conns, 0 active | NULL |

    | 18 | system user | connecting host | handlersocket | Connect | NULL | handlersocket: mode=wr, 0 conns, 0 active | NULL |

    +----+-------------+-----------------+---------------+---------+------+-------------------------------------------+------------------+

通过netstat查看端口:

    [root@mysql1 3306]# netstat -lnt

    Active Internet connections (only servers)

    Proto Recv-Q Send-Q Local Address Foreign Address State

    ........

    ........

    tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN

    tcp 0 0 0.0.0.0:9998 0.0.0.0:* LISTEN

    tcp 0 0 0.0.0.0:9999 0.0.0.0:* LISTEN

至此,HandlerSocket插件安装完毕,可以向前端应用提供服务了。