« 既製品の管理ツールを使わないことでウェブサービスの TCO を下げる話について hbstudy#8 で話してきた件 | Main | Japanize の翻訳データ更新障害を解消しました »

March 25, 2010

Q4M 0.9.3 prerelease (with support for "concurrent compaction")

Q4M (Queue for MySQL) periodically performs an operation called "compaction", which is sort of a garbage collection, that collects empty space from a queue file and returns to the OS.

The pitfall that exists until now was that during compaction, all operation on the queue table was being blocked.

My opinion was (is) that it is not a serious problem for most users, since the time required for compaction will be small in most cases (the time depends on the number (and size) of the rows alive on the queue table, and the number of the rows alive will be mostly small).

But for usecases where fast response is a requirement, I have added a "queue_use_concurrent_compaction" option to Q4M in the 0.9.3 prerelease.  When the variable is set to one in my.cnf, INSERTs will not be blocked during compaction.  Another configuration variable queue_concurrent_compaction_interval is also available to fine-tune response time of INSERTs during compaction.  The response of INSERTs during compaction will become faster as you set the variable smaller, although the compaction will become slower as a side effect.

my.cnf
# enable / disable concurrent compaction (0: disabled (default), 1: enabled)
queue_use_concurrent_compaction=1

# handle INSERTs for every N bytes of data is compacted (default: 1048576)
queue_concurrent_compaction_interval=1048576

If you are already using Q4M without any problems, I recommend sticking to the version you are using, since the stability of Q4M might have degraded due to the introduction of the feature.

On the other hand if you were having problems with the issue or planning to use Q4M for a new application, I recommend using this release.  Have fun!  And if you find any problems, please send stacktraces to me :-)

TrackBack

TrackBack URL for this entry:
http://bb.lekumo.jp/t/trackback/404050/23691966

Listed below are links to weblogs that reference Q4M 0.9.3 prerelease (with support for "concurrent compaction"):

Comments

Post a comment