SSDB-Evolution - A fast NoSQL database, derived from SSDB(https://github.com/ideawu/ssdb), for storing big list of data

Compared with SSDB, SSDB-Evolution has an more comprehensive and better support for redis commands. Most of the codes are rewrited to achieve the improvements listed below.

Improvements

• Fix some bugs and incompatible implementations
• Full support expiration for all kinds of structs
• Re-implement replication with binlog
• Hot sharding
• Full support DEL for all kinds of sturcts
• Splits metadata with userdata
• Re-implement low-level encoding/decoding for sharding

Redis Commands

Keys

+-------------------+------------+-------------------------------------------------------------+
|      Command      | Supported? | Format                                                      |
+-------------------+------------+-------------------------------------------------------------+
|        DEL        |    Yes     | DEL key [key …]                                            |
+-------------------+------------+-------------------------------------------------------------+
|       DUMP        |    No      | DUMP key                                                    |
+-------------------+------------+-------------------------------------------------------------+
|      EXISTS       |    Yes     | EXISTS key                                                  |
+-------------------+------------+-------------------------------------------------------------+
|      EXPIRE       |    Yes     | EXPIRE key seconds|                                         |
+-------------------+------------+-------------------------------------------------------------+
|     EXPIREAT      |    Yes     | EXPIREAT key timestamp|                                     |
+-------------------+------------+-------------------------------------------------------------+
|       KEYS        |    No      | KEYS pattern|                                               |
+-------------------+------------+-------------------------------------------------------------+
|      MIGRATE      |    No      | MIGRATE host port key destination-db timeout|               |
+-------------------+------------+-------------------------------------------------------------+
|       MOVE        |    No      | MOVE key db|                                                |
+-------------------+------------+-------------------------------------------------------------+
|      OBJECT       |    No      | OBJECT subcommand [arguments [arguments …]]|                |
+-------------------+------------+-------------------------------------------------------------+
|      PERSIST      |    No      | PERSIST key|                                                |
+-------------------+------------+-------------------------------------------------------------+
|      PEXPIRE      |    Yes     | PEXPIRE key milliseconds|                                   |
+-------------------+------------+-------------------------------------------------------------+
|     PEXPIREAT     |    Yes     | PEXPIREAT key milliseconds-timestamp|                       |
+-------------------+------------+-------------------------------------------------------------+
|      PTTL         |    No      | PTTL key|                                                   |
+-------------------+------------+-------------------------------------------------------------+
|     RANDOMKEY     |    No      | RANDOMKEY|                                                  |
+-------------------+------------+-------------------------------------------------------------+
|      RENAME       |    No      | RENAME key newkey|                                          |
+-------------------+------------+-------------------------------------------------------------+
|     RENAMENX      |    No      | RENAMENX key newkey|                                        |
+-------------------+------------+-------------------------------------------------------------+
|      RESTORE      |    No      | RESTORE key ttl serialized-value|                           |
+-------------------+------------+-------------------------------------------------------------+
|      SORT         |    No    | SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]   |
+-------------------+------------+-------------------------------------------------------------+
|       TTL         |    Yes     | TTL key|                                                    |
+-------------------+------------+--------------------------------——----------------------------+
|      TYPE         |    No      | TYPE key|                                                   |
+-------------------+------------+-------------------------------------------------------------+
|      SCAN         |    No      | SCAN cursor [MATCH pattern] [COUNT count]|                  |
+-------------------+------------+-------------------------------------------------------------+

Strings

+-------------------+------------+------------------------------------------------------------+
|      Command      | Supported? | Format                                                     |
+-------------------+------------+------------------------------------------------------------+
|       APPEND      |    No      | APPEND key value                                           |
+-------------------+------------+------------------------------------------------------------+
|      BITCOUNT     |    Yes     | BITCOUNT key [start] [end]                                 |
+-------------------+------------+------------------------------------------------------------+
|       BITOP       |    No      | BITOP operation destkey key [key ...]                      |
+-------------------+------------+------------------------------------------------------------+
|       DECR        |    Yes     | DECR key                                                   |
+-------------------+------------+------------------------------------------------------------+
|      DECRBY       |    Yes     | DECRBY key decrement                                       |
+-------------------+------------+------------------------------------------------------------+
|       GET         |    Yes     | GET key                                                    |
+-------------------+------------+------------------------------------------------------------+
|      GETBIT       |    Yes     | GETBIT key offset                                          |
+-------------------+------------+------------------------------------------------------------+
|     GETRANGE      |    No      | GETRANGE key start end                                     |
+-------------------+------------+------------------------------------------------------------+
|      GETSET       |    Yes     | GETSET key value                                           |
+-------------------+------------+------------------------------------------------------------+
|      INCR         |    Yes     | INCR key                                                   |
+-------------------+------------+------------------------------------------------------------+
|      INCRBY       |    Yes     | INCRBY key increment                                       |
+-------------------+------------+------------------------------------------------------------+
|     INCRBYFLOAT   |    No      | INCRBYFLOAT key increment                                  |
+-------------------+------------+------------------------------------------------------------+
|      MGET         |    Yes     | MGET key [key ...]                                         |
+-------------------+------------+------------------------------------------------------------+
|      MSET         |    Yes     | MSET key value [key value ...]                             |
+-------------------+------------+------------------------------------------------------------+
|      MSETNX       |    No      | MSETNX key value [key value ...]                           |
+-------------------+------------+------------------------------------------------------------+
|      PSETEX       |    No      | PSETEX key milliseconds value                              |
+-------------------+------------+------------------------------------------------------------+
|      SET          |    Yes     | SET key value [EX seconds] [PX milliseconds] [NX|XX]       |
+-------------------+------------+------------------------------------------------------------+
|      SETBIT       |    Yes     | SETBIT key offset value                                    |
+-------------------+------------+------------------------------------------------------------+
|      SETEX        |    Yes     | SETEX key seconds value                                    |
+-------------------+------------+------------------------------------------------------------+
|      SETNX        |    Yes     | SETNX key value                                            |
+-------------------+------------+------------------------------------------------------------+
|      SETRANGE     |    No     | SETRANGE key offset value                                  |
+-------------------+------------+------------------------------------------------------------+
|      STRLEN       |    No     | STRLEN key                                                 |
+-------------------+------------+------------------------------------------------------------+

Lists

+-------------------+------------+------------------------------------------------------------+
|      Command      | Supported? | Format|                                                     |
+-------------------+------------+------------------------------------------------------------+
|       BLPOP       |    No      | BLPOP key [key ...] timeout|                                |
+-------------------+------------+------------------------------------------------------------+
|       BRPOP       |    No      | BRPOP key [key ...] timeout|                                |
+-------------------+------------+------------------------------------------------------------+
|     BRPOPLPUSH    |    No      | BRPOPLPUSH source destination timeout|                      |
+-------------------+------------+------------------------------------------------------------+
|      LINDEX       |    Yes     | LINDEX key index|                                           |
+-------------------+------------+------------------------------------------------------------+
|      LINSERT      |    Yes     | LINSERT key BEFORE|AFTER pivot value|                       |
+-------------------+------------+------------------------------------------------------------+
|      LLEN         |    Yes     | LLEN key|                                                   |
+-------------------+------------+------------------------------------------------------------+
|      LPOP         |    Yes     | LPOP key|                                                   |
+-------------------+------------+------------------------------------------------------------+
|      LPUSH        |    Yes     | LPUSH key value [value ...]|                                |
+-------------------+------------+------------------------------------------------------------+
|      LPUSHX       |    Yes     | LPUSHX key value|                                           |
+-------------------+------------+------------------------------------------------------------+
|      LRANGE       |    Yes     | LRANGE key start stop|                                      |
+-------------------+------------+------------------------------------------------------------+
|      LREM         |    Yes     | LREM key count value|                                       |
+-------------------+------------+------------------------------------------------------------+
|      LSET         |    Yes     | LSET key index value|                                       |
+-------------------+------------+------------------------------------------------------------+
|      LTRIM        |    Yes     | LTRIM key start stop|                                       |
+-------------------+------------+------------------------------------------------------------+
|      RPOP         |    Yes     | RPOP key|                                                   |
+-------------------+------------+------------------------------------------------------------+
|     RPOPLPUSH     |    No      | RPOPLPUSH source destination|                               |
+-------------------+------------+------------------------------------------------------------+
|      RPUSH        |    Yes     | RPUSH key value [value ...]|                                |
+-------------------+------------+------------------------------------------------------------+
|      RPUSHX       |    No      | RPUSHX key value|                                           |
+-------------------+------------+------------------------------------------------------------+

Hashs

+-------------------+------------+------------------------------------------------------------+
|      Command      | Supported? | Format                                                     |
+-------------------+------------+------------------------------------------------------------+
|       HDEL        |    Yes     | HDEL key field [field ...]                                 |
+-------------------+------------+------------------------------------------------------------+
|      HEXISTS      |    Yes     | HEXISTS key field                                          |
+-------------------+------------+------------------------------------------------------------+
|       HGET        |    Yes     | HGET key field                                             |
+-------------------+------------+------------------------------------------------------------+
|      HGETALL      |    Yes     | HGETALL key                                                |
+-------------------+------------+------------------------------------------------------------+
|      HINCRBY      |    Yes     | HINCRBY key field increment                                |
+-------------------+------------+------------------------------------------------------------+
|    HINCRBYFLOAT   |    No      | HINCRBYFLOAT key field increment                           |
+-------------------+------------+------------------------------------------------------------+
|      HKEYS        |    Yes     | HKEYS key                                                  |
+-------------------+------------+------------------------------------------------------------+
|      HLEN         |    Yes     | HLEN key                                                   |
+-------------------+------------+------------------------------------------------------------+
|      HMGET        |    Yes     | HMGET key field [field ...]                                |
+-------------------+------------+------------------------------------------------------------+
|      HMSET        |    Yes     | HMSET key field value [field value ...]                    |
+-------------------+------------+------------------------------------------------------------+
|      HSET         |    Yes     | HSET key field value                                       |
+-------------------+------------+------------------------------------------------------------+
|      HSETNX       |    No      | HSETNX key field value                                     |
+-------------------+------------+------------------------------------------------------------+
|      HVALS        |    Yes     | HVALS key                                                  |
+-------------------+------------+------------------------------------------------------------+
|      HSCAN        |    No      | HSCAN key cursor [MATCH pattern] [COUNT count]             |
+-------------------+------------+------------------------------------------------------------+

Sets

+-------------------+------------+------------------------------------------------------------+
|      Command      | Supported? | Format                                                     |
+-------------------+------------+------------------------------------------------------------+
|      SADD         |    Yes     | SADD key member [member ...]                               |
+-------------------+------------+------------------------------------------------------------+
|      SCARD        |    Yes     | SCARD key                                                  |
+-------------------+------------+------------------------------------------------------------+
|      SDIFF        |    No      | SDIFF key [key ...]                                        |
+-------------------+------------+------------------------------------------------------------+
|     SDIFFSTORE    |    No      | SDIFFSTORE destination key [key ...]                       |
+-------------------+------------+------------------------------------------------------------+
|      SINTER       |    No      | SINTER key [key ...]                                       |
+-------------------+------------+------------------------------------------------------------+
|    SINTERSTORE    |    No      | SINTERSTORE destination key [key ...]                      |
+-------------------+------------+------------------------------------------------------------+
|     SISMEMBER     |    Yes     | SISMEMBER key member                                       |
+-------------------+------------+------------------------------------------------------------+
|     SMEMBERS      |    Yes     | SMEMBERS key                                               |
+-------------------+------------+------------------------------------------------------------+
|      SMOVE        |    No      | SMOVE source destination member                            |
+-------------------+------------+------------------------------------------------------------+
|      SPOP         |    No      | SPOP key                                                   |
+-------------------+------------+------------------------------------------------------------+
|    SRANDMEMBER    |    No      | SRANDMEMBER key [count]                                    |
+-------------------+------------+------------------------------------------------------------+
|      SREM         |    Yes     | SREM key member [member ...]                               |
+-------------------+------------+------------------------------------------------------------+
|     SUNION        |    No      | SUNION key [key ...]                                       |
+-------------------+------------+------------------------------------------------------------+
|   SUNIONSTORE     |    No      | SUNIONSTORE destination key [key ...]                      |
+-------------------+------------+------------------------------------------------------------+
|      SSCAN        |    No      | SSCAN key cursor [MATCH pattern] [COUNT count]             |
+-------------------+------------+------------------------------------------------------------+

Sorted Sets

+-------------------+------------+------------------------------------------------------------+
|      Command      | Supported? | Format                                                     |
+-------------------+------------+------------------------------------------------------------+
|      ZADD         |    Yes     | ZADD key score member [score] [member]                     |
+-------------------+------------+------------------------------------------------------------+
|      ZCARD        |    Yes     | ZCARD key                                                  |
+-------------------+------------+------------------------------------------------------------+
|      ZCOUNT       |    Yes     | ZCOUNT key min max                                         |
+-------------------+------------+------------------------------------------------------------+
|      ZINCRBY      |    Yes     | ZINCRBY key increment member                               |
+-------------------+------------+------------------------------------------------------------+
|     ZINTERSTORE   |    No      | ZINTERSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
+---------------------------------------------------------------------------------------------+
|      ZLEXCOUNT    |    No      | ZLEXCOUNT key min max                                      |
+-------------------+------------+------------------------------------------------------------+
|      ZRANGE       |    Yes     | ZRANGE key start stop [WITHSCORES]                         |
+---------------------------------------------------------------------------------------------+
|    ZRANGEBYLEX    |    No      | ZRANGEBYLEX key min max [LIMIT offset count]               |
+-------------------+------------+------------------------------------------------------------+
|    ZRANGEBYSCORE  |    Yes     | ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]|
+-------------------+------------+------------------------------------------------------------+
|      ZRANK        |    Yes     | ZRANK key member                                           |
+-------------------+------------+------------------------------------------------------------+
|       ZREM        |    Yes     | ZREM key member [member ...]                               |
+-------------------+------------+------------------------------------------------------------+
|   ZREMRANGEBYLEX  |    No      | ZREMRANGEBYLEX key min max                                 |
+-------------------+------------+------------------------------------------------------------+
|   ZREMRANGEBYRANK |    Yes     | ZREMRANGEBYRANK key start stop                             |
+-------------------+------------+------------------------------------------------------------+
|  ZREMRANGEBYSCORE |    Yes     | ZREMRANGEBYSCORE key min max                               |
+-------------------+------------+------------------------------------------------------------+
|    ZREVRANGE      |    Yes     | ZREVRANGE key start stop [WITHSCORES]                      |
+-------------------+------------+------------------------------------------------------------+
|  ZREVRANGEBYSCORE |    Yes     | ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]                                                      |
+-------------------+------------+------------------------------------------------------------+
|     ZREVRANK      |    Yes     | ZREVRANK key member                                        |
+-------------------+------------+------------------------------------------------------------+
|     ZSCORE        |    Yes     | ZSCORE key member                                          |
+-------------------+------------+------------------------------------------------------------+
|    ZUNIONSTORE    |    No      | ZUNIONSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
+-------------------+------------+------------------------------------------------------------+
|      ZSCAN        |    No      | ZSCAN key cursor [MATCH pattern] [COUNT count]             |
+-------------------+------------+------------------------------------------------------------+

Other Command

+--------------------------+--------------------------------------------+
|  change_master_to        | change_master_to ip port last_seq last_key |
+--------------------------+--------------------------------------------+
|  start_slave             | start_slave                                |
+--------------------------+--------------------------------------------+
|  stop_slave              | stop_slave [last_seq]                      |
+--------------------------+--------------------------------------------+
|  key_slot                | key_slot key                               |
+--------------------------+--------------------------------------------+
|  set_slot                | set_slot slot                              |
+--------------------------+--------------------------------------------+
|  unset_slot              | unset slot                                 |
+--------------------------+--------------------------------------------+
|  lock_db_with_read_lock  | lock_db_with_read_lock                     |
+--------------------------+--------------------------------------------+
|  unlock_db               | unlock_db                                  |
+--------------------------+--------------------------------------------+
|  migrate_slot            | migrate_slot slot ip port timeout speed    |
+--------------------------+--------------------------------------------+
|  slot_premigrating       | slot_premigrating                          |
+--------------------------+--------------------------------------------+
|  slot_postmigrating      | slot_postmigrating                         |
+--------------------------+--------------------------------------------+
|  slot_preimporting       | slot_preimporting                          |
+--------------------------+--------------------------------------------+
|  slot_postimporting      | slot_postimporting                         |
+--------------------------+--------------------------------------------+
|  dump                    | dump pattern                               |
+--------------------------+--------------------------------------------+
|  dump_slot               | dump_slot slot                             |
+--------------------------+--------------------------------------------+

Description

change_master_to: swith replication context to designated ip and port with last_seq(binlog_seq) and last_key(last sync key in snapshot).

start_slave: run replication thread with current replication context with read only mode.

stop_slave: stop replication thread.

key_slot: test an key the slot it belongs to.

set_slot: set the slot that the instance responsible for.

unset_slot: clean the slot that the instance responsible for.

lock_db_with_read_lock: read only mode.

unlock_db: enable write.

migrate_slot: migrate specified slot to another instance with the timeout(s) and speed(M).

slot_premigrating: flag an slot ready to migrate.

slot_postmigrating: flag an slot migration done and clean the responsiblity of this slot.

slot_preimporting: flag an slot ready to import.

slot_postimporting: flag an slot migration done and set the responsiblity of this slot.

dump: dump keys with specified pattern to file 'keys' at the working directory.

dump_slot: dump keys in specified slot to file 'keys-[slot]' at the working directiory.

Run singleton

By default, all the slot(0-16383) is disable. Slots need to be enable before running redis command. 'set_slot/unset_slot' could help to assign the range of responsible slots.