Skip to content

Migrating from InnoDB to RocksDB

Alex Yang edited this page Mar 16, 2017 · 8 revisions

Currently, fb-mysql supports running only one storage engine at a time. There is no online migration framework to move data between storage engines. Users need to dump logical data from source MySQL server with InnoDB engine and load it to destination MySQL server with RocksDB engine. This can be achieved via a utility such as (mysqldump)[https://github.com/facebook/mysql-5.6/wiki/Logical-backup-with-mysqldump].

The rough outline of the process is:

  • Copy all the database and table schema from source to destination
  • Dump each table to a file using "SELECT INTO OUTFILE"
  • Send the files to the destination and load them using "LOAD DATA INFILE"

To speed the loading process on the destination, it is recommended to use the following options:

* --sql_log_bin=0
* --foreign_key_checks=0
* --unique_checks=0
* --rocksdb_compaction_sequential_deletes=0
* --rocksdb_compaction_sequential_deletes_window=0
* --rocksdb_write_disable_wal=1
* --rocksdb_bulk_load=1
* --rocksdb_commit_in_the_middle=1
* --rocksdb_max_background_flushes=12
* --rocksdb_max_background_compactions=12
* --rocksdb_base_background_compactions=12

Note the above options are safe in migration scenarios. It is not recommended to use them elsewhere, as the options make assumptions such as primary key ordering for bulk loading, and no duplicate data. See (Data loading)[https://github.com/facebook/mysql-5.6/wiki/data-loading] for more details, and important information on how to drastically improve load speed for Primary Keys using bulk load and Secondary Keys using fast secondary index creation.

Clone this wiki locally