BabuDB usage in Java

How to use BabuDB for Java

Startup

On startup BabuDB will load the databases from disk, the operations log will be replayed and the worker threads are started.

import org.xtreemfs.babudb.BabuDB;
import org.xtreemfs.babudb.BabuDBFactory;
import org.xtreemfs.babudb.BabuDBRequestResult;
import org.xtreemfs.babudb.BabuDBRequestListener;

...

BabuDB databaseSystem = BabuDBFactory.createBabuDB(new BabuDBConfig("babudb/databases/", "babudb/dblog/", 4, 1024*1024*16, 5*60, SyncMode.FSYNC, 50, 0, false, 16, 1024*1024*512));

This will start BabuDB with the database indices stored in babudb/databases/. The operations logs will be written into babudb/dblog/. In this example, we start 4 worker threads. We set that a checkpoint should be created when the operations log exceeds 16MB. The checkpointer will check the log file size every five minutes. The syncMode is set to FSYNC, that means before acknowledging the operation an fsync is executed on the logfile. If the next value is greater zero requests are immediateley aknowledged and synced to disk every given interval (here 50 ms). We require that the queue for each worker is unlimited, if this value is greater than 0 all workers are limited to the given value. And finally disable the index-compression, by setting its flag to false. Further explanations on these parameters can be found here.

Hint: To simplify the configuration, it is also possible to use the configuration builder, e.g.

BabuDB databaseSystem = BabuDBFactory.createBabuDB(new ConfigBuilder().setDataPath("/tmp/babudb").build());

This will set up BabuDB with a configuration that stores all data in /tmp/babudb and uses default parameters for all other settings.

Creating a new Database

DatabaseManager dbm = databaseSystem.getDatabaseManager();
Database db = dbm.createDatabase("myDB",5);

Creates a new database called "myDB" with five indices. Indices are identified by automatically assigned numbers, from zero to four. So far, it is not possible to add/remove indices afterwards!

Database operations

Every operation performed on a database will return a future-object hereafter called result. This object implements the generic BabuDBRequestResult-interface which provides the operation get() to wait synchronously for a DB request to finish and registerListener(BabuDBRequestListener<?> listener) to install an asynchronous generic result listener for the call.

The last argument for every database operation is an arbitrary object context, which hereafter will be set to null. This object can be used by the programmer to give asynchronous requests an individual signature. It will by returned to the BabuDBRequestListener on the end of the request, if one was installed.

Object context = null;

Inserting and Deleting Key-Value Pairs

A single key-value pair can be inserted atomic in an index as follows:

BabuDBRequestResult<Object> result = db.singleInsert(2, "key".getBytes(), "value".getBytes(), context);
result.get();

An atomic insertion of multiple key-value pairs can be performed as follows:

BabuDBInsertGroup ig = db.createInsertGroup();
ig.addInsert(2, "key".getBytes(), "value".getBytes());
ig.addInsert(2, "key2".getBytes(), "value2".getBytes());
BabuDBRequestResult<Object> result = db.insert(ig, context);
result.get();

Key-value pairs can be deleted from an index by setting the value to null:

BabuDBRequestResult<Object> result = db.singleInsert(2, "key".getBytes(), null, context);
result.get();

or

ig.addInsert(2, "key2".getBytes(), null);
BabuDBRequestResult<Object> result = db.insert(ig, context);
result.get();

Performing Lookups

A value can be looked up for a key with the following code:

BabuDBRequestResult<byte[]> result = db.lookup(2, "key".getBytes(), context);
byte[] value = result.get();

It is also possible to perform prefix lookups:

BabuDBRequestResult<Iterator<Entry<byte[], byte[]>>> result = database.prefixLookup(2, "k".getBytes(), context);
Iterator<Entry<byte[], byte[]>> iterator = result.get();
while(iterator.hasNext()) {
    Entry<byte[], byte[]> keyValuePair = iterator.next();
    ...
}

A prefix lookup generates an iterator that returns all entries starting with the given prefix in ascending order.

Asynchronous Requests

In addition to operate asynchron on BabuDB's databases you can register BabuDBRequestListener<?> on the future-object returned by the operation.

This listener is generic and has to provide the following operations to match the interface:

BabuDBRequestListener<?> listener = new BabuDBRequestListener<?>() {
    public void failed(BabuDBException error, Object context) {
        ...
    }

    public void finished(? value, Object context) {
        ...
    }
});

Asynchronous requests can then be executed as follows:

BabuDBRequestResult<Object> result = database.insert(ig, context);
result.registerListener(new BabuDBRequestListner<Object>({
    public void failed(BabuDBException error, Object context) {
        ...
    }

    public void finished(Object value, Object context) {
        ...
    }
}));

BabuDBRequestResult<byte[]> result = database.lookup("myDB", 2, key, context);
result.registerListener(new BabuDBRequestListener<byte[]>({
    public void failed(BabuDBException error, Object context) {
        ...
    }

    public void finished(byte[] value, Object context) {
        ...
    }
}));

BabuDBRequestResult<Iterator<Entry<byte[], byte[]>>> result = database.prefixLookup("myDB", 2, "k".getBytes(), context);
result.registerListener(new BabuDBRequestListener<Iterator<Entry<byte[], byte[]>>>({
    public void failed(BabuDBException error, Object context) {
        ...
    }

    public void finished(Iterator<Entry<byte[], byte[]>> value, Object context) {
        ...
    }
}));

Sample Application

import org.xtreemfs.babudb.BabuDB;
import org.xtreemfs.babudb.BabuDBException;
import org.xtreemfs.babudb.BabuDBFactory;
import org.xtreemfs.babudb.log.DiskLogger.SyncMode;
import org.xtreemfs.babudb.lsmdb.BabuDBInsertGroup;
import org.xtreemfs.babudb.lsmdb.Database;
import org.xtreemfs.babudb.lsmdb.DatabaseManager;
import org.xtreemfs.include.common.config.BabuDBConfig;

public class SimpleDemo {

    public static void main(String[] args) throws InterruptedException {
        try {           
            //start the database
            BabuDB databaseSystem = BabuDBFactory.createBabuDB(new BabuDBConfig("myDatabase/", "myDatabase/", 2, 1024 * 1024 * 16, 5 * 60, SyncMode.SYNC_WRITE, 0, 0, false, 16, 1024 * 1024 * 512));
            DatabaseManager dbm = databaseSystem.getDatabaseManager();
                        
            //create a new database called myDB
            dbm.createDatabase("myDB", 2);
            Database db = dbm.getDatabase("myDB");
            
            //create an insert group for atomic inserts
            BabuDBInsertGroup group = db.createInsertGroup();

            //insert one key in each index
            group.addInsert(0, "Key1".getBytes(), "Value1".getBytes());
            group.addInsert(1, "Key2".getBytes(), "Value2".getBytes());

            //and execute group insert
            db.insert(group,null).get();

            //now do a lookup
            byte[] result = db.lookup(0, "Key1".getBytes(),null).get();

            //create a checkpoint for faster start-ups
            databaseSystem.getCheckpointer().checkpoint();

            //shutdown database
            databaseSystem.shutdown();
        } catch (BabuDBException ex) {
            ex.printStackTrace();
        }
    }  
}

Replication - currently under development

If you want to have a higher availability of your service and better protection against data loss the TBA 0.5.0 release will provide a plugin for replicating databases among multiple instances that of course should run on different servers. Your application may access any of these instances in a way described on this page and the plugin will redirect or replicate your requests in background. If you want to learn more about how to configure such a multi-instance BabuDB system click here.