Ethereum Data Extraction Library

Introduction

We need an easy to use library to access the data published on-chain (or in the future maybe also off-chain). The main use cases are:

• the recreation of the Merkle tree
• allow block explorers to be created for our sidechain

For a general overview of some of the data structures see here.

Data

The library will access the Ethereum block-chain to read in the necessary data:

• Data available in the raw tx data from Ethereum blocks (mainly from the operator (tx.origin))
• Data from events from the Exchange contract

Because it's not easy to call a function on a contract for an old state we should not rely on contract functions to get certain data.

If we notice while creating this library that we can't easily access certain data we'll have to fix that.

Some data needs some simple additional processing to get the actual useful values. For example, we allow splitting the fee between the wallet and the ring-matcher. How much each party actually gets paid isn't directly available. As part of the on-chain data-availability we have the fee amount paid by the order owner and the wallet split percentage, it's trivial to calculate how much each party gets using that information.

On-chain data-availability format

Please see the docs in the code for commitBlock so that this information is documented in a single place.

Exchange Events

Please see here for all the events.

Most notable:

• BlockCommitted: The tx data of the Ethereum block will need to be extracted
• BlockFinalized: The block was verified
• Revert: All state changes up to and including the reverted block need to be reverted
• DepositRequested: The account info and the balance of the given token needs to be updated
• WithdrawalRequested: The account info (in shutdown mode) and the balance of the given token needs to be updated

Not all events are applied immediately. The DepositRequested and WithdrawalRequested events are only actually processed when they are included in a block (which block can be known by inspecting the data passed into commitBlock, but the data used for these updates is availably in these events).

Request logic

The logic required is in general pretty simple. For code references, please refer to

• The simulator code used for validation. This is the easiest to read, but does not contain code to generate the Merkle proofs.
• The operator code used for creating the blocks in the tests. Contains all the code needed to create valid blocks.
• The circuit code

API

The requirements for the library:

• A program/library needs to be able to easily recreate the Merkle tree by using the data provided by the library
• The library provides all the necessary data needed for the creation of a block explorer for our sidechain

Merkle tree recreation

This should be the most straightforward requirement. By playing back all the state changes from start to end, modifying the Merkle tree at every step, the recreated Merkle tree should match exactly with the on-chain Merkle tree for every block. In practice, recreatedMerkleTreeRoot == onchainMerkleTreeRoot for every block.

Block Explorer integration

This ones a bit more difficult because we don't want to start from scratch here, we want to integrate/modify one of the existing open source blockchain explorers. EthVM created by MEW looks interesting, but is still being developed. There is already an alpha release.

Pseudo Spec

The library will need to store some state because the complete exchange state is only known when all Ethereum blocks since the creation of the Exchange is inspected.

//
//    Data structures
//

// A block
struct Block
{
    blockIdx: number;
    blockType: BlockType;
    blockState: BlockState;
    operator: number;
    transactions: Transactions[];
}

// A token transfer (in the Merkle tree)
struct TokenTransfer
{
    from: number;
    to: number;
    token: number;
    amount: BN;
}

// A trade history update (in the Merkle tree)
struct TradeHistoryUpdate
{
    filled: BN;
    cancelled: boolean;
    orderID: number;
}

// A Ring Settlement transaction
struct RingSettlementTransaction
{ 
    ring-matcher: number;
    fee-recipient: number;
    tokenTransfers: TokenTransfer[];
    tradeHistoryUpdates: TradeHistoryUpdate[2];
};

// A Deposit transaction
struct DepositTransaction
{
    accountID: number,
    tokenID: number,
    amount: BN,
    pubKeyX: string,
    pubKeyY: string,
};

// An on-chain withdrawal transaction
struct OnchainWithdrawalTransaction
{
    shutdown: boolean,
    accountID: number,
    tokenID: number,
    amount: BN,
};

// An off-chain withdrawal transaction
struct OffchainWithdrawalTransaction
{
    accountID: number,
    tokenID: number,
    amount: BN,
    tokenTransfers: TokenTransfer[];
};

// An off-chain withdrawal transaction
struct CancelTransaction
{
    accountID: number,
    tokenID: number,
    tradeHistoryUpdate: TradeHistoryUpdate;
    tokenTransfers: TokenTransfer[];
};

//
//    Functions
//

// Sets up the state for an exchange
initialize(exchangeContractAddress: string, exchangeCreationEthereumBlockIdx: number)

// Runs over all Ethereum blocks until the specified ending Ethereum block idx
// so the internal state can be updated. As the starting point the
// previous ending Ethereum block idx is used.
processEthereumBlocksUpTo(endingEthereumBlockIdx: number)

// Returns the number of blocks
getNumBlocks()

// Returns the specified block
getBlock(blockIdx: number)