Questions about datafuse

[XuPeng-SH]

I have some questions about datafuse:

1. How does datafuse process a large number of small inserts?
2. Does datafuse support real-time point deletes|updates like in OLTP databases?

[BohuTANG]

I have some questions about datafuse:

1. How does datafuse process a large number of small inserts?

Small inserts is not recommended, we can have a buffer WAL as ClickHouse does.

2. Does datafuse support real-time point deletes|updates like in OLTP databases?

y

[XuPeng-SH]

Thanks for your quick reply.

If the underlying storage is similar to Clickhouse's MergeTree solution, how do you achieve efficient real-time deletion? How can datafuse quickly locate the block where the target tuple is located when performing a delete operation? how to apply delete? Does it maintain a globally unique primary key for each table?

Does each delete request merge with the target block data and synchronously rewrite a new block, or generate a new file as a deletion mark file? Or each delete submission will add a version to the target block, and all versions are maintained in memory (which can be rebuilt through WAL), and the following queries need to process these multi-version data.

What isolation level does datafuse intend to support？

[BohuTANG]

Q1: delete/update on MergeTree
We didn't exactly copy ClickHouse's MergeTree approach, we referred to iceberg's(and others) design, aim to support fast delete/update, it's in design.

Q2: isolation level
snapshot for time travel and transaction.