Feature/prequet file

[akvlad]

write & merge parquet files engine

How to use

1. Run the quackpipe with the parquet ingester enabled:

docker run \
  -e QUACK_PIPE_ENABLED=true \
  -e QUACK_PIPE_ROOT=/data \
  -e QUACK_PIPE_MERGE_TIMEOUT_S=10 \
  -v "$(pwd)/_data:/data" \
  -p 8080:8123 \
  ghcr.io/metrico/quackpipe:latest

2. Create a "table"

cat <<EOF | curl -X POST http://localhost:8080/quackdb/create --data-binary @/dev/stdin
create_table: test
fields:
  timestamp_ns: INT8
  fingerprint: INT8
  str: STRING
  value: FLOAT8
engine: Merge
order_by:
  - timestamp_ns
timestamp:
  field: timestamp_ns
  precision: ns
partition_by: ""
EOF

3. Send the data:

cat <<EOF | curl -X POST http://localhost:8080/quackdb/test/insert -H 'Content-Type: application/x-ndjson' --data-binary @/dev/stdin
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
{"timestamp_ns": 1668326823000000000, "fingerprint": 1234567890, "str": "hello", "value": 123.456}
EOF