Skip to content

Commit

Permalink
chore: rename CH_ENGINE to CH_MIGRATIONS_ENGINE for consistency (#7)
Browse files Browse the repository at this point in the history
and make the password flag optional
  • Loading branch information
shini4i authored Jun 28, 2024
1 parent 0b76c05 commit 7a48470
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 21 deletions.
30 changes: 19 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
<div align="center">

# clickhouse-migrations

![NPM Version](https://img.shields.io/npm/v/%40shini4i%2Fclickhouse-migrations)
![NPM Downloads](https://img.shields.io/npm/dm/%40shini4i%2Fclickhouse-migrations)
[![codecov](https://codecov.io/gh/shini4i/clickhouse-migrations/graph/badge.svg?token=8QWRD6EAQJ)](https://codecov.io/gh/shini4i/clickhouse-migrations)
![GitHub License](https://img.shields.io/github/license/shini4i/clickhouse-migrations)

</div>

## Install

Expand All @@ -12,7 +19,7 @@ npm install -g @shini4i/clickhouse-migrations

Create a directory, where migrations will be stored. It will be used as the value for the `--migrations-home` option (or for environment variable `CH_MIGRATIONS_HOME`).

In the directory, create migration files, which should be named like this: `1_some_text.sql`, `2_other_text.sql`, `10_more_test.sql`. What's important here is that the migration version number should come first, followed by an underscore (`_`), and then any text can follow. The version number should increase for every next migration. Please note that once a migration file has been applied to the database, it cannot be modified or removed.
In the directory, create migration files, which should be named like this: `01_some_text.sql`, `02_other_text.sql`, `10_more_test.sql`. What's important here is that the migration version number should come first, followed by an underscore (`_`), and then any text can follow. The version number should increase for every next migration. Please note that once a migration file has been applied to the database, it cannot be modified or removed.

For migrations' content should be used correct SQL ClickHouse queries. Multiple queries can be used in a single migration file, and each query should be terminated with a semicolon (;). The queries could be idempotent - for example: `CREATE TABLE IF NOT EXISTS table ...;` Clickhouse settings, that can be included at the query level, can be added like `SET allow_experimental_object_type = 1;`. For adding comments should be used `--`, `# `, `#!`.

Expand All @@ -22,39 +29,40 @@ If the database provided in the `--db` option (or in `CH_MIGRATIONS_DB`) doesn't
Usage
$ clickhouse-migrations migrate <options>
Required options
--url=<url> Clickhouse URL
(ex. https://clickhouse:8123)
Required flags
--url=<url> Clickhouse URL (ex. https://clickhouse:8123)
--user=<name> Username
--password=<password> Password
--db=<name> Database name
--migrations-home=<dir> Migrations' directory
--engine=<engine> The engine to use for DB creation (optional)
Optional flags
--password=<password> Password
--engine=<engine> The engine to use for DB creation
Environment variables
Instead of options can be used environment variables.
CH_MIGRATIONS_URL Clickhouse hostname (--url)
CH_MIGRATIONS_USER Username (--user)
CH_MIGRATIONS_PASSWORD Password (--password)
CH_MIGRATIONS_DB Database name (--db)
CH_MIGRATIONS_HOME Migrations' directory (--migrations-home)
CH_ENGINE The engine to use for DB creation (optional)
CH_MIGRATIONS_ENGINE The engine to use for DB creation (optional)
CLI examples
clickhouse-migrations migrate --url=http://localhost:8123
--user=default --password='' --db=analytics
--user=default -db=analytics
--migrations-home=/app/clickhouse/migrations
clickhouse-migrations migrate
```

Migration file example:
```
-- an example of migration file 1_init.sql
-- an example of migration file 01_init.sql
SET allow_experimental_object_type = 1;
CREATE TABLE IF NOT EXISTS events (
event JSON
);
```
```
14 changes: 7 additions & 7 deletions src/migrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ const log = (type: 'info' | 'error' = 'info', message: string, error?: string) =
}
};

const connect = (url: string, username: string, password: string, db_name?: string): ClickHouseClient => {
const connect = (url: string, username: string, password?: string, db_name?: string): ClickHouseClient => {
const db_params: ClickhouseDbParams = {
url,
username,
Expand All @@ -28,7 +28,7 @@ const connect = (url: string, username: string, password: string, db_name?: stri
return createClient(db_params);
};

const create_db = async (url: string, username: string, password: string, db_name: string, engine?: string): Promise<void> => {
const create_db = async (url: string, username: string, db_name: string, password?: string, engine?: string): Promise<void> => {
const client = connect(url, username, password);

let q = `CREATE DATABASE IF NOT EXISTS "${db_name}"`;
Expand Down Expand Up @@ -220,13 +220,13 @@ const migration = async (
migrations_home: string,
url: string,
username: string,
password: string,
db_name: string,
password?: string,
engine?: string
): Promise<void> => {
const migrations = get_migrations(migrations_home);

await create_db(url, username, password, db_name, engine);
await create_db(url, username, db_name, password, engine);

const client = connect(url, username, password, db_name);

Expand All @@ -247,12 +247,12 @@ const migrate = () => {
.description('Apply migrations.')
.requiredOption('--url <url>', 'Clickhouse URL (ex: http://clickhouse:8123)', process.env.CH_MIGRATIONS_URL)
.requiredOption('--user <name>', 'Username', process.env.CH_MIGRATIONS_USER)
.requiredOption('--password <password>', 'Password', process.env.CH_MIGRATIONS_PASSWORD)
.requiredOption('--db <name>', 'Database name', process.env.CH_MIGRATIONS_DB)
.requiredOption('--migrations-home <dir>', "Migrations' directory", process.env.CH_MIGRATIONS_HOME)
.option('--engine <name>', 'Engine name', process.env.CH_ENGINE)
.option('--password <password>', 'Password', process.env.CH_MIGRATIONS_PASSWORD)
.option('--engine <name>', 'Engine name', process.env.CH_MIGRATIONS_ENGINE)
.action(async (options: CliParameters) => {
await migration(options.migrationsHome, options.url, options.user, options.password, options.db, options.engine);
await migration(options.migrationsHome, options.url, options.user, options.db, options.password, options.engine);
});

program.parse();
Expand Down
4 changes: 2 additions & 2 deletions src/types/cli.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ type ClickhouseDbParams = {
max_open_connections?: number;
compression?: { response?: boolean; request?: boolean };
username: string;
password: string;
password?: string;
application?: string;
database?: string;
clickhouse_settings?: ClickHouseSettings;
Expand All @@ -31,8 +31,8 @@ type CliParameters = {
migrationsHome: string;
url: string;
user: string;
password: string;
db: string;
password?: string;
engine?: string;
};

Expand Down
2 changes: 1 addition & 1 deletion tests/db.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ describe('Migration tests', () => {
const execSpy = jest.spyOn(createClient1, 'exec');
const insertSpy = jest.spyOn(createClient1, 'insert');

await migration('tests/migrations/one', 'http://sometesthost:8123', 'default', '', 'analytics');
await migration('tests/migrations/one', 'http://sometesthost:8123', 'default', 'analytics');

expect(execSpy).toHaveBeenCalledTimes(3);
expect(querySpy).toHaveBeenCalledTimes(1);
Expand Down

0 comments on commit 7a48470

Please sign in to comment.