Skip to content

Commit

Permalink
docs: update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
betarixm committed Dec 7, 2023
1 parent 5a8a2f2 commit 9c5a9ed
Showing 1 changed file with 45 additions and 4 deletions.
49 changes: 45 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,56 @@
[![CSED434@POSTECH](https://img.shields.io/badge/CSED434-POSTECH-c80150)](https://www.postech.ac.kr/eng)
[![CSED434@POSTECH](https://img.shields.io/badge/Fall-2023-775E64)](https://www.postech.ac.kr/eng)

**Paranode** is a project developed as part of POSTECH's CSED434 curriculum, dedicated to the implementation of distributed and parallel sorting for key-value records using the Scala programming language. Its primary objective is the development of a robust sorting system tailored for the distribution of extensive datasets, surpassing the limitations of available RAM and storage capacity. Leveraging Scala, Paranode orchestrates parallel data processing across multiple machines, preserving record integrity through key-based sorting.
**Paranode** is a project developed as part of POSTECH's CSED434 curriculum, dedicated to the implementation of **distributed and parallel sorting system** for key-value records using the Scala programming language. Its primary objective is the development of a robust sorting system tailored for the distribution of extensive datasets, surpassing the limitations of available RAM and storage capacity. Leveraging Scala, Paranode orchestrates parallel data processing across multiple machines, preserving record integrity through key-based sorting.

## Getting Started
## Usage

### Development Environment
### Build

#### Install pre-commit hooks
To build the project, follow these instructions:

1. Clone the repository.
1. Run `sbt assembly`.

### Run

To launch a master node and worker nodes, follow these steps:

1. Build the project using the instructions above.
1. Run [`master` script](bin/master) with parameters.
- For example, execute `./master <NUMBER_OF_WORKERS>` from the [`bin`](bin) directory.
1. Run [`worker` script](bin/worker) with parameters.
- For example, `./worker <MASTER_HOST>:<MASTER_PORT> -I <INPUT_DIRECTORY> <INPUT_DIRECTORY> <...> -O <OUTPUT_DIRECTORY>` from the [`bin`](bin) directory.

It is possible to run jar files downloaded from releases page instead of building the project.

## Development

### Environment

- Scala 2.13.12
- SBT 1.9.6
- Java 20

### Test

To run unit tests, follow the instructions below:

1. Clone the repository.
1. Run `sbt test`.

To run e2e tests, follow the instructions below:

1. Clone the repository.
1. Run `sbt e2e/test`.

### Pre-commit hooks

1. Install `pre-commit` package by [instructions](https://pre-commit.com/#installation).
1. Run `pre-commit install`.

Note that `pre-commit install` should be re-run if [`.pre-commit-config.yaml`](.pre-commit-config.yaml) changed.

## Disclaimer

This project is developed as part of the CSED434 course at POSTECH, and is not intended for production or commercial use. Any usage is at your own risk, and the contributors are not responsible for any potential issues.

0 comments on commit 9c5a9ed

Please sign in to comment.