Release v0.0 · hikettei/Caten

Features

Fundamentals of Building a Compiler Based on Polyhedral Compiler and Pattern Matcher
- caten/ajit Polyhedral Compiler based on ISL (and more!), targeting multiple hardware.
  - Currently supports basic loop-oriented optimization, including: Loop Fusion, Loop Unrolling (Automatic Parallelization and Vectorizing), and Permuting.
- caten/air An optimizing pattern matcher based on trivia.
Implement a MemoryPlanner.
- all unary ops activations are always in-place.
VM
- Lisp VM (for testing)
JIT
- Currently supports Clang and Metal JIT.
Basic Neural Network Operations (Linear, ConvND, Activations, Norms, etc...)
Fundamental Tensor Operations (mathematical functions, arithmetic, reduce, broadcast, view, reshape, permute, etc..)
Implements reverse-mode Autograd but it is still unstable.

(Metal JIT) Complete Implementing Metal JIT, by implementing BEAM Search to determine the optimal global_size/local_size. #44
(AJIT) Implement Hide Latency Optimization and Index Computation Simplification.
(AIR) Implement Attr class for the type-safety. #70
(NN) Add tests for ConvND and Norms
(CI) Add a pipeline for the benchmarking (my goal is to beat PyTorch)
(AJIT) Gemm-specific optimization (8x8 gemm tiling)
(AJIT) packed-funcall -> simd mutation for Clang and Metal
(CI) Implement YoLoV3 and running on CI

[CI Debugging+Enhancement] All mathematical functions should (aim to) satisfy the 1.0ULP accuracy by @hikettei in #4
[wip] refactors, implemented nn ops etc by @hikettei in #5
[WIP] polyhedral compiler passing all tests by @hikettei in #8
[Feature] Activations/ReduceMax/ReduceMin by @hikettei in #11
[WIP] Establish APIs, NN Ops, Control Flow etc... by @hikettei in #12
Threefry2x32 by @hikettei in #13
AOT Compilation, Shape Inference by @hikettei in #15
[Draft] Make JIT completely equivalent to VM (Forward Mode) by @hikettei in #16
[WIP] (Backward Mode) Make JIT completely equivalent to the VM Mode by @hikettei in #18
[JIT] Kernel Creation, Improving memory-planner, etc... by @hikettei in #32
Patch for CCL and ECL by @hikettei in #37
gc-reachable isl objects by @hikettei in #39
[Feature] New Backends: Metal and Lisp by @hikettei in #42
[Enhancements] (Much Better) Loop Fusion and Packed Funcall Mutation (for unrolling/vectorizing/tiling). by @hikettei in #49
[WIP] Fix invaild shape inference in composed gemm and index-components by @hikettei in #51
Revisit: fix %where by @hikettei in #53
CI: Improvements on CI and Testing by @hikettei in #54
CI: Testing against all nn ops by @hikettei in #66
[Refactor] Introduce Memory Optimization based on MIP by @hikettei in #69

Full Changelog: https://github.com/hikettei/Caten/commits/v0.0