RBLN Model Zoo

This repository provides a collection of machine learning model examples for ATOM, Rebellions' neural processing unit (NPU). We are continuously enhancing the rbln-model-zoo by adding more models in the following categories:

• Natural Language Processing
• Generative AI
• Speech Processing
• Computer Vision

All deep learning examples in the RBLN Model Zoo include two files: compile.py and inference.py.

• compile.py: compile the model and save the compiled results to local storage
• inference.py: load the saved compiled results and perform inference

Install Prerequisites

• General Requirements: Rebellions Compiler

  The rebel-compiler Python package is required for all workflows involing RBLN NPUs. Please install it before processing. You need an RBLN portal account to install rebel-compiler.

  pip3 install -i https://pypi.rbln.ai/simple rebel-compiler

• HuggingFace Models

  Optimum RBLN serves as a bridge connecting the HuggingFace transformers/diffusers libraries to RBLN NPUs. It offers a set of tools that enable easy model compilation and inference for both single and multi-NPU (Rebellions Scalable Design) configurations, across a range of downstream tasks. You need an RBLN portal account to install optimum-rbln. To install prereuisites for HuggingFace models, navigate to the model's directory and use its requirements.txt:

  pip3 install -i https://pypi.rbln.ai/simple optimum-rbln
  pip3 install -r <model_directory>/requirements.txt

  For instance:

  pip3 install -r huggingface/question-answering/bert/requirements.txt

• PyTorch

  Each PyTorch model now includes its own requirements.txt. Install the prerequisites for your specific model by navigtating to the relevant directory:

  pip3 install -r pytorch/<model_directory>/requirements.txt 

• TensorFlow

  Similarly, TensorFlow models provide a requirements.txt in their respective directories. Install prerequisites as follows:

  pip3 install -r tensorflow/<model_directory>/requirements.txt

• Language Binding

  • C/C++

    • The C/C++ API can be installed via the APT repository. Please refer to C/C++ binding Installation Guide for more details.

Model List

You can find the complete list of models on our homepage and in the online documentation.

Developer Resources

Explore RBLN SDK documentation to access detailed information including:

• Tutorials
  • PyTorch: ResNet50
  • TensorFlow: BERT-base
  • HuggingFace transformers: LLama2-7b
  • HuggingFace diffusers: SDXL-turbo
  • C/C++ binding: ResNet50
  • C/C++ binding: Yolov8m
• APIs
  • Python Compile & Runtime
  • HuggingFace Model API (optimum-rbln)
  • C/C++ Binding API
• Supported Models
• Supported Operations
• Model Serving Guide using Nvidia Triton Inference Server
• vLLM Support
• Tools

Release Notes

For detailed information on updates and changes, please refer to the release notes.

Getting Help

If you encounter any problem with the examples provided, please open an issue on GitHub. Our team will assist you as soon as possible.