Skip to content

Adding New Tool

Jump to bottom
ad-daniel edited this page Jan 31, 2023 · 22 revisions

New tools added to the OpenDR GitHub repo should provide the following files:

  • source code to apply the tool for training and inference
  • ROS and ROS2 nodes using the tool for inference
  • documentation of the tools including:
    • a short description of the method itself. It is not necessary to give all the details, but it is good to link to the related paper or reference if any.
    • details description of the API needed by the user to apply the tool to her scenario
    • step-by-step description about how to run the example
    • ROS/ROS2 documentation entries
  • unit tests
  • example (preferably using Webots) of how to apply the tool to a scenario
  • at least one pretrained model ready for inference

The following instructions will provide paths using the following wildcards:

  • <type>: tool type, e.g., control, perception, planning
  • <task_name>: task addressed by the tool, for example "pose_estimation"
  • <algorithm_name>: name of the algorithms used to solve the task, for example, "lightweight_open_pose"

Source Code

New Tool Public API

New tools should follow the OpenDR code guidelines and ensure a common interface:

Files

The source code of the tool has to be stored in the /src/<type>/<task_name>/<algorithm_name> folder. The expected files are:

  1. README.md: containing a short description of the task and algorithms (1-2 lines), the list of the sources of third-party code (if any), and the modifications to the original code
  2. <algorithm_name>_learner.py: main class providing the entry point to the functionality
  3. algorithm/: folder containing the algorithm implementation
  4. dependencies.ini: list of dependencies needed to run the tool. See Dependencies section for more details about the format.

CI tests

The files in /src are tested by the CI (Continuous Integration) system to check the compatibility with the OpenDR guidelines. Tests on original files taken from third-party that are copied and pasted without modifications can be skipped. In particular, the OpenDR license test should be skipped because it is now allowed to change license and copyrights of third-party code. It is possible to skip tests for a folder by adding the path to the skippedDirectoryPaths variable in the test itself.

Tests are run automatically based on the label set in the pull request. The following options are available:

  • test sources (mandatory): ensures codestyle guides are met.
  • test tools (mandatory): installs the toolkit from sources and ensures the addition does not introduce conflicts with existing tools.
  • test release (mandatory if a pip wheel is created): generates a pip wheel from the current branch, installs it on a clean system and ensures the test provided in the test suite works accordingly.

Third-party Code and License

If not modified and available on a public GitHub repo, third-party code should be linked (as a "submodule") instead of copied in the OpenDR repository. If not available on a GitHub repo or some modifications are applied:

  • commit the algorithms sources in the OpenDR repo
  • specify the link to the original code in the tool README.md file, the author, and list all the modifications

OpenDR license doesn't have to be applied on third-party code!

OpenDR Apache License version 2.0 is a permissive license: work can be redistributed even for commercial use. So, third-party code integrated into the OpenDR toolkit should also be permissive. Non-commercial licensed code cannot be integrated. Common compatible licenses are:

Dependencies

Dependencies will be automatically installed using a script and the Makefile system. For each tool, a dependencies.ini file should be created listing all the dependencies required to run the method. This file should be written using the INI format. Three different sections are supported:

  • runtime: includes dependencies required to execute the tool
  • compilation: includes dependencies required to compile the C/C++ code. Python tools require runtime dependencies only.
  • device: includes the opendr_device key

In the runtime and compilation section, the following keys can be used:

  • python: listing the python modules. The value syntax should be the standard pip requirements file format
  • python-dependencies: listing additional python packages that will be installed before the python dependencies to resolve prerequisites
  • linux: listing all the system packages required on Linux systems that will be installed using the system package manager
  • macOS: not implemented yet
  • windows: not implemented yet

In the device section, the following keys can be used:

  • opendr_device: with valid options cpu and gpu. If the listed dependencies strictly require a GPU, the key has to be set to gpu. Otherwise, i.e., if a CPU suffices, the key should be set to cpu or can be omitted.

Here is a sample dependencies.ini file.

[runtime]
python=torch==1.7.1
       opencv-python
linux=unzip wget

[compilation]
linux=g++

[device]
opendr_device=cpu

ROS and ROS2 nodes

The new tool should be accompanied by a ROS node and a ROS2 node demonstrating its inference capabilities through ROS.

ROS Node

General

The ROS nodes should be placed in /projects/opendr_ws/src/opendr_<type>/scripts/ and declared in CMakeLists.txt located in /projects/opendr_ws/src/opendr_<type>/, e.g. the CMakeLists.txt file for the opendr_perception package. Any additional ROS dependencies should be declared in package.xml located in /projects/opendr_ws/src/opendr_<type>/, e.g. the package.xml file for the opendr_perception package.

Node source code

The new node should be as consistent with already existing nodes as possible. Some important points are:

  1. Node file name should be consistent with other nodes, i.e. <task_name>_node.py, or <task_name>_<algorithm_name>_node.py if there are multiple nodes per <task_name> with different <algorithm_names>.
  2. Similarly, when initializing the node its name should be opendr_<task_name>_node or opendr_<task_name>_<algorithm_name>_node so it is easily identifiable through ROS.
  3. When initializing the input topic subscriber make sure to set the queue appropriately so the node does not suffer from any delay, see known issues.
  4. OpenDR ROS nodes use argparse so as to have a consistent interface and to streamline their usage. Consequently, the node should have a similar interface to other existing nodes depending on its input/output modality.
  5. In addition to the previous point, it is important that arguments should follow a naming format similar to other ROS nodes, e.g. image input topics should be similar to input_rgb_image_topic or input_infra_image_topic.
  6. The node should define some helpful shortcuts in argparse, e.g. -i for the input topic argument.
  7. All OpenDR nodes have default topic names for their most expected usage, so they are plug and play for simple testing. For example a node that takes RGB images as input, should default to /usb_cam/image_raw input topic so it works with the basic usb_cam package straight away.
  8. The device on which the node will execute (CPU/GPU) should be handled automatically, depending on the argument provided.
  9. Check whether the new node needs any new bridge methods for its data input/output and make sure it uses OpenDR data formats.
  10. If the node requires any custom service or message, it can be added in the opendr_bridge package, similar to other OpenDR services and messages. Remember to declare the new messages and services in CMakeLists.txt.

ROS2 Node

General

The ROS2 nodes should be placed in /projects/opendr_ws_2/src/opendr_<type>/opendr_<type>/ and declared in setup.py located in /projects/opendr_ws_2/src/opendr_<type>/, e.g. the setup.py file for the opendr_perception package. Any additional ROS2 dependencies should be declared in package.xml located in /projects/opendr_ws_2/src/opendr_<type>/, e.g. the package.xml file for the opendr_perception package.

Node source code

The ROS2 node should follow the same guidelines as the ROS node source code above with the following differences:

  1. The node initialization and setting of the queue from points 2 and 3 from above are done in slightly different ways as can be seen here and here, but the same rules apply.
  2. The default input topic name of usb_cam from point 7 is /image_raw for the ROS2 package.
  3. The opendr_bridge package of ROS2 for point 9 from above can be found here.
  4. ROS2 requires a C++ package to define services and messages, so any custom services and messages needed are defined in the opendr_interface package instead of opendr_bridge in ROS, here and here respectively. Remember to declare the new messages and services in CMakeLists.txt.

Documentation

All the documentation needed by the user should be added in the /docs/reference folder, i.e. in the Reference Manual of the OpenDR toolkit. A single file named after the method should be added and should contain these informations:

  1. Description of the method
  2. Specifications of the API
  3. Instructions to run the example
  4. ROS/ROS2 node documentation

Adding documentation directly in the code is recommended to make the code more readable and understandable. But this is not considered part of the OpenDR API documentation needed to use the toolkit.

When adding a new documentation file, please also make sure to add it to the documentation index: https://github.com/opendr-eu/opendr/blob/master/docs/reference/index.md

Description of the method

This includes a description of the method: what is the purpose of the method, which are the techniques used, an overview of the performance, etc. This is not intended to be a detailed description of how the method is implemented or all the research behind it, but it should give enough information to the user to understand if this method could be applied to his scenario. It is good to link to the related paper or similar so that interested users can look into the details.

Specifications of the API

All the public methods of the tool have to be listed and described. This includes the description of the function arguments, returned values, and functionality.

Instructions to run the examples

The last section of the documentation page should include the step-by-step instructions to run the provided example. A short description of the example and the expected result should able be included.

ROS/ROS2 documentation

If the nodes are perception nodes, new entries should be added in the ROS README index and ROS2 README index, that link to the corresponding entries on the main ROS README and main ROS2 README respectively.

If any new bridge methods are added in the opendr_bridge packages, they should be documented in ROS opendr_bridge doc and ROS2 opendr_bridge doc respectively.

Unit tests

Unit tests are run by the CI system regularly (e.g., every night) or upon request from a GitHub Pull Request. For each tool, we should provide some tests to check the main tool functionalities and check for regression issues. The test file should be added at /test/sources/tools/<type>/<task_name>/<algorithm_name>. For Python implementation, the Python unittest framework is used.

If you add a new folder for testing your tool, then make sure that it is included in the test package list at: https://github.com/opendr-eu/opendr/blob/master/.github/workflows/tests_suite.yml A total of four entries should be added in the following jobs:

  • test-tools: always
  • test-wheel: only if a pip wheel should be created for the added tool, in which case it should also be added to ./packages.txt.
  • test-wheel-separate: only if a pip wheel should be created for the added tool, in which case it should also be added to ./packages.txt.
  • test-docker: always

If the test needed specific dependencies besides the tool dependencies, they should be added in /tests/sources/requirements.txt. This file uses the standard pip requirements file format.

Example

The example files have to be added to a dedicated folder in the /projects folder. For example: /projects/<type>/<task_name>/algorithm_name The example should be as simple as possible but still providing a meaningful application where the tool's potentialities are demonstrated. It should preferably be a Webots project.

Keeping in mind that OpenDR is a set of many different tools it is important to carefully choose the dependencies and try when possible and meaningful to reuse dependencies already included instead of adding new ones. New dependencies needed to run this specific example and that are not generically needed to run the tool, should be listed in the /projects/<type>/<task_name>/<algorithm_name>/dependencies.ini file. More information about listing dependencies is specified in the Dependencies section.

The example should be designed in such a way that it is as simple and fast to execute as possible but still providing a meaningful application example.

PR checklist

Please check all these points before contributing a tool:

  1. The main functionality is provided in a Learner-based class.
  2. infer(), train() and eval() work correctly with OpenDR data types (e.g., Dataset, Image, Target, etc.).
  3. save() and load() functions are provided.
  4. Any binary models or data are removed from the contributed files. Binary models and/or data should go to the FTP server.
  5. There is no duplicate functionality in the code that is provided elsewhere (e.g. pip package, other repo that can be linked, etc.). If so, this should not be re-included in the repository.
  6. There is no leftover functionality that is not going to be used in the contributed code. If so, this should be removed to make the code easier to maintain.
  7. Appropriate tests to ensure that the code runs correctly (at least one interface check that checks that model works correctly with OpenDR datatypes and at least one functionality check with a pre-trained model).
  8. A continuous integration test for the tool has been added to .github/workflows/tests_sources.yml, .github/workflows/tests_suite.yml and .github/workflows/tests_suite_develop.yml.
  9. Documentation has been included:
  • A markdown document should be included under ./docs/reference, choosing a meaningful name like for example lightweight-open-pose.md. The document should state all the methods provided by the tool, the arguments it takes, the default values and lastly a snippet about how to use the tool.
  • The documentation index must reference the tool documentation.
  • The demo provided along with the tool should include a README.md file that explains how the demo can be run and which are the available parameters.
  • The documentation index must reference the demo.
  1. ROS and ROS2 nodes are provided, and their documentation entries are added.
  2. At least one pre-trained model is provided.
  3. A meaningful name has been used for the PR.
  4. Make sure you update the documentation index (please also refer to https://github.com/opendr-eu/opendr/blob/master/docs/reference/index.md)
  5. Make sure that an appropriate __init__.py file has been created to expose the public interface of your implementation.
  6. A changelog entry should be added about the added tool in the CHANGELOG.md file.

Nice to have (not mandatory at the moment):

  1. C-based Inference Interface
  2. DL model optimization targeting specific platforms based on a third-party tool (e.g., ONNX, TensorRT, etc.)
Clone this wiki locally