feat: implement orb discovery

.gitignore

@@ -0,0 +1,3 @@
+build/
+__pycache__/
+.vscode/

README.md

@@ -1 +1,183 @@
-# orb-discovery
+# orb-discovery
+
+
+### Config RFC
+```yaml
+discovery:
+  config:
+    target: grpc://localhost:8080/diode
+    api_key: ${DIODE_API_KEY}
+```
+
+### Policy RFC
+```yaml
+discovery_1:
+    config:
+        rerun_interval: 10s
+        netbox:
+            site: New York NY
+    data:
+    - hostname: 192.168.0.32
+      username: ${USER}
+      password: admin
+    - driver: eos
+      hostname: 127.0.0.1
+      username: admin
+      password: ${ARISTA_PASSWORD}
+      optional_args:
+        enable_password: ${ARISTA_PASSWORD}
+```
+
+## REST API
+The default `discovery` address is `localhost:8072`. to change that you can specify host and port when starting `discovery`:
+```sh
+docker build orb-
+```
+
+### Routes (v1)
+
+#### Get runtime and capabilities information
+
+<details>
+ <summary><code>GET</code> <code><b>/api/v1/status</b></code> <code>(gets discovery runtime data)</code></summary>
+
+##### Parameters
+
+> None
+
+##### Responses
+
+> | http code     | content-type                      | response                                                            |
+> |---------------|-----------------------------------|---------------------------------------------------------------------|
+> | `200`         | `application/json; charset=utf-8` | JSON data                                                           |
+
+##### Example cURL
+
+> ```javascript
+>  curl -X GET -H "Content-Type: application/json" http://localhost:8072/api/v1/status
+> ```
+
+</details>
+
+<details>
+ <summary><code>GET</code> <code><b>/api/v1/capabilities</b></code> <code>(gets otelcol-contrib capabilities)</code></summary>
+
+##### Parameters
+
+> None
+
+##### Responses
+
+> | http code     | content-type                      | response                                                            |
+> |---------------|-----------------------------------|---------------------------------------------------------------------|
+> | `200`         | `application/json; charset=utf-8` | JSON data                                                           |
+
+##### Example cURL
+
+> ```javascript
+>  curl -X GET -H "Content-Type: application/json" http://localhost:8072/api/v1/capabilities
+> ```
+
+</details>
+
+#### Policies Management
+
+<details>
+ <summary><code>GET</code> <code><b>/api/v1/policies</b></code> <code>(gets all existing policies)</code></summary>
+
+##### Parameters
+
+> None
+
+##### Responses
+
+> | http code     | content-type                      | response                                                            |
+> |---------------|-----------------------------------|---------------------------------------------------------------------|
+> | `200`         | `application/json; charset=utf-8` | JSON array containing all applied policy names                      |
+
+##### Example cURL
+
+> ```javascript
+>  curl -X GET -H "Content-Type: application/json" http://localhost:8072/api/v1/policies
+> ```
+
+</details>
+
+
+<details>
+ <summary><code>POST</code> <code><b>/api/v1/policies</b></code> <code>(Creates a new policy)</code></summary>
+
+##### Parameters
+
+> | name      |  type     | data type               | description                                                           |
+> |-----------|-----------|-------------------------|-----------------------------------------------------------------------|
+> | None      |  required | YAML object             | yaml format specified in [Policy RFC](#policy-rfc-v1)                 |
+
+
+##### Responses
+
+> | http code     | content-type                       | response                                                            |
+> |---------------|------------------------------------|---------------------------------------------------------------------|
+> | `201`         | `application/x-yaml; charset=UTF-8`| YAML object                                                         |
+> | `400`         | `application/json; charset=UTF-8`  | `{ "message": "invalid Content-Type. Only 'application/x-yaml' is supported" }`|
+> | `400`         | `application/json; charset=UTF-8`  | Any policy error                                                    |
+> | `400`         | `application/json; charset=UTF-8`  | `{ "message": "only single policy allowed per request" }`           |
+> | `403`         | `application/json; charset=UTF-8`  | `{ "message": "config field is required" }`                         |
+> | `409`         | `application/json; charset=UTF-8`  | `{ "message": "policy already exists" }`                            |
+
+
+##### Example cURL
+
+> ```javascript
+>  curl -X POST -H "Content-Type: application/x-yaml" --data @post.yaml http://localhost:8072/api/v1/policies
+> ```
+
+</details>
+
+<details>
+ <summary><code>GET</code> <code><b>/api/v1/policies/{policy_name}</b></code> <code>(gets information of a specific policy)</code></summary>
+
+##### Parameters
+
+> | name              |  type     | data type      | description                         |
+> |-------------------|-----------|----------------|-------------------------------------|
+> |   `policy_name`   |  required | string         | The unique policy name              |
+
+##### Responses
+
+> | http code     | content-type                        | response                                                            |
+> |---------------|-------------------------------------|---------------------------------------------------------------------|
+> | `200`         | `application/x-yaml; charset=UTF-8` | YAML object                                                         |
+> | `404`         | `application/json; charset=UTF-8`   | `{ "message": "policy not found" }`                                 |
+
+##### Example cURL
+
+> ```javascript
+>  curl -X GET http://localhost:8072/api/v1/policies/my_policy
+> ```
+
+</details>
+
+<details>
+ <summary><code>DELETE</code> <code><b>/api/v1/policies/{policy_name}</b></code> <code>(delete a existing policy)</code></summary>
+
+##### Parameters
+
+> | name              |  type     | data type      | description                         |
+> |-------------------|-----------|----------------|-------------------------------------|
+> |   `policy_name`   |  required | string         | The unique policy name              |
+
+##### Responses
+
+> | http code     | content-type                      | response                                                            |
+> |---------------|-----------------------------------|---------------------------------------------------------------------|
+> | `200`         | `application/json; charset=UTF-8` | `{ "message": "my_policy was deleted" }`                            |
+> | `404`         | `application/json; charset=UTF-8` | `{ "message": "policy not found" }`                                 |
+
+##### Example cURL
+
+> ```javascript
+>  curl -X DELETE http://localhost:8072/api/v1/policies/my_policy
+> ```
+
+</details>

orb-discovery/orb_discovery/__init__.py

@@ -0,0 +1,3 @@
+#!/usr/bin/env python
+# Copyright 2024 NetBox Labs Inc
+"""NetBox Labs - Orb Discovery namespace."""

orb-discovery/orb_discovery/cli/__init__.py

@@ -0,0 +1,3 @@
+#!/usr/bin/env python
+# Copyright 2024 NetBox Labs Inc
+"""NetBox Labs - CLI namespace."""

orb-discovery/orb_discovery/cli/cli.py

@@ -0,0 +1,164 @@
+#!/usr/bin/env python
+# Copyright 2024 NetBox Labs Inc
+"""Diode NAPALM Agent CLI."""
+
+import argparse
+import logging
+import sys
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from importlib.metadata import version
+
+import netboxlabs.diode.sdk.version as SdkVersion
+from dotenv import load_dotenv
+from napalm import get_network_driver
+
+from orb_discovery.client import Client
+from orb_discovery.discovery import discover_device_driver, supported_drivers
+from orb_discovery.parser import (
+    Diode,
+    DiscoveryConfig,
+    Napalm,
+    Policy,
+    parse_config_file,
+)
+from orb_discovery.version import version_semver
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+def run_driver(info: Napalm, config: DiscoveryConfig):
+    """
+    Run the device driver code for a single info item.
+
+    Args:
+    ----
+        info: Information data for the device.
+        config: Configuration data containing site information.
+
+    """
+    if info.driver is None:
+        logger.info(f"Hostname {info.hostname}: Driver not informed, discovering it")
+        info.driver = discover_device_driver(info)
+        if not info.driver:
+            raise Exception(
+                f"Hostname {info.hostname}: Not able to discover device driver"
+            )
+    elif info.driver not in supported_drivers:
+        raise Exception(
+            f"Hostname {info.hostname}: specified driver '{info.driver}' was not found in the current installed drivers list: "
+            f"{supported_drivers}.\nHINT: If '{info.driver}' is a napalm community driver, try to perform the following command:"
+            f"\n\n\tpip install napalm-{info.driver.replace('_', '-')}\n"
+        )
+
+    logger.info(f"Hostname {info.hostname}: Get driver '{info.driver}'")
+    np_driver = get_network_driver(info.driver)
+    logger.info(f"Hostname {info.hostname}: Getting information")
+    with np_driver(
+        info.hostname, info.username, info.password, info.timeout, info.optional_args
+    ) as device:
+        data = {
+            "driver": info.driver,
+            "site": config.netbox.get("site", None),
+            "device": device.get_facts(),
+            "interface": device.get_interfaces(),
+            "interface_ip": device.get_interfaces_ip(),
+        }
+        Client().ingest(info.hostname, data)
+
+
+def start_policy(name: str, cfg: Policy, max_workers: int):
+    """
+    Start the policy for the given configuration.
+
+    Args:
+    ----
+        name: Policy name
+        cfg: Configuration data for the policy.
+        max_workers: Maximum number of threads in the pool.
+
+    """
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        futures = [executor.submit(run_driver, info, cfg.config) for info in cfg.data]
+
+        for future in as_completed(futures):
+            try:
+                future.result()
+            except Exception as e:
+                logger.error(f"Error while processing policy {name}: {e}")
+
+
+def start_agent(cfg: Diode, workers: int):
+    """
+    Start the diode client and execute policies.
+
+    Args:
+    ----
+        cfg: Configuration data containing policies.
+        workers: Number of workers to be used in the thread pool.
+
+    """
+    client = Client()
+    client.init_client(target=cfg.config.target, api_key=cfg.config.api_key)
+    for policy_name in cfg.policies:
+        start_policy(policy_name, cfg.policies.get(policy_name), workers)
+
+
+def main():
+    """
+    Main entry point for the Diode NAPALM Agent CLI.
+
+    Parses command-line arguments and starts the agent.
+    """
+    parser = argparse.ArgumentParser(description="Diode Agent for NAPALM")
+    parser.add_argument(
+        "-V",
+        "--version",
+        action="version",
+        version=f"Diode Agent version: {version_semver()}, NAPALM version: {version('napalm')}, "
+        f"Diode SDK version: {SdkVersion.version_semver()}",
+        help="Display Diode Agent, NAPALM and Diode SDK versions",
+    )
+    parser.add_argument(
+        "-c",
+        "--config",
+        metavar="config.yaml",
+        help="Agent yaml configuration file",
+        type=str,
+        required=True,
+    )
+    parser.add_argument(
+        "-e",
+        "--env",
+        metavar=".env",
+        help="File containing environment variables",
+        type=str,
+    )
+    parser.add_argument(
+        "-w",
+        "--workers",
+        metavar="N",
+        help="Number of workers to be used",
+        type=int,
+        default=2,
+    )
+    args = parser.parse_args()
+
+    if hasattr(args, "env") and args.env is not None:
+        if not load_dotenv(args.env, override=True):
+            sys.exit(
+                f"ERROR: Unable to load environment variables from file {args.env}"
+            )
+
+    try:
+        config = parse_config_file(args.config)
+        start_agent(config, args.workers)
+    except (KeyboardInterrupt, RuntimeError):
+        pass
+    except Exception as e:
+        sys.exit(f"ERROR: Unable to start agent: {e}")
+
+
+if __name__ == "__main__":
+    main()