Skip to content

Commit

Permalink
feat(bff) crud for api/v1/model_registry/{model_registry_id}/artifacts (
Browse files Browse the repository at this point in the history
#720)

* feat(bff) crud for api/v1/model_registry/{model_registry_id}/artifacts

Signed-off-by: Eder Ignatowicz <ignatowicz@gmail.com>

* fixing lint

Signed-off-by: Eder Ignatowicz <ignatowicz@gmail.com>

---------

Signed-off-by: Eder Ignatowicz <ignatowicz@gmail.com>
  • Loading branch information
ederign authored Jan 21, 2025
1 parent 8a39ad5 commit 39beea8
Show file tree
Hide file tree
Showing 12 changed files with 491 additions and 42 deletions.
115 changes: 98 additions & 17 deletions clients/ui/bff/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
# Kubeflow Model Registry UI BFF

The Kubeflow Model Registry UI BFF is the _backend for frontend_ (BFF) used by the Kubeflow Model Registry UI.

## Pre-requisites:

### Dependencies
### Dependencies

- Go >= 1.22.2

### Running model registry & ml-metadata
To be operational, our BFF needs the Model Registry & Ml-metadata backend running.

To be operational, our BFF needs the Model Registry & Ml-metadata backend running.

> **NOTE:** Docker compose must be installed in your environment.
Expand All @@ -24,37 +27,46 @@ When shutting down the docker compose, you might want to clean-up the SQLite db
# Development

Run the following command to build the BFF:

```shell
make build
```

After building it, you can run our app with:

```shell
make run
```

If you want to use a different port, mock kubernetes client or model registry client - useful for front-end development, you can run:

```shell
make run PORT=8000 MOCK_K8S_CLIENT=true MOCK_MR_CLIENT=true
```

If you want to change the log level on deployment, add the LOG_LEVEL argument when running, supported levels are: ERROR, WARN, INFO, DEBUG. The default level is INFO.

```shell
# Run with debug logging
make run LOG_LEVEL=DEBUG
make run LOG_LEVEL=DEBUG
```

# Building and Deploying

Run the following command to build the BFF:

```shell
make build
```

The BFF binary will be inside `bin` directory

You can also build BFF docker image with:

```shell
make docker-build
```


## Getting started

### Endpoints
Expand All @@ -66,40 +78,48 @@ See the [OpenAPI specification](../api/openapi/mod-arch.yaml) for a complete lis
You will need to inject your requests with a `kubeflow-userid` header and namespace for authorization purposes.

When running the service with the mocked Kubernetes client (MOCK_K8S_CLIENT=true), the user `user@example.com` is preconfigured with the necessary RBAC permissions to perform these actions.

```
# GET /v1/healthcheck
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/healthcheck"
```

```
# GET /v1/user
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/user"
```

```
# GET /v1/namespaces (only works when DEV_MODE=true)
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/namespaces"
```

```
# GET /v1/model_registry
# GET /v1/model_registry
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/model_registry?namespace=kubeflow"
```

```
# GET /v1/model_registry using groups permissions
curl -i \
curl -i \
-H "kubeflow-userid: non-user@example.com" \
-H "kubeflow-groups: dora-namespace-group ,group2,group3" \
"http://localhost:4000/api/v1/model_registry?namespace=dora-namespace"
```

```
# GET /v1/model_registry/{model_registry_id}/registered_models
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/model_registry/model-registry/registered_models?namespace=kubeflow"
```

```
# GET /v1/model_registry/{model_registry_id}/registered_models using group permissions
curl -i \
-H "kubeflow-userid: non-user@example.com" \
-H "kubeflow-groups: dora-namespace-group ,dora-service-group,group3" \
"http://localhost:4000/api/v1/model_registry/model-registry-dora/registered_models?namespace=dora-namespace"
```

```
#POST /v1/model_registry/{model_registry_id}/registered_models
curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/api/v1/model_registry/model-registry/registered_models?namespace=kubeflow" \
Expand All @@ -118,10 +138,12 @@ curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/ap
"state": "LIVE"
}}'
```

```
# GET /v1/model_registry/{model_registry_id}/registered_models/{registered_model_id}
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/model_registry/model-registry/registered_models/1?namespace=kubeflow"
```

```
# PATCH /v1/model_registry/{model_registry_id}/registered_models/{registered_model_id}
curl -i -H "kubeflow-userid: user@example.com" -X PATCH "http://localhost:4000/api/v1/model_registry/model-registry/registered_models/1?namespace=kubeflow" \
Expand All @@ -130,14 +152,17 @@ curl -i -H "kubeflow-userid: user@example.com" -X PATCH "http://localhost:4000/a
"description": "New description"
}}'
```

```
# GET /api/v1/model_registry/{model_registry_id}/model_versions
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/model_versions?namespace=kubeflow"
```

```
# GET /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}
# GET /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/model_versions/1?namespace=kubeflow"
```

```
# POST /api/v1/model_registry/{model_registry_id}/model_versions
curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/api/v1/model_registry/model-registry/model_versions?namespace=kubeflow" \
Expand All @@ -157,6 +182,7 @@ curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/ap
"registeredModelId": "1"
}}'
```

```
# PATCH /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}
curl -i -H "kubeflow-userid: user@example.com" -X PATCH "http://localhost:4000/api/v1/model_registry/model-registry/model_versions/1?namespace=kubeflow" \
Expand All @@ -165,10 +191,12 @@ curl -i -H "kubeflow-userid: user@example.com" -X PATCH "http://localhost:4000/a
"description": "New description 2"
}}'
```

```
# GET /v1/model_registry/{model_registry_id}/registered_models/{registered_model_id}/versions
curl -i -H "kubeflow-userid: user@example.com" "localhost:4000/api/v1/model_registry/model-registry/registered_models/1/versions?namespace=kubeflow"
```

```
# POST /v1/model_registry/{model_registry_id}/registered_models/{registered_model_id}/versions
curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/api/v1/model_registry/model-registry/registered_models/1/versions?namespace=kubeflow" \
Expand All @@ -188,10 +216,12 @@ curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/ap
"registeredModelId": "1"
}}'
```

```
# GET /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}/artifacts
# GET /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}/artifacts
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/model_versions/1/artifacts?namespace=kubeflow"
```

```
# POST /api/v1/model_registry/{model_registry_id}/model_versions/{model_version_id}/artifacts
curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/api/v1/model_registry/model-registry/model_versions/1/artifacts?namespace=kubeflow" \
Expand All @@ -211,65 +241,113 @@ curl -i -H "kubeflow-userid: user@example.com" -X POST "http://localhost:4000/ap
}}'
```

```
# GET /api/v1/model_registry/{model_registry_id}/artifacts
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/artifacts?namespace=kubeflow"
```

```
# GET /api/v1/model_registry/{model_registry_id}/artifacts/{artifact_id}
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/artifacts/{artifact_id}?namespace=kubeflow"
```

```
# POST /api/v1/model_registry/{model_registry_id}/artifacts
curl -i \
-H "kubeflow-userid: user@example.com" \
-X POST "http://localhost:4000/api/v1/model_registry/model-registry/artifacts?namespace=kubeflow" \
-H "Content-Type: application/json" \
-d '{
"data": {
"artifactType": "model-artifact",
"name": "dora-classifier-v2",
"description": "MNIST digit classification model trained on TensorFlow",
"uri": "gs://my-models/mnist-classifier/v2",
"externalId": "model-12345678",
"modelFormatName": "tensorflow",
"modelFormatVersion": "2.9.0",
"storageKey": "models/mnist/1.0.0",
"storagePath": "/models/mnist/1.0.0/model.savedmodel"
}
}'
```

```
# PATCH /api/v1/model_registry/{model_registry_id}/artifacts/{artifact_id}
curl -i -H "kubeflow-userid: user@example.com" -X PATCH "http://localhost:4000/api/v1/model_registry/model-registry/artifacts/1?namespace=kubeflow" \
-H "Content-Type: application/json" \
-d '{ "data": {
"artifactType": "model-artifact",
"description": "New description 2"
}}'
```

### Pagination

The following query parameters are supported by "Get All" style endpoints to control pagination.

| Parameter Name | Description |
|----------------|-----------------------------------------------------------------------------------------------------------|
| -------------- | --------------------------------------------------------------------------------------------------------- |
| pageSize | Number of entities in each page |
| orderBy | Specifies the order by criteria for listing entities. Available values: CREATE_TIME, LAST_UPDATE_TIME, ID |
| sortOrder | Specifies the sort order for listing entities. Available values: ASC, DESC. Default: ASC |
| nextPageToken | Token to use to retrieve next page of results. |
| nextPageToken | Token to use to retrieve next page of results. |

### Sample local calls

```
# Get with a page size of 5 getting a specific page.
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/registered_models?pageSize=5&nextPageToken=CAEQARoCCAE"
```

```
# Get with a page size of 5, order by last update time in descending order.
curl -i -H "kubeflow-userid: user@example.com" "http://localhost:4000/api/v1/model_registry/model-registry/registered_models?pageSize=5&orderBy=LAST_UPDATE_TIME&sortOrder=DESC"
```


### FAQ

#### 1. How do we filter model registry services from other Kubernetes services?

We filter Model Registry services by using the Kubernetes label `component: model-registry. This label helps distinguish Model Registry services from other services in the cluster.

For example, in our service manifest, the `component label is defined as follows:

```yaml
# ...
labels:
# ...
component: model-registry
#...
```

You can view the complete Model Registry service manifest [here](https://github.com/kubeflow/model-registry/blob/main/manifests/kustomize/base/model-registry-service.yaml#L10).

#### 2. What is the structure of the mock Kubernetes environment?

The mock Kubernetes environment is activated when the environment variable `MOCK_K8S_CLIENT` is set to `true`. It is based on `env-test` and is designed to simulate a realistic Kubernetes setup for testing. The mock has the following characteristics:

- **Namespaces**:

- `kubeflow`
- `dora-namespace`
- `bella-namespace`

- **Users**:

- `user@example.com` (has `cluster-admin` privileges)
- `doraNonAdmin@example.com` (restricted to the `dora-namespace`)
- `bellaNonAdmin@example.com` (restricted to the `bella-namespace`)

- **Groups**:
- `dora-service-group` (has access to `model-registry-dora` inside `dora-namespace`)
- `dora-namespace-group` (has access to the `dora-namespace`)

- `dora-namespace-group` (has access to the `dora-namespace`)
- **Services (Model Registries)**:
- `model-registry`: resides in the `kubeflow` namespace with the label `component: model-registry`.
- `model-registry-one`: resides in the `kubeflow` namespace with the label `component: model-registry`.
- `non-model-registry`: resides in the `kubeflow` namespace *without* the label `component: model-registry`.
- `non-model-registry`: resides in the `kubeflow` namespace _without_ the label `component: model-registry`.
- `model-registry-dora`: resides in the `dora-namespace` namespace with the label `component: model-registry`.

#### 3. How BFF authorization works for kubeflow-userid and kubeflow-groups?
Expand All @@ -279,22 +357,24 @@ Authorization is performed using Kubernetes SubjectAccessReview (SAR), which val
- `kubeflow-userid`: Required header that specifies the user’s email. Access is checked directly for the user via SAR.
- `kubeflow-groups`: Optional header with a comma-separated list of groups. If the user does not have access, SAR checks group permissions using OR logic. If any group has access, the request is authorized.


Access to Model Registry List:

- To list all model registries (/v1/model_registry), we perform a SAR check for get and list verbs on services within the specified namespace.
- If the user or any group has permission to get and list services in the namespace, the request is authorized.

Access to Specific Model Registry Endpoints:

- For other endpoints (e.g., /v1/model_registry/{model_registry_id}/...), we perform a SAR check for get and list verbs on the specific service (identified by model_registry_id) within the namespace.
- If the user or any group has permission to get or list the specific service, the request is authorized.

#### 4. How do I allow CORS requests from other origins

When serving the UI directly from the BFF there is no need for any CORS headers to be served, by default they are turned off for security reasons.
When serving the UI directly from the BFF there is no need for any CORS headers to be served, by default they are turned off for security reasons.

If you need to enable CORS for any reasons you can add origins to the allow-list in several ways:

##### Via the make command

Add the following parameter to your command: `ALLOWED_ORIGINS` this takes a comma separated list of origins to permit serving to, alterantively you can specify the value `*` to allow all origins, **Note this is not recommended in production deployments as it poses a security risk**

Examples:
Expand All @@ -314,14 +394,15 @@ make run ALLOWED_ORIGINS=""
```

#### Via environment variable

Setting CORS via environment variable follows the same rules as using the Makefile, simply set the environment variable `ALLOWED_ORIGINS` with the same value as above.

#### Via the command line arguments

Setting CORS via command line arguments follows the same rules as using the Makefile. Simply add the `--allowed-origins=` flag to your command.

Examples:

```shell
./bff --allowed-origins="http://my-domain.com,http://my-other-domain.com"
```

Loading

0 comments on commit 39beea8

Please sign in to comment.