Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingestion/mlflow): improve mlflow connector to pull run and experiments #12587

Open
wants to merge 45 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
fbd38e1
initial commit
yoonhyejin Feb 10, 2025
de57fcd
fix typo
yoonhyejin Feb 10, 2025
1a7a656
fix dpi urn generation logic
yoonhyejin Feb 10, 2025
0351a7c
update mlflow script
yoonhyejin Feb 10, 2025
65d2a96
make updates to versions / experiments
yoonhyejin Feb 13, 2025
9701915
update dpi creation process
yoonhyejin Feb 14, 2025
7ced7d8
lintFix
yoonhyejin Feb 17, 2025
2dec34c
fix mypy
yoonhyejin Feb 17, 2025
d200082
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Feb 17, 2025
1932262
fix lint
yoonhyejin Feb 17, 2025
7a83011
revert unwanted changes
yoonhyejin Feb 17, 2025
ecbac5c
revert back unwanted files
yoonhyejin Feb 17, 2025
bdeb206
add cypress test on experiment & runs
yoonhyejin Feb 18, 2025
b592b51
fix cypress lint
yoonhyejin Feb 18, 2025
5d45632
fix cypress lint
yoonhyejin Feb 18, 2025
772548f
update golden files
yoonhyejin Feb 19, 2025
da5b4d7
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Feb 19, 2025
dcccc61
update golden files
yoonhyejin Feb 19, 2025
785e60b
update cypress test
yoonhyejin Feb 19, 2025
ca6bded
fix mlflow test to generate static id/outputurls
yoonhyejin Feb 19, 2025
c1b223b
fix lint
yoonhyejin Feb 19, 2025
6dbe505
update golden files
yoonhyejin Feb 19, 2025
cde3b23
address pr reviews
yoonhyejin Feb 20, 2025
32139b0
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Feb 20, 2025
a252310
fix lint
yoonhyejin Feb 20, 2025
2b378e7
fix tests
yoonhyejin Feb 20, 2025
82fb534
fix tests
yoonhyejin Feb 20, 2025
4ace078
fix tests
yoonhyejin Feb 20, 2025
d624777
revert unnecessary changes
yoonhyejin Feb 20, 2025
10cc89a
reflect code review
yoonhyejin Feb 21, 2025
260447d
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Feb 21, 2025
6ba92e9
revert unncessary changes
yoonhyejin Feb 21, 2025
5cbc2fc
revert unncessary changes
yoonhyejin Feb 21, 2025
1163855
fix ingestion tests
yoonhyejin Feb 21, 2025
e23d5f3
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Feb 24, 2025
e62f3df
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Mar 4, 2025
b947eec
modify versio-related stuff
yoonhyejin Mar 4, 2025
4f12eb2
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Mar 4, 2025
c0100cd
add more cypress tests
yoonhyejin Mar 4, 2025
5d67288
delete versionset and add sortid
yoonhyejin Mar 5, 2025
eb88a71
update golden files
yoonhyejin Mar 5, 2025
ddc8161
include value in model tag ingestion
yoonhyejin Mar 5, 2025
bc550e6
Merge branch 'master' into mlflow-connector-latest
yoonhyejin Mar 5, 2025
da253bb
reflect code review
yoonhyejin Mar 7, 2025
ea6e399
remove todo comment
yoonhyejin Mar 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
source:
type: mlflow
config:
tracking_uri: "http://127.0.0.1:5000"

sink:
type: datahub-rest
config:
server: "http://localhost:8080"
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,8 @@ class BIAssetSubTypes(StrEnum):
# SAP Analytics Cloud
SAC_STORY = "Story"
SAC_APPLICATION = "Application"


class MLAssetSubTypes(StrEnum):
MLFLOW_TRAINING_RUN = "ML Training Run"
MLFLOW_EXPERIMENT = "ML Experiment"
Loading
Loading