Bringup tt-torch models in forge #1314

kamalrajkannan78 · 2025-02-25T05:54:18Z

Summary

This PR addresses Issue #1321 by porting models from tt-torch to tt-forge. The current compilation status of these models are available in the logs.
A list of skipped models along with the reasons for their exclusion can be found in Issue #1339. Any models that were missed in this PR will be included in PR #1337.

Note:

The following test cases are passing end-to-end (E2E), so a push marker & post processing steps has been added:

forge/test/models/pytorch/text/albert/test_albert.py::test_albert_sequence_classification_pytorch
forge/test/models/pytorch/text/perceiverio/test_perceiverio.py::test_perceiverio_masked_lm_pytorch
forge/test/models/pytorch/text/bert/test_bert.py::test_bert_question_answering_pytorch[phiyodr/bert-large-finetuned-squad2]

Logs:

github-actions · 2025-02-25T08:53:07Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	604 ran	478 passed	126 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T08:53:45Z

	Tests	Passed ☑️	Skipped ⚠️	Failed ❌️
TT-Forge-FE Tests	604 ran	477 passed	126 skipped	1 failed

Test	Result
TT-Forge-FE Tests
pytest
test_dla.test_dla_pytorch[dla34]	❌ failure

github-actions · 2025-02-25T08:57:41Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	662 ran	525 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T09:04:39Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	662 ran	525 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T10:00:46Z

	Tests	Passed ❌️	Skipped	Failed
TT-Forge-FE Tests	0 ran	0 passed	0 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T10:23:26Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	662 ran	525 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T10:46:46Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	604 ran	478 passed	126 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-02-25T10:49:02Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	662 ran	525 passed	137 skipped	0 failed

Test	Result
No test annotations available

forge/test/models/pytorch/text/mamba/test_mamba.py

github-actions · 2025-02-25T22:15:00Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	4 ran	0 passed	0 skipped	4 failed

Test	Result
TT-Forge-FE Tests
pytest
test_bloom.forge.test.models.pytorch.text.bloom.test_bloom	❌ failure
test_glpn_kitti.forge.test.models.pytorch.vision.glpn_kitti.test_glpn_kitti	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-25T22:15:07Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	4 ran	0 passed	0 skipped	4 failed

Test	Result
TT-Forge-FE Tests
pytest
test_bloom.forge.test.models.pytorch.text.bloom.test_bloom	❌ failure
test_glpn_kitti.forge.test.models.pytorch.vision.glpn_kitti.test_glpn_kitti	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-25T22:16:21Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	4 ran	0 passed	0 skipped	4 failed

Test	Result
TT-Forge-FE Tests
pytest
test_bloom.forge.test.models.pytorch.text.bloom.test_bloom	❌ failure
test_glpn_kitti.forge.test.models.pytorch.vision.glpn_kitti.test_glpn_kitti	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-25T22:16:39Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	4 ran	0 passed	0 skipped	4 failed

Test	Result
TT-Forge-FE Tests
pytest
test_bloom.forge.test.models.pytorch.text.bloom.test_bloom	❌ failure
test_glpn_kitti.forge.test.models.pytorch.vision.glpn_kitti.test_glpn_kitti	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-26T09:46:14Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	3 ran	0 passed	0 skipped	3 failed

Test	Result
TT-Forge-FE Tests
pytest
test_mlp_mixer.forge.test.models.pytorch.vision.mlp_mixer.test_mlp_mixer	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-26T09:50:26Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	3 ran	0 passed	0 skipped	3 failed

Test	Result
TT-Forge-FE Tests
pytest
test_mlp_mixer.forge.test.models.pytorch.vision.mlp_mixer.test_mlp_mixer	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-26T09:53:50Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	3 ran	0 passed	0 skipped	3 failed

Test	Result
TT-Forge-FE Tests
pytest
test_mlp_mixer.forge.test.models.pytorch.vision.mlp_mixer.test_mlp_mixer	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-02-26T09:55:19Z

	Tests	Passed	Skipped	Failed ❌️
TT-Forge-FE Tests	3 ran	0 passed	0 skipped	3 failed

Test	Result
TT-Forge-FE Tests
pytest
test_mlp_mixer.forge.test.models.pytorch.vision.mlp_mixer.test_mlp_mixer	❌ failure
test_rmbg.forge.test.models.pytorch.vision.rmbg.test_rmbg	❌ failure
test_yolos.forge.test.models.pytorch.vision.yolo.test_yolos	❌ failure

github-actions · 2025-03-01T09:43:22Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	677 ran	540 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-01T09:43:27Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	677 ran	540 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-01T09:49:01Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	618 ran	489 passed	129 skipped	0 failed

Test	Result
No test annotations available

forge/test/models/pytorch/text/albert/test_albert.py

forge/test/models/pytorch/text/perceiverio/test_perceiverio.py

github-actions · 2025-03-03T10:42:48Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	618 ran	489 passed	129 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-03T10:43:40Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	677 ran	540 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-03T10:46:03Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	677 ran	540 passed	137 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-03T10:48:51Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	618 ran	489 passed	129 skipped	0 failed

Test	Result
No test annotations available

meenakshiramanathan1 · 2025-03-03T10:51:04Z

LGTM

vkovinicTT · 2025-03-03T11:18:23Z

forge/test/models/pytorch/text/perceiverio/test_perceiverio.py

+    verify(inputs, framework_model, compiled_model)
+
+    # Inference
+    output = compiled_model(*inputs)


Take a look at this PR. When this merges into main, verify() would also return outputs of the forward pass.

vkovinicTT · 2025-03-03T11:25:51Z

forge/test/models/pytorch/vision/mgp_str_base/utils/utils.py

+
+
+def load_input():
+    url = "https://i.postimg.cc/ZKwLg2Gw/367-14.png"


I think it would be nice to have one centralized folder e.g. data_inputs where we would store few images so all models would use the same inputs.

kamalrajkannan78 · 2025-03-04T11:30:11Z

@vkovinicTT. I created a separate issue(#1364) for this suggestion.
We can implement that in future as a separate PR.

vkovinicTT · 2025-03-04T11:33:51Z

@vkovinicTT. I created a separate issue(#1364) for this suggestion. We can implement that in future as a separate PR.

Awesome, thanks :)

vkovinicTT

LGTM, just waiting for the verify() change.

github-actions · 2025-03-04T12:01:32Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	626 ran	486 passed	140 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-04T12:02:40Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	684 ran	544 passed	140 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-04T12:04:30Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	626 ran	486 passed	140 skipped	0 failed

Test	Result
No test annotations available

github-actions · 2025-03-04T12:06:25Z

	Tests	Passed ✅	Skipped ⚠️	Failed
TT-Forge-FE Tests	684 ran	544 passed	140 skipped	0 failed

Test	Result
No test annotations available

kamalrajkannan78 · 2025-03-04T12:20:59Z

LGTM, just waiting for the verify() change.

This verify() change will introduces the bunch of changes in all tests .To unblock this PR, can we handle verify() changes in separate PR ?

ashokkumarkannan1

LGTM! Thanks for the PR @kamalrajkannan78

vkovinicTT · 2025-03-04T13:20:06Z

LGTM, just waiting for the verify() change.

This verify() change will introduces the bunch of changes in all tests .To unblock this PR, can we handle verify() changes in separate PR ?

Yep, go ahead.

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch from 66ff210 to 09430d9 Compare February 25, 2025 09:39

nvukobratTT reviewed Feb 25, 2025

View reviewed changes

forge/test/models/pytorch/text/mamba/test_mamba.py Show resolved Hide resolved

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch 2 times, most recently from 899912e to 823a81e Compare February 25, 2025 19:42

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch from 823a81e to da83043 Compare February 26, 2025 09:11

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch 6 times, most recently from e8ebdde to 8b18209 Compare February 27, 2025 10:21

kamalrajkannan78 marked this pull request as ready for review February 27, 2025 10:22

kamalrajkannan78 requested review from vkovinicTT and mstojkovicTT as code owners February 27, 2025 10:22

meenakshiramanathan1 requested changes Mar 3, 2025

View reviewed changes

forge/test/models/pytorch/text/albert/test_albert.py Show resolved Hide resolved

forge/test/models/pytorch/text/perceiverio/test_perceiverio.py Show resolved Hide resolved

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch from fb01ef7 to 5bd73b7 Compare March 3, 2025 10:01

meenakshiramanathan1 self-requested a review March 3, 2025 10:50

meenakshiramanathan1 approved these changes Mar 3, 2025

View reviewed changes

vkovinicTT reviewed Mar 3, 2025

View reviewed changes

vkovinicTT approved these changes Mar 3, 2025

View reviewed changes

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch from 5bd73b7 to 42c0352 Compare March 3, 2025 18:11

Bringup tt-torch models in forge

eed2d0d

kamalrajkannan78 force-pushed the kkannan/tt_torch_models_bringup branch from 42c0352 to eed2d0d Compare March 4, 2025 11:20

kamalrajkannan78 mentioned this pull request Mar 4, 2025

Use local images instead of URLs in Forge tests #1364

Open

vkovinicTT reviewed Mar 4, 2025

View reviewed changes

ashokkumarkannan1 approved these changes Mar 4, 2025

View reviewed changes

vkovinicTT merged commit 59941b6 into main Mar 4, 2025
10 checks passed

vkovinicTT deleted the kkannan/tt_torch_models_bringup branch March 4, 2025 13:21



		def load_input():
		url = "https://i.postimg.cc/ZKwLg2Gw/367-14.png"

Bringup tt-torch models in forge #1314

Bringup tt-torch models in forge #1314

Conversation

kamalrajkannan78 commented Feb 25, 2025 • edited Loading

Summary

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 25, 2025

github-actions bot commented Feb 26, 2025

github-actions bot commented Feb 26, 2025

github-actions bot commented Feb 26, 2025

github-actions bot commented Feb 26, 2025

github-actions bot commented Mar 1, 2025

github-actions bot commented Mar 1, 2025

github-actions bot commented Mar 1, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 3, 2025

meenakshiramanathan1 commented Mar 3, 2025

vkovinicTT Mar 3, 2025

Choose a reason for hiding this comment

vkovinicTT Mar 3, 2025

Choose a reason for hiding this comment

kamalrajkannan78 commented Mar 4, 2025

vkovinicTT commented Mar 4, 2025 • edited Loading

vkovinicTT left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

kamalrajkannan78 commented Mar 4, 2025

ashokkumarkannan1 left a comment

Choose a reason for hiding this comment

vkovinicTT commented Mar 4, 2025

kamalrajkannan78 commented Feb 25, 2025 •

edited

Loading

vkovinicTT commented Mar 4, 2025 •

edited

Loading