Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memory report visualization into tt explorer #1618

Open
tapspatel opened this issue Dec 17, 2024 · 6 comments · May be fixed by #2319
Open

Add memory report visualization into tt explorer #1618

tapspatel opened this issue Dec 17, 2024 · 6 comments · May be fixed by #2319
Assignees
Labels
explorer Issues related to Explorer Visualization tool good-first-issue Good for newcomers

Comments

@tapspatel
Copy link
Collaborator

#1616

Once this PR lands, you will be able to get a dump of the device state memory after every op. Usage:

ttrt run <flatbuffer> --memory

3 files are produced:

  • detailed_memory_usage.csv
  • l1_usage_summary.csv
  • memory_usage_summary.csv

They will be produced under the artifacts directory, with the name of the parent folder being the loc() of that op that was run.

@tapspatel
Copy link
Collaborator Author

tapspatel commented Jan 21, 2025

Edit: this dump was updated with this PR: #1726. Running ttrt run out.ttnn --memory --save-artifacts will output memory report file.

@vprajapati-tt vprajapati-tt added the good-first-issue Good for newcomers label Feb 4, 2025
@vprajapati-tt
Copy link
Contributor

@tapspatel This would probably be a good introductory issue for anyone onboarding onto the explorer project. I can create a more detailed roadmap if anyone new is going to take up this issue.

@odjuricicTT
Copy link
Contributor

@vprajapati-tt we discussed this briefly offline. I think that there is a very detailed memory report in ttnn-visualizer and that we might not need to recreate this feature in explorer.

Using ttnn-visualizer reports from ttrt execution is going to be a requirement by the perf team, so i'd like to test this flow out first and if it's not sufficient we should discuss adding this to explorer. Opened an issue here #2110

@vprajapati-tt
Copy link
Contributor

Yeah, for detailed memory reports I definitely suggest using ttnn-visualizer. That being said, this visualization provides a single attribute to be viewed after execution for a holistic view of DRAM and L1 usage (as a percentage). I agree, let's test this out and see what the flow looks like.

@odjuricicTT
Copy link
Contributor

@vprajapati-tt Yes, i was referencing a more detailed view of the memory. Having a single percentage is fine now.

@vprajapati-tt
Copy link
Contributor

@jgrimTT Here's a plan to action for taking this on as a first issue!

  1. Visualizing the holistic memory usage for DRAM and L1 requires polling for the information first. This will be done through the execution command on tt_adapter. You will need to find where the "3 files produced" are stored relative to the ttrt_artifacts directory and create some helper function in tt_adapter.runner.ModelRunner that get_memory_usage given a model_path.
  2. Once execution is finished, a convert call will be placed. Try and mirror how get_perf_trace gets called in tt_adapter:main:TTAdapter:convert and passed into the build_graph function.
  3. In tt_adapter:mlir:build_graph you can then use the given memory_usage data to create a new attribute for each op based on the location in the memory_usage that contains a percentage of MEMORY_USED/TOTAL_MEMORY. Do this for both L1 and DRAM.
  4. The frontend has an unique visualizer for these percentages (Feature: memory info model-explorer#23). Enable the memory visualization on the percentage using add_to_dataclass in tt_adapter.util.

This should provide a brief overview of where to look in the code but I may have omitted some more detailed semantics. Feel free to ping for help, but this should be the rough roadmap to implement this feature :)

@jgrimTT jgrimTT linked a pull request Feb 27, 2025 that will close this issue
@vprajapati-tt vprajapati-tt linked a pull request Feb 27, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
explorer Issues related to Explorer Visualization tool good-first-issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants