Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure Project Structure and Refactor Imports for et_replay #111

Conversation

TaekyungHeo
Copy link
Contributor

@TaekyungHeo TaekyungHeo commented May 10, 2024

Summary

  • Created pyproject.toml for package setup and script entry points.
  • Refactored import paths in et_replay.py and validate_trace.py to align with new package structure.
  • Added .gitignore to exclude build/ and et_replay.egg-info/.

Test Plan

$ cd et_replay

$ pip install .
Processing /Users/theo/param/et_replay
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: et_replay
  Building wheel for et_replay (pyproject.toml) ... done
  Created wheel for et_replay: filename=et_replay-1.0.0-py3-none-any.whl size=59701 sha256=8bf68250c2b168b1e67ab75ea3d2059d018e2738ec9437bc712c2e3f00da6511
  Stored in directory: /private/var/folders/z0/c9mq5j4s6n14n0_gs7nlt6mc0000gp/T/pip-ephem-wheel-cache-_rrl79h0/wheels/3b/3f/aa/d3fc853f83c22c6f3eeb09763570c2cc8031a1a414cb3c18b6
Successfully built et_replay
Installing collected packages: et_replay
  Attempting uninstall: et_replay
    Found existing installation: et_replay 1.0.0
    Uninstalling et_replay-1.0.0:
      Successfully uninstalled et_replay-1.0.0
Successfully installed et_replay-1.0.0

$ validate_trace ~/Downloads/llama_pytorch24.05/megatron_et_0.json 
num ops = 51343, num comms = 32, num triton ops = 0
Trace validation result =  False

$ et_replay                                                       
Traceback (most recent call last):
  File "/Users/theo/venv/bin/et_replay", line 5, in <module>
    from et_replay.tools.et_replay import main
  File "/Users/theo/venv/lib/python3.10/site-packages/et_replay/tools/et_replay.py", line 15, in <module>
    from ..lib.et_replay_utils import (
  File "/Users/theo/venv/lib/python3.10/site-packages/et_replay/lib/et_replay_utils.py", line 5, in <module>
    from fbgemm_gpu.split_table_batched_embeddings_ops import PoolingMode, WeightDecayMode
ModuleNotFoundError: No module named 'fbgemm_gpu'

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 10, 2024
@TaekyungHeo TaekyungHeo changed the title Add .gitignore and pyproject.toml for et_replay Configure Project Structure and Refactor Imports for et_replay May 10, 2024
@TaekyungHeo TaekyungHeo force-pushed the et-replay-refactor-install branch 5 times, most recently from f751d0c to 4e4473c Compare May 10, 2024 17:12
et_replay/pyproject.toml Outdated Show resolved Hide resolved
et_replay/tools/et_replay.py Show resolved Hide resolved
@TaekyungHeo TaekyungHeo force-pushed the et-replay-refactor-install branch 3 times, most recently from 5b481f3 to 2ffb6a4 Compare May 14, 2024 03:03
@TaekyungHeo TaekyungHeo force-pushed the et-replay-refactor-install branch from 2ffb6a4 to 3a16df5 Compare May 14, 2024 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants