[dicp][ascend] Optimization for dynamic shape code logic. #791
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1.Remove redundant Cast operator. 2.Change logic of Expand shape proxy. 3.Merge output stride executing path.
This reverts commit 9025019.
pdx1989
changed the title
tangzhiyi11
reviewed
May 14, 2024
jinminxi104
requested changes
May 14, 2024
jinminxi104
reviewed
May 14, 2024
tangzhiyi11
reviewed
May 15, 2024
tangzhiyi11
reviewed
May 17, 2024
tangzhiyi11
reviewed
May 22, 2024
tangzhiyi11
approved these changes
May 23, 2024
jinminxi104
approved these changes
May 29, 2024
caikun-pjlab
pushed a commit
to DeepLink-org/deeplink.framework.dev
that referenced
this pull request
Jun 11, 2024
…rg#791) * Refine code structure of dynamic shape handling. * Adjust symint_to_args relationship code logic. * Remove redundant code. * Enable 70B get_qkv stage dynamic shape. * Fix complex size append. * Change load_and_run in/out shape assignment. * Refine variable replacement in in/out shape structure. * Fix merge bugs. * Change one comment and variable name. * Fix an array assignment change. * Code refinement including: 1.Remove redundant Cast operator. 2.Change logic of Expand shape proxy. 3.Merge output stride executing path. * Get clear idea for expand Cast situation. * Apply some idea from Gpt AI. * Revert "Apply some idea from Gpt AI." This reverts commit 9025019. * Remove dead use, replace const proxy. * Support 7B dynamic shape version. * Pass 1st dynamic graph model. * Pass both two graph model for 7B dynamic shape version. * Fix ci case incre_flash_attention. * Change split execution path for both shape mode. * Add execution path for copy_with_offset. * Merge copy_with_offset shape path mode. * Add const proxy for int split_size. * Move some common functions into util. * Add path for flash_attention to pass head, kvhead and dim in. * Cancel path split for slice start proxy form. * Add sequenceAt & triu test unit case. * Return several code logic back to original design, and fix flash_incre_attention unit test. * Modify the logic of split implementation. * Add split dynamic case. * Remove identity additional logic, wrap convert into promote_dtype. * Pass ge unit test. * Modify logic of lt dtype, and prompt_attention fp16 conversion. * Add promote_dtype priority logic. * Fix promote_dtype bug. * Cast back fa to float32 if dtype not consistent. * Change to return q_dtype tensor. * Improve promote_dtype logic. * Add const proxy logic for promote_dtype. * Fix flash_attention declaration. * Remove Symint & Proxy from 7B static path. * Change const judge method. --------- Co-authored-by: chenchiyu <chenchiyu@pjlab.org.cn>
Wrench-Git
pushed a commit
to DeepLink-org/deeplink.framework.dev
that referenced
this pull request
Jul 16, 2024
…rg#791) * Refine code structure of dynamic shape handling. * Adjust symint_to_args relationship code logic. * Remove redundant code. * Enable 70B get_qkv stage dynamic shape. * Fix complex size append. * Change load_and_run in/out shape assignment. * Refine variable replacement in in/out shape structure. * Fix merge bugs. * Change one comment and variable name. * Fix an array assignment change. * Code refinement including: 1.Remove redundant Cast operator. 2.Change logic of Expand shape proxy. 3.Merge output stride executing path. * Get clear idea for expand Cast situation. * Apply some idea from Gpt AI. * Revert "Apply some idea from Gpt AI." This reverts commit 9025019. * Remove dead use, replace const proxy. * Support 7B dynamic shape version. * Pass 1st dynamic graph model. * Pass both two graph model for 7B dynamic shape version. * Fix ci case incre_flash_attention. * Change split execution path for both shape mode. * Add execution path for copy_with_offset. * Merge copy_with_offset shape path mode. * Add const proxy for int split_size. * Move some common functions into util. * Add path for flash_attention to pass head, kvhead and dim in. * Cancel path split for slice start proxy form. * Add sequenceAt & triu test unit case. * Return several code logic back to original design, and fix flash_incre_attention unit test. * Modify the logic of split implementation. * Add split dynamic case. * Remove identity additional logic, wrap convert into promote_dtype. * Pass ge unit test. * Modify logic of lt dtype, and prompt_attention fp16 conversion. * Add promote_dtype priority logic. * Fix promote_dtype bug. * Cast back fa to float32 if dtype not consistent. * Change to return q_dtype tensor. * Improve promote_dtype logic. * Add const proxy logic for promote_dtype. * Fix flash_attention declaration. * Remove Symint & Proxy from 7B static path. * Change const judge method. --------- Co-authored-by: chenchiyu <chenchiyu@pjlab.org.cn>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Optimize dynamic shape handling: