dipu ci

Fixed hccl bug during multi-card training on ascend: sdma memory copy failed #5067

Sign in to view logs

Triggered via pull request March 24, 2024 05:44

zhaoguochun1995

synchronize #682

DeepLink-org:zgc/dipu_support_ascend_hccl3

Status Failure

Total duration 40m 46s

Artifacts –

main.yml

on: pull_request

runs on nv step1 / Build-dipu-cuda

runs on nv step1 / Run tidy (cuda)

Build-dipu-pytorch-for-ascend-910b

runs on nv step2 / Build-dipu-cuda-latest-target

runs on nv step2 / Build-dipu-cuda-pt211

Build-dipu-camb

Build-dipu-camb-latest-target

Build-dipu-camb-pt211

runs on nv step2 / Test-dipu-cuda

runs on nv step2 / Test-one-iter-cuda

Build-dipu-camb-pt210

Build-dipu-ascend-910b

Build-dipu-ascend-latest-target-910b

Build-dicp-on-dipu-ascend

runs on nv step2 / Test-dipu-cuda-latest-target

runs on nv step2 / Test-dipu-cuda-pt211

Test-one-iter-camb

Test-dipu-camb-latest-target

Test-dipu-camb-pt211

Test-dipu-ascend-910b

Test-one-iter-ascend-910b

Test-dipu-ascend-latest-target-910b

Test-dicp-on-dipu-ascend

Annotations

3 errors

Test-one-iter-ascend-910b

Process completed with exit code 1.

Test-one-iter-ascend-910b

Process completed with exit code 1.

Test-dipu-ascend-910b

Process completed with exit code 1.