Code

Support Low Rank Adaptation (LoRA).#745

Code

Merged

denera merged 8 commits intoNVIDIA:mainNVIDIA/TransformerEngine:mainfrom mingxu1067:mingh/add_loramingxu1067/TransformerEngine:mingh/add_loraCopy head branch name to clipboardApr 16, 2024

+412-2

Commits

Commits on Apr 10, 2024

• Adding LoRA and related docs.

  Show description for e3a8a04
  mingxu1067

  committed
  e3a8a04

  Copy full SHA for e3a8a04

• Adding tests to LoRA.

  Show description for 161b424
  mingxu1067

  committed
  161b424

  Copy full SHA for 161b424

• Fix the missing license issue.

  Show description for 042cb00
  mingxu1067

  committed
  042cb00

  Copy full SHA for 042cb00

• Adding alpha to LoRA

  Show description for a5ad42d
  mingxu1067

  committed
  a5ad42d

  Copy full SHA for a5ad42d

• Adding scope controller to LoRA.

  Show description for 152f341
  mingxu1067

  committed
  152f341

  Copy full SHA for 152f341

• Fixing the wrong shape of axes to LoRA weights.

  Show description for 8a00711
  mingxu1067

  committed
  8a00711

  Copy full SHA for 8a00711

• Fix the wrong axes shape of lora_a in MLP.

  Show description for 73b7254
  mingxu1067

  committed
  73b7254

  Copy full SHA for 73b7254

• Adding None handle to LoRA.

  Show description for c7f570b
  mingxu1067

  committed
  c7f570b

  Copy full SHA for c7f570b