Code

Several Inference Endpoint fixes#66

Code

Merged

tengomucho merged 12 commits intomainhuggingface/optimum-tpu:mainfrom ie-fixeshuggingface/optimum-tpu:ie-fixesCopy head branch name to clipboardJul 3, 2024

+156-120

Commits

Commits on Jun 27, 2024

• fix(tgi): remove all the variables from entrypoint.sh

  mfuntowicz

  committed
  6148c28

  Copy full SHA for 6148c28

Commits on Jul 1, 2024

• fix(tgi): correct version

  tengomucho

  committed
  54ed49c

  Copy full SHA for 54ed49c

• fix(tgi): pin numpy version <2.0

  tengomucho

  committed
  f4a31f6

  Copy full SHA for f4a31f6

• feat(tgi): entrypoint adds GKE specific command

  tengomucho

  committed
  9f5fa56

  Copy full SHA for 9f5fa56

• fix(generator): correct CachedBatch serialization when it's None

  Show description for bbbfda9
  tengomucho

  committed
  bbbfda9

  Copy full SHA for bbbfda9

Commits on Jul 2, 2024

• feat(generator): prefill input preparation is done on CPU

  Show description for a5d50bb
  tengomucho

  committed
  a5d50bb

  Copy full SHA for a5d50bb

• feat(generator): decode input preparation is done on CPU

  tengomucho

  committed
  9c44b86

  Copy full SHA for 9c44b86

• feat(generator): support TGI truncate parameter in Request

  tengomucho

  committed
  ae0991b

  Copy full SHA for ae0991b

• fix(generator): warmup clears after prefill

  Show description for 9b9bd3a
  tengomucho

  committed
  9b9bd3a

  Copy full SHA for 9b9bd3a

• fix(tgi): correct clear implementation

  Show description for f5041f1
  tengomucho

  committed
  f5041f1

  Copy full SHA for f5041f1

• feat(ci): release TGI images only when release is published

  tengomucho

  committed
  ed1a0dd

  Copy full SHA for ed1a0dd

Commits on Jul 3, 2024

• chore(generator): turn log info -> debug on clear

  tengomucho

  committed
  c4d46fe

  Copy full SHA for c4d46fe