Skip to content

Commit

Permalink
Revert "fix: attention mask should be 1 or 0" (#8)
Browse files Browse the repository at this point in the history
This reverts commit dc15e65.

The fix was actually wrong, and it needs to be reverted.
  • Loading branch information
tengomucho authored Mar 26, 2024
1 parent edf1b9e commit fecab96
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,7 @@ def decode(self, batches: List[CachedBatch]) -> Tuple[List[Generation], CachedBa
dtype=torch.int64,
device=self.model.device,
)
attention_mask.index_put_([torch.tensor([1])], slot.attention_mask)
attention_mask.index_put_([torch.tensor([i])], slot.attention_mask)
position_ids.index_put_([torch.tensor([i])], torch.tensor(slot.cur_position))
if input_ids is None:
raise ValueError("Unable to decode tokens for non-prefilled batches (probably due to a previous failure)")
Expand Down

0 comments on commit fecab96

Please sign in to comment.