Skip to content

Commit

Permalink
[AWSs3 Input] Handle unexpectedEOF errors (#42420)
Browse files Browse the repository at this point in the history
* return EOF error again as errS3DownloadFailed

Signed-off-by: Andreas Gkizas <andreas.gkizas@elastic.co>

---------

Signed-off-by: Andreas Gkizas <andreas.gkizas@elastic.co>
Co-authored-by: Kavindu Dodanduwa <Kavindu-Dodan@users.noreply.github.com>
  • Loading branch information
gizas and Kavindu-Dodan authored Mar 6, 2025
1 parent b8d6142 commit e6781f3
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ otherwise no tag is added. {issue}42208[42208] {pull}42403[42403]
- Fix entityanalytics activedirectory provider full sync use before initialization bug. {pull}42682[42682]
- In the `http_endpoint` input, fix the check for a missing HMAC HTTP header. {pull}42756[42756]
- Prevent computer details being returned for user queries by Activedirectory Entity Analytics provider. {issue}11818[11818] {pull}42796[42796]
- Handle unexpectedEOF error in aws-s3 input and enforce retrying using download failed error {pull}42420[42756]

*Heartbeat*

Expand Down
7 changes: 6 additions & 1 deletion x-pack/filebeat/input/awss3/s3_objects.go
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,12 @@ func (p *s3ObjectProcessor) addGzipDecoderIfNeeded(body io.Reader) (io.Reader, e
}

func (p *s3ObjectProcessor) readJSON(r io.Reader) error {
dec := json.NewDecoder(r)
var buf bytes.Buffer
if _, err := io.Copy(&buf, r); err != nil {
// If an error occurs during the download, handle it here
return fmt.Errorf("%w: %w", errS3DownloadFailed, err)
}
dec := json.NewDecoder(&buf)
dec.UseNumber()

for dec.More() && p.ctx.Err() == nil {
Expand Down

0 comments on commit e6781f3

Please sign in to comment.