XAmzContentSHA256Mismatch: The provided 'x-amz-content-sha256' header does not match what was computed

[sumitjhajharia]

Checkboxes for prior research

• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.
• I've gone through Developer Guide and API reference

Describe the bug

Hi,

We are getting XAmzContentSHA256Mismatch issue randomly when try to upload ~400 files to S3.
From primary investigation its looks like it fails when there is race condition OR multi thread condition.

Checked similar mentioned issues - aws/aws-sdk-ruby#1594 , #6115 , but it seems it's still not fixed.

Regression Issue

• Select this option if this issue appears to be a regression.

SDK version number

"@aws-sdk/client-s3": "3.744.0", "@aws-sdk/lib-storage": "3.744.0"

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

20.14.0

Reproduction Steps

const s3Client = new S3Client({
  credentials: { accessKeyId, secretAccessKey },
  region,
  signatureVersion: 'v4',
  maxRetries: 5,
});

const uploadFileToS3 = ({client, key, file, uploadParams}) => {
    const body = fs.createReadStream(file);
  const command = new Upload({
    client,
    params: {
      Key: key,
      Body: body,
      Bucket: bucket,
      BucketName: bucketName,
    },
    leavePartsOnError: false, // default - abort the multipart upload if an error occurs, automatically call AbortMultipartUpload
    queueSize: 4, // default -  queue manager to upload parts in parallel
  });
  const promise = command
    .done()
    .then((response) => response)
    .catch((error) => {
      throw new Error(error);
    });
  return { promise };
};

and calling uploadFileToS3 in Promise.all([<uploadFileToS3(...), uploadFileToS3(...), uploadFileToS3(...)>]);

Observed Behavior

` XAmzContentSHA256Mismatch: The provided 'x-amz-content-sha256' header does not match what was computed.

#43 281.4     at throwDefaultError (/app/node_modules/@smithy/smithy-client/dist-cjs/index.js:867:20)
#43 281.4     at /app/node_modules/@smithy/smithy-client/dist-cjs/index.js:876:5
#43 281.4     at de_CommandError (/app/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4970:14)
#43 281.4     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
#43 281.4     at async /app/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
#43 281.4     at async /app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:483:18
#43 281.4     at async /app/node_modules/@smithy/middleware-retry/dist-cjs/index.js:321:38
#43 281.4     at async /app/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/index.js:315:18
#43 281.4     at async /app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:109:22
#43 281.4     at async /app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:136:14 {
#43 281.4   '$fault': 'client',
#43 281.4   '$metadata': {
#43 281.4     httpStatusCode: 400,
#43 281.4     requestId: '5PD542KYBKS97QDX',
#43 281.4     extendedRequestId: '+o3EAtt2QMJCuG9zqoqlbmWAA1bZ9gn8vRuQBA8J20XLVQoWzwOBsDL31Cp9fK5DNKfuq56/GvUSoN0ro6qIPg==',
#43 281.4     cfId: undefined,
#43 281.4     attempts: 1,
#43 281.4     totalRetryDelay: 0
#43 281.4   },

#43 281.4   Code: 'XAmzContentSHA256Mismatch',
#43 281.4   ClientComputedContentSHA256: 'f5df4223ba30e7d19fd37ee4c61628b9712f91998b320473abc51d64bc04da83',
#43 281.4   S3ComputedContentSHA256: '626a70019720d2d30bd88df3fccd18fa723a7dfa69412bffe62c0542f5abd266',
#43 281.4   RequestId: '5PD542KYBKS97QDX',
#43 281.4   HostId: '+o3EAtt2QMJCuG9zqoqlbmWAA1bZ9gn8vRuQBA8J20XLVQoWzwOBsDL31Cp9fK5DNKfuq56/GvUSoN0ro6qIPg=='
#43 281.4 }
#43 281.4 Failed to compile.
#43 281.4 
#43 281.4 XAmzContentSHA256Mismatch: The provided 'x-amz-content-sha256' header does not match what was `computed.``

Expected Behavior

aws sdk should create and match authorization headers if not provided manually.

Possible Solution

No response

Additional Information/Context

No response

[zshzbh]

Hey @sumitjhajharia ,

I can't reproduce this issue using the version "@aws-sdk/client-s3": "3.744.0", "@aws-sdk/lib-storage": "3.744.0".

As you mentioned that you are uploading ~400 files, I assume that is related to the amount of the files. We found that reducing the batch size to 5 seems to resolve the issue. While I'm still waiting for service team to get an update, could you please lower the batch size as a workaround?

[zshzbh]

Posting my code -

import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import fs from "fs"; 

const s3Client = new S3Client({
  region: "us-east-1",
  signatureVersion: "v4",
  maxRetries: 5,
});

async function uploadFileToS3({ client, key, file }) {
  try {
    const fileContent = await fs.readFile(file);
    const command = new Upload({
      client,
      params: {
        Bucket: "new-xxx-xxx-ma",
        Key: key,
        Body: fileContent,
      },
      leavePartsOnError: false,
      queueSize: 4,
    });

    const response = await command.done();
    return response;
  } catch (error) {
    throw new Error(`Failed to upload file: ${error.message}`);
  }
}


async function main() {
  try {
    const result = await uploadFileToS3({
      client: s3Client,
      key: "6893.txt",
      file: "test.txt"
    });
    console.log("Upload successful:", result);
  } catch (error) {
    console.error("Upload failed:", error);
  }
}

main();