Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add content-md5 header for artifacts #14140

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

akloss-cibo
Copy link

This change will allow objects to be written to buckets with S3 Object Lock enabled.

I have manually fixed bits of the jsonschema and swagger, but I've been unable to build the Java client locally. It's not entirely clear to me that this PR needs to include those changes, but it seems like it does. If a kind soul can give me a tip on getting it to build, I'm happy to try again. (See this comment in the issue.)

Fixes #11182

Motivation

As described in 11182, being able to use an object-lock enabled S3 bucket is desirable for organizations wanted to protect assets from destruction.

Modifications

Added sendContentMD5 setting to S3 artifact configuration, and use that setting to enable sending the MD5 via argoproj/pkg.

Verification

I have built images using this code and enabled this setting and objects appear correctly in S3.

Here's the relevant part of the workflow-controller-configmap:

archiveLogs: true
s3:
  endpoint: s3.amazonaws.com
  bucket: <redacted>
  region: us-west-2
  keyFormat: "workflow\
    /{{workflow.name}}\
    /{{pod.name}}\
    /{{workflow.creationTimestamp.Y}}\
    /{{workflow.creationTimestamp.m}}\
    /{{workflow.creationTimestamp.d}}"
  accessKeySecret:
    name: argo-artifact-access
    key: access-key
  secretKeySecret:
    name: argo-artifact-access
    key: secret-key
  useSDKCreds: false
  sendContentMd5: true

Here's an example artifact:

% aws s3api head-object --bucket com-<redacted> --key workflow/<redacted>-c68c7dc83a/<redacted>-2013690313/2025/01/31/main.log
{
    "AcceptRanges": "bytes",
    "Expiration": "expiry-date=\"Mon, 03 Mar 2025 00:00:00 GMT\", rule-id=\"delete-workflow/\"",
    "LastModified": "2025-01-31T15:52:52+00:00",
    "ContentLength": 756,
    "ETag": "\"740a82039415c5a950e6458f0dc0ad9d\"",
    "VersionId": "HSVfA71cjnXTsaQNw_etCqGSr3yAx4su",
    "ContentType": "text/plain; charset=utf-8",
    "ServerSideEncryption": "aws:kms",
    "Metadata": {},
    "SSEKMSKeyId": "arn:aws:kms:us-west-2:<redacted>:key/<redacted>",
    "ReplicationStatus": "COMPLETED",
    "ObjectLockMode": "COMPLIANCE",
    "ObjectLockRetainUntilDate": "2025-03-10T15:52:51.505000+00:00"
}
%

This change will allow objects to be written to buckets with S3 Object
Lock enabled.

fixes argoproj#11182

Signed-off-by: Alec Kloss <akloss@cibotechnologies.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Artifacts cannot be stored in object-lock enabled S3 buckets
1 participant