S3 Presigned URLs SSE-KMS: Common Pitfalls and How to Avoid Them

This article was originally written in Japanese and published on Qiita. It has been translated with the help of AI. Original article: https://qiita.com/sassssan68/items/9f37fac0f4e2210deb0c You switched S3 encryption from SSE-S3 to SSE-KMS, and suddenly your presigned URLs started returning errors. Presigned URLs and SSE-KMS are both common features when used on their own, but combining them comes with some tricky permission requirements. This article covers: Why presigned URLs fail when you switch to SSE-KMS Easy-to-miss permission pitfalls (IAM policies and KMS key policies) Different gotchas for downloads (GET) and uploads (PUT) A presigned URL grants temporary access to an S3 object. https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html Common use cases: Letting a user download a file temporarily Letting a user upload a file (e.g., form submission) # Generate a download presigned URL (valid for 3600 seconds) aws s3 presign s3://your-bucket/your-file.pdf --expires-in 3600 The important thing to keep in mind is that a presigned URL borrows the permissions of the user who generated it. SSE-KMS is one of S3's server-side encryption methods. It encrypts objects using a customer managed key in AWS KMS. https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html Comparing S3 encryption methods: Method Key Manager Compatibility with Presigned URLs SSE-S3 Fully managed by AWS Works without issues SSE-KMS (AWS managed key) Managed by AWS (aws/s3) Has constraints SSE-KMS (customer managed key) Managed by user Has constraints With SSE-S3, S3 manages the keys internally, so no extra permission setup is needed. separate permissions to access the KMS key are required. This is the part you need to watch out for. Worth noting: this applies even when you use the AWS managed key (aws/s3) for SSE-KMS. as long as it's SSE-KMS, KMS permissions are required regardless of who manages the key. When you access an object via a presigned URL, here's what happens internally: With SSE-S3 Request comes in via the presigned URL S3 decrypts the data with its internal key Data is returned With SSE-KMS Request comes in via the presigned URL S3 asks KMS to decrypt the data key ← Without KMS key permissions, this fails KMS decrypts and returns the data key S3 decrypts the data with the data key Data is returned With SSE-KMS, you need permissions for both S3 and the KMS key. The IAM user or role generating the presigned URL needs to have KMS-related permissions added to its policy. { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowS3GetObject", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my-bucket/*" }, { "Sid": "AllowKMSDecrypt", "Effect": "Allow", "Action": "kms:Decrypt", "Resource": "arn:aws:kms:ap-northeast-1:123456789012:key/your-key-id" } ] } s3:GetObject alone isn't enough — without kms:Decrypt, you'll get an error. { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowS3PutObject", "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::my-bucket/*" }, { "Sid": "AllowKMSForUpload", "Effect": "Allow", "Action": "kms:GenerateDataKey", "Resource": "arn:aws:kms:ap-northeast-1:123456789012:key/your-key-id" } ] } For uploads, kms:GenerateDataKey is required. ⚠️ Note kms:Decrypt is also required. When using SSE-KMS, Signature Version 4 (SigV4) is mandatory. https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html#aws-signature-version-4-sse-kms Recent versions of Boto3 use SigV4 by default, but depending on your environment or configuration, it may fall back to SigV2. InvalidArgument error like this: <Error> <Code>InvalidArgument</Code> <Message> Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4. </Message> <ArgumentName>Authorization</ArgumentName> </Error> To make sure it works reliably, explicitly specify SigV4 when creating the client. from botocore.config import Config s3_client = boto3.client("s3", config=Config(signature_version="s3v4")) I actually got tripped up by this myself. ⚠️ Note Older versions of Boto3 / Botocore When signature_version is explicitly set in ~/.aws/config Legacy region settings When using SSE-KMS, always specify Config(signature_version="s3v4") to be safe. If you created the KMS key with default settings, the root delegation policy means IAM policies alone are sufficient. the KMS key policy must also grant permission, or you'll get an error. KMS keys have a key policy that controls "who can use this key." both the IAM policy and the key policy. { "Sid": "AllowPresignedUrlRole", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:role/PresignedUrlGeneratorRole" }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" } Note * for Resource in a KMS key policy is fine. * refers to that very key. KMS keys created with default settings include a policy with root as the Principal. root delegates the decision to the account's IAM policies, which means IAM policy alone is enough to use the KMS key. This applies when you don't use bucket default encryption and instead specify the KMS key explicitly in the request. When generating a presigned URL for upload, you need to include the SSE-KMS parameters in the signature. The AWS CLI's presign command is designed for GET (download) use cases. import boto3 from botocore.config import Config s3_client = boto3.client("s3", config=Config(signature_version="s3v4")) # For downloads (GET) download_url = s3_client.generate_presigned_url( "get_object", Params={ "Bucket": "my-bucket", "Key": "my-file.pdf", }, ExpiresIn=3600, ) # For uploads (PUT) upload_url = s3_client.generate_presigned_url( "put_object", Params={ "Bucket": "my-bucket", "Key": "upload/new-file.pdf", "ServerSideEncryption": "aws:kms", "SSEKMSKeyId": "arn:aws:kms:ap-northeast-1:123456789012:key/your-key-id", }, ExpiresIn=3600, ) For PUT presigned URLs, include ServerSideEncryption and SSEKMSKeyId in Params. ⚠️ Note curl -X PUT \ -H "x-amz-server-side-encryption: aws:kms" \ -H "x-amz-server-side-encryption-aws-kms-key-id: arn:aws:kms:ap-northeast-1:123456789012:key/your-key-id" \ --upload-file ./new-file.pdf \ "<presigned-url>" Missing headers will result in a SignatureDoesNotMatch error. Note upload_part), SSE parameters are specified at the create_multipart_upload step. ServerSideEncryption or SSEKMSKeyId. Here's a checklist for when you hit errors with presigned URLs and SSE-KMS: # Check Target 1 Does the IAM policy include kms:Decrypt? Presigned URL generator 2 Does the IAM policy include kms:GenerateDataKey (for PUT)? Presigned URL generator 3 Is Config(signature_version="s3v4") set in Boto3? Application 4 Does the KMS key policy permit the generator's principal? KMS key 5 Are SSE parameters included in the signature for PUT (when not using default encryption)? Application 6 Do the PUT request headers match the signature (when not using default encryption)? Client 7 Is the presigned URL still within its expiration window? URL 8 Is the KMS key in an enabled state (not disabled or pending deletion)? KMS key The biggest reason presigned URLs and SSE-KMS trip people up is that switching to SSE-KMS introduces a separate requirement: permissions on the KMS key itself. SSE Method Additional Permission Setup Required SSE-S3 None SSE-KMS IAM policy + KMS key policy Key takeaways: When you switch to SSE-KMS, add KMS permissions — kms:Decrypt, plus kms:GenerateDataKey for PUT In Boto3, explicitly set Config(signature_version="s3v4") — without SigV4, you'll get InvalidArgument Check the KMS key policy too — IAM policies alone may not be enough Include SSE parameters in PUT presigned URLs — header mismatches lead to SignatureDoesNotMatch You're most likely to hit these errors when migrating from SSE-S3 to SSE-KMS, so use this checklist when you do!