Access denied on AWS s3 bucket even with bucket and/or user policy

Question

I've tried pretty much every possible bucket policy. Also tried adding a policy to the user, but I get Access Denied every time I try to download an object from s3 bucket using the AWS Console.

Bucket Policy:

{
    "Version": "2012-10-17",
    "Id": "MyPolicy",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::12345678901011:user/my-username"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my-bucket/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "XX.XXX.XXX.XXX/24",
                        "XXX.XXX.XXX.XXX/24"
                    ]
                }
            }
        }
    ]
}

That doesn't work so I tried adding a policy to my-username:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "StmtXXXXXXXXXX",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

Answer 1

As strange as it sounds, it is possible to upload an object to Amazon S3 that the account owning the bucket cannot access.

When an object is uploaded to Amazon S3 (PutObject), it is possible to specify an Access Control List (ACL). Possible values are:

• private
• public-read
• public-read-write
• authenticated-read
• aws-exec-read
• bucket-owner-read
• bucket-owner-full-control

You should normally upload objects with the bucket-owner-full-control ACL. This allows the owner of the bucket access to the object and permission to control the object (eg delete it).

If this permission is not supplied, then they cannot access nor modify the object.

I know that it contradicts the way you'd think buckets should work, but it's true!

How to fix it:

• Re-upload the objects with bucket-owner-full-control ACL, or
• The original uploader can loop through the objects and do an in-place CopyObject with a new ACL. This changes the permissions without having to re-upload.

UPDATE: In November 2021, a new feature was released: Amazon S3 Object Ownership can now disable access control lists to simplify access management for data in S3. This avoids the need to specify object ownership and fixes most problems with object ownership.

Answer 2

You can solve it by using : http://docs.aws.amazon.com/cli/latest/reference/s3api/put-object-acl.html

put-object-acl : This has to be done by original uploader.

But is definitely faster than copying data again.

I had TB's of data to deal with.

aws s3api put-bucket-acl --bucket $foldername --key $i --grant-full-control uri=http://acs.amazonaws.com/groups/global/AllUsers