S3: storage for everything
Buckets and objects, storage classes, bucket policies, versioning and lifecycle rules that keep the bill small.
By the end of this lesson you'll be able to
0 of 5 completeBuckets and objects
Amazon S3 is object storage. You do not create a disk and mount it on a server. You put an object into a bucket and address it by a key.
A bucket has a globally unique name and lives in one AWS Region. An object has a key, a value and metadata. A key such as logs/2026/10/access.log looks like a path, but S3 does not have real folders. The slashes are prefixes that the console groups for you.
The first checks I run after creating a bucket are simple:
$ aws s3api get-bucket-location --bucket $BUCKET --query LocationConstraint --output text ap-south-1 $ aws s3api list-objects-v2 --bucket $BUCKET --query 'Contents[].{Key:Key,Size:Size,StorageClass:StorageClass}' --output table ----------------------------------------------- | ListObjectsV2 | +--------------------+-------+----------------+ | Key | Size | StorageClass | +--------------------+-------+----------------+ | docs/readme.txt | 12 | STANDARD | | logs/demo.log | 12 | STANDARD | | tmp/to-expire.txt | 12 | STANDARD | +--------------------+-------+----------------+
aws s3api get-bucket-location --bucket $BUCKET --query LocationConstraint --output textawsthe AWS command line--querypick fields from the JSON response--outputoutput format
The object key is the important part. Prefixes such as docs/, logs/ and tmp/ make a large bucket easier to browse and make lifecycle rules easier to target.
Storage classes
Storage class is a cost and access decision. The data is still an S3 object, but AWS uses a different price and retrieval model.
| Class | Use it when | Watch out for |
|---|---|---|
| Standard | The application reads the object often | The normal storage price applies |
| Standard-IA | The object is important but rarely read | Retrieval charges and a minimum storage period |
| Intelligent-Tiering | Access patterns are not predictable | Small monitoring charges apply |
| Glacier classes | The object is an archive | Retrieval can take time and costs extra |
Do not choose Glacier because the name sounds cheap. Start with how often the application reads the object, how quickly it must come back and how long it must stay there. The right answer is often Standard for active artifacts and a lifecycle transition for older ones.
Lifecycle rules
An S3 lifecycle rule is a scheduled policy for objects. It can transition objects to a colder class or expire them after a period. Filter by prefix or object tags so a cleanup rule does not remove production data by accident.
This rule expires only objects under tmp/ after one day:
$ aws s3api put-bucket-lifecycle-configuration \ --bucket $BUCKET \ --lifecycle-configuration '{"Rules":[{"ID":"ExpireTemporaryObjects","Status":"Enabled","Filter":{"Prefix":"tmp/"},"Expiration":{"Days":1}}]}' $ aws s3api get-bucket-lifecycle-configuration \ --bucket $BUCKET \ --query 'Rules[].{ID:ID,Status:Status,Prefix:Filter.Prefix,Days:Expiration.Days}' --output table --------------------------------------------------------- | GetBucketLifecycleConfiguration | +------+--------------------------+---------+-----------+ | Days | ID | Prefix | Status | +------+--------------------------+---------+-----------+ | 1 | ExpireTemporaryObjects | tmp/ | Enabled | +------+--------------------------+---------+-----------+
aws s3api put-bucket-lifecycle-configuration \awsthe AWS command line
Lifecycle jobs do not run immediately when you save the rule. Treat the rule as automation that will run later, and verify the scope before you enable it.
Versioning
Versioning protects against accidental overwrites and deletes. Each write gets a version ID. A normal delete adds a delete marker instead of immediately destroying the previous version.
$ aws s3api put-bucket-versioning \ --bucket $BUCKET \ --versioning-configuration Status=Enabled $ aws s3api get-bucket-versioning \ --bucket $BUCKET { "Status": "Enabled" }
aws s3api put-bucket-versioning \awsthe AWS command line
Versioning is not a backup strategy by itself. Old versions still consume storage, and someone with permission to delete versions can remove the recovery point. Pair it with lifecycle rules, restricted delete permissions and, for important data, a separate backup or replication design.
Access control
Keep a new bucket private. S3 access can come from IAM identity policies, bucket policies, access points or presigned URLs. Start with no public access and grant one narrow use case at a time.
Block Public Access is a set of guard rails. It blocks common ways of making a bucket or its objects public. It does not replace IAM; it stops public exposure while your identity and resource policies decide which authenticated callers are allowed.
A presigned URL is a temporary link signed by an AWS identity. It is useful when a private application needs to let someone upload or download one object without giving that person AWS credentials. Keep the expiry short and scope the URL to one object and one operation.
For a public static site, do not simply turn off Block Public Access. Put CloudFront in front of the bucket and use Origin Access Control so visitors reach the distribution, not the bucket directly. Static website hosting is a simple S3 feature, but its public endpoint does not provide the same private-origin design.
Encryption
S3 encrypts new objects by default. SSE-S3 uses keys managed by Amazon S3. SSE-KMS uses a KMS key that you control through a key policy and grants.
- Use SSE-S3 when you need straightforward server-side encryption with no separate key administration.
- Use SSE-KMS when you need key rotation controls, audit events or separate permissions for using the key.
Encryption at rest does not decide who can read the object. You still need IAM and bucket policy controls. It also does not replace TLS while the object is moving between the client and S3.
Static website hosting
S3 can return HTML, CSS, JavaScript and images as a static website. It is useful for a simple public site, but it has limits: the S3 website endpoint is HTTP-only and the bucket must be publicly readable.
For a production site, use a private bucket behind CloudFront. CloudFront supplies HTTPS, caching and edge delivery. Origin Access Control keeps the bucket private and makes the distribution the only intended reader.
Replication
S3 Replication copies objects to another bucket, usually in another Region. It is useful for disaster recovery, data residency designs and keeping a second copy near another group of users.
Replication needs versioning on both buckets, an IAM role that S3 can assume and a rule that selects which objects to copy. It is not free: the destination stores another copy and replication creates requests and transfer charges. Design the recovery requirement first, then estimate the cost before enabling it.
Presigned URLs are temporary delegation
S3 can sign one request so an untrusted client uploads or downloads without receiving AWS credentials. The URL is a bearer token, so keep its expiry short and bind it to one key. Use CloudFront signed URLs for CDN delivery; use an S3 presigned URL for one direct object transfer.
$ URL=$(aws s3 presign "s3://$BUCKET/uploads/photo.jpg" --expires-in 600 --region ap-south-1) $ curl --fail --silent --show-error "$URL" -o photo.jpg
curl --fail --silent --show-error "$URL" -o photo.jpgcurlmake an HTTP request-owrite the body to this file
Event notifications start a workflow
An S3 notification can publish object-created events to SNS, SQS or Lambda. Filter by prefix and suffix so a consumer does not process its own output. SQS buffers work while a worker is unavailable; SNS fans out to several subscribers.
$ aws s3api get-bucket-notification-configuration --bucket "$BUCKET" --region ap-south-1 { "QueueConfigurations": [] }
aws s3api get-bucket-notification-configuration --bucket "$BUCKET" --region ap-south-1awsthe AWS command line--regionwhich Region
Configure the queue ARN returned by aws sqs get-queue-attributes; never paste an invented ARN into a lesson.
Object Lock and multipart upload solve different problems
Object Lock protects versioned objects from deletion during retention. Multipart upload splits a large object into retryable parts; it is for throughput, not immutability. Add a lifecycle rule to abort incomplete uploads.
A private, recoverable artifact bucket
Create a private S3 bucket in ap-south-1, upload a small artifact twice, verify encryption and versioning, add a lifecycle rule for tmp/ and then remove every object version before deleting the bucket.
A private bucket with two versions of one object, AES256 encryption, an enabled tmp/ lifecycle rule and a verified cleanupReveal one safe solution
$ export AWS_REGION=ap-south-1 $ export BUCKET="devops-academy-$(aws sts get-caller-identity --query Account --output text)-$RANDOM" $ printf '%s\n' 'artifact version one' > artifact.txt $ aws s3api create-bucket --bucket "$BUCKET" --region "$AWS_REGION" \ --create-bucket-configuration LocationConstraint="$AWS_REGION" $ aws s3api put-public-access-block --bucket "$BUCKET" \ --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true $ aws s3api put-bucket-encryption --bucket "$BUCKET" \ --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}' $ aws s3api put-bucket-versioning --bucket "$BUCKET" --versioning-configuration Status=Enabled $ aws s3api put-object --bucket "$BUCKET" --key artifacts/build.txt --body artifact.txt $ printf '%s\n' 'artifact version two' > artifact.txt $ aws s3api put-object --bucket "$BUCKET" --key artifacts/build.txt --body artifact.txt $ aws s3api put-bucket-lifecycle-configuration --bucket "$BUCKET" \ --lifecycle-configuration '{"Rules":[{"ID":"ExpireTemporaryObjects","Status":"Enabled","Filter":{"Prefix":"tmp/"},"Expiration":{"Days":1}}]}' $ aws s3api get-bucket-encryption --bucket "$BUCKET" \ --query 'ServerSideEncryptionConfiguration.Rules[].ApplyServerSideEncryptionByDefault.SSEAlgorithm' --output text AES256 $ aws s3api list-object-versions --bucket "$BUCKET" \ --query 'Versions[].{Key:Key,IsLatest:IsLatest,Size:Size,VersionId:VersionId}' --output table --------------------------------------------- | ListObjectVersions | +-----------+-----------------------+-------+ | IsLatest | Key | Size | VersionId | +----------+-----------------------+-------+------------------------------------+ | True | artifacts/build.txt | 21 | L0d2SEzwFCnMQV0jdfbMJe31cqyvZ9xQ | | False | artifacts/build.txt | 21 | BYoXEGMp2.ZBa4hjl3R05uMDW.8sI9b0 | +----------+-----------------------+-------+------------------------------------+ $ aws s3api list-object-versions --bucket "$BUCKET" \ --query '{Objects: Versions[].{Key:Key,VersionId:VersionId}}' > versions.json $ aws s3api delete-objects --bucket "$BUCKET" --delete file://versions.json $ aws s3api list-object-versions --bucket "$BUCKET" \ --query '{Objects: DeleteMarkers[].{Key:Key,VersionId:VersionId}}' > delete-markers.json $ aws s3api delete-objects --bucket "$BUCKET" --delete file://delete-markers.json $ aws s3api delete-bucket --bucket "$BUCKET" --region "$AWS_REGION" $ aws s3api head-bucket --bucket "$BUCKET" --region "$AWS_REGION" An error occurred (404) when calling the HeadBucket operation: Not Found
printf '%s\n' 'artifact version one' > artifact.txt>write output to a file, replacing it
aws s3api create-bucket --bucket "$BUCKET" --region "$AWS_REGION" \awsthe AWS command line--regionwhich Region
The two delete-objects calls remove versions and delete markers without requiring you to copy IDs by hand. The second file can contain an empty Objects list when no delete markers exist. S3 does not let you delete a non-empty versioned bucket.
Try it yourself
0 of 5 steps doneRun each task in your own account or terminal, then tick it off.
Quick check
A team stores build artifacts in S3 and reads them for 30 days. Which starting storage class fits best?
Official reading for this lesson
Or get every quiz answer right and it completes itself.