S3 — The Standard for Object Storage
S3 — The Standard for Object Storage
S3 (Simple Storage Service) was the first AWS service to reach GA, in 2006, and it popularized the object storage category. Neither file system nor block storage, the model of "an infinitely large bucket accessed by key" became the standard shape for cloud storage that followed.
1. About S3
| When | Event |
|---|---|
| 2006-03 | S3 GA — the first AWS GA service. |
| 2010 | Versioning. |
| 2011 | Server-Side Encryption. |
| 2012 | Glacier (storage class). |
| 2018 | Intelligent-Tiering · Block Public Access. |
| 2020 | Strong consistency (read-after-write). |
S3 units:
- Bucket — The container of objects. The name must be globally unique. Region-scoped.
- Object — Key + data + metadata. Up to 5 TB.
- Key — The identifier of an object. A string with slashes such as
images/2025/cover.png, but it is not a directory (prefix matching).
2. Strong consistency
For a long time S3 used an eventually consistent model for some operations, but since December 2020 all PUT/GET/DELETE operations are strongly consistent. An object you just wrote can be read immediately.
3. Storage classes
| Class | Character |
|---|---|
| Standard | Frequent access. Default. |
| Intelligent-Tiering | Auto-tier movement based on access pattern. |
| Standard-IA | Infrequent access, immediate retrieval. Cheaper + retrieval cost. |
| One Zone-IA | Single AZ. Cheaper, lower availability. |
| Glacier Instant Retrieval | Archive with instant retrieval. |
| Glacier Flexible Retrieval | Restore in minutes to hours. |
| Glacier Deep Archive | Restore in 12 hours, the cheapest. |
Changing the class within a region is a metadata change; the data itself is not moved.
4. Lifecycle
Rules that transition class or delete based on object age, prefix, or tag. Common flows: send logs and temporary files to Glacier after a fixed period, or delete after 30 days.
{
"Rules": [{
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Transitions": [{ "Days": 30, "StorageClass": "GLACIER" }],
"Expiration": { "Days": 365 }
}]
}
5. Versioning · Replication
When versioning is enabled, every PUT to the same key creates a new version. DELETE only places a "delete marker"; the data itself remains. Useful for intentional restore, audit, and ransomware protection. Cleaning up old versions automatically with Lifecycle is common.
Replication:
- CRR (Cross-Region Replication) — Asynchronous replication to a bucket in another region. DR · placing data closer to regions.
- SRR (Same-Region Replication) — Same region, different bucket. Separation, log aggregation.
Versioning must be enabled on both buckets.
6. Pre-signed URL
A model that embeds a short-lived signature in a URL to grant temporary access to an object. Both upload and download are possible. The server only issues the URL; the client talks directly to S3.
url = s3.generate_presigned_url(
'put_object',
Params={'Bucket': 'my', 'Key': 'uploads/a.png', 'ContentType': 'image/png'},
ExpiresIn=600,
)
Often used to keep large file uploads out of the server's path.
7. Multipart Upload
Multipart uploads are recommended for files over 100 MB. Split into 5 MB ~ 5 GB parts, upload in parallel, and combine at the end. Failed parts can be retried selectively.
8. Static website hosting
Place index.html and error.html in a bucket and enable static hosting to expose it via HTTP. For HTTPS, put CloudFront in front (S3 + CloudFront is effectively the standard combination).
9. Compatible services
| Service | Notes |
|---|---|
| MinIO | Self-hostable S3-compatible server. On-prem · k8s. |
| Cloudflare R2 (2022) | Pricing emphasizes zero egress. S3-compatible API. |
| Backblaze B2 | Low cost + S3-compatible API. |
| Wasabi | Flat pricing · free egress model. |
| Google Cloud Storage | Similar model to S3, different API. |
| Azure Blob Storage | Azure counterpart. |
The S3 API has effectively become the lingua franca of object storage, and pointing the same SDK at different backends (changing endpoint_url) is common.
10. CLI · security defaults
aws s3 cp ./build s3://my-static/ --recursive
aws s3 sync ./build s3://my-static/ --delete
Security defaults:
- Enable Block Public Access at both account and bucket level. Open only where public access is explicitly needed.
- Use bucket policy for access control. Reduce ACL usage as much as possible (since 2023, ACLs are disabled by default for new buckets).
- KMS encryption (
aws:kms) or SSE-S3 (AES256). Since 2023, all objects are encrypted by default with SSE-S3. - Object Lock — WORM (Write Once Read Many). For compliance.
- VPC Endpoint (Gateway) — Reach S3 from inside the VPC without traversing the internet.
11. Cost structure
- Storage — GB · month. Big variation by class.
- Requests — PUT · GET · LIST priced separately.
- Egress — Free to the same AWS service in the same region; per GB to internet or other regions.
Request cost can balloon unexpectedly. Many tiny objects can make LIST · GET cost more than the storage cost.
12. Common pitfalls
Public exposure incidents — Old buckets sometimes still have ACL public-read. Audit with Block Public Access + Access Analyzer.
Glacier restore time — Deep Archive takes 12+ hours. Putting it where instant access is expected leads to incidents.
Lifecycle cost accumulation — Failing to clean up old versions makes a versioning-enabled bucket grow without bound.
Pre-signed URL expiry · signature headers — Short expiry or extra client headers break the signature.
CORS missing — Direct browser PUT to S3 requires bucket CORS configuration.
Request cost spikes — Frequent LIST calls can be surprisingly expensive. Replace with cache or DB indexes.
Closing thoughts
S3 is effectively the standard for object storage. For small operations, Cloudflare R2's free-egress model or self-hosted MinIO can be cost-friendlier. Security starts with Block Public Access — public exposure of old buckets is the most common incident pattern.
Next
- rds
- cloudfront
S3 user guide · S3 pricing · Object Lock · MinIO · Cloudflare R2 · Backblaze B2 · s3cmd · rclone for reference.