1613 lines
18 KiB
Markdown
1613 lines
18 KiB
Markdown
---
|
||
source_url: "https://d1.awsstatic.com/events/Summits/reinvent2023/STG314_Dive-deep-on-Amazon-S3.pdf"
|
||
ingested: 2026-06-30
|
||
sha256: 970f9400ef22f30b45946dcacdeea413b8061d2553aafd5d2c52e1067c2d3dda
|
||
discovered_from:
|
||
platform: discord
|
||
channel_id: "1028287639918497822"
|
||
channel_name: "chat"
|
||
message_id: "1521490006915678228"
|
||
author_id: "890908900520505354"
|
||
posted_at: "2026-06-30T12:18:05.566000000Z"
|
||
message_excerpt: "https://d1.awsstatic.com/events/Summits/reinvent2023/STG314_Dive-deep-on-Amazon-S3.pdf"
|
||
---
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
STG314
|
||
|
||
Dive deep on Amazon S3
|
||
|
||
Amy Therrien
|
||
|
||
Seth Markle
|
||
|
||
(she/her)
|
||
Director
|
||
S3 Engineering
|
||
Amazon S3
|
||
|
||
(he/him)
|
||
Senior Principal Engineer
|
||
Amazon S3
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
There is no compression algorithm
|
||
for experience
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Three epochs
|
||
|
||
Reactive
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Threat
|
||
modeling
|
||
|
||
Proactive
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Culture: Threat reviews
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
•
|
||
|
||
Actual written document
|
||
|
||
•
|
||
|
||
Interactive working meeting
|
||
|
||
•
|
||
|
||
Review with experienced team members
|
||
|
||
•
|
||
|
||
Learning opportunity for junior engineers
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
What is Amazon S3?
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
350+ microservices
|
||
All AWS Regions
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Clients
|
||
|
||
Front end
|
||
Web servers, DNS, and network
|
||
|
||
Index
|
||
Key/value mapping to storage
|
||
|
||
Storage
|
||
Data durably stored on devices
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The S3 front end is big
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The S3 front end is big
|
||
Traffic peaks at over 1PB/sec
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The threat of scale and going wide
|
||
Mechanisms for mitigation
|
||
|
||
Use multipart
|
||
uploads to
|
||
parallelize puts
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Use ranges to
|
||
parallelize
|
||
gets
|
||
|
||
Spread requests
|
||
across many IPs in
|
||
the fleet
|
||
|
||
MPU and ranges
|
||
100 MB
|
||
|
||
100 MB
|
||
|
||
X MB/s
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Break into parts and parallelize data transfer in
|
||
100 MB
|
||
20 MB
|
||
20 MB
|
||
20 MB
|
||
|
||
20 MB
|
||
20 MB
|
||
|
||
100 MB
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
5x MB/s
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Break into parts and parallelize data transfer
|
||
USING
|
||
|
||
THE
|
||
|
||
MULTIPART
|
||
|
||
UPLOAD
|
||
|
||
API
|
||
|
||
create-multipart-upload
|
||
[--acl <value>]
|
||
[--grant-full-control <value>]
|
||
[--grant-read <value>]
|
||
[--grant-read-acp <value>]
|
||
[--grant-write-acp <value>]
|
||
[--metadata <value>]
|
||
[--storage-class <value>]
|
||
[--server-side-encryption <value>]
|
||
[--ssekms-key-id <value>]
|
||
[--bucket-key-enabled]
|
||
[--tagging <value>]
|
||
[--object-lock-mode <value>]
|
||
|
||
upload-part
|
||
--part-number <value>
|
||
--upload-id <value>
|
||
--key <value>
|
||
--bucket <value>
|
||
|
||
n times
|
||
|
||
upload-part
|
||
--part-number <value>
|
||
--upload-id <value>
|
||
--key <value>
|
||
--bucket <value>
|
||
|
||
complete-multipart-upload
|
||
--checksum-algorithm <value>
|
||
--key <value>
|
||
--bucket <value>
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
--upload-id <value>
|
||
--key <value>
|
||
--bucket <value>
|
||
|
||
S3API CLI docs:
|
||
|
||
Same thing in reverse for data transfer out
|
||
100 MB
|
||
|
||
100 MB
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
X MB/s
|
||
|
||
5x MB/s
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
20 MB
|
||
20 MB
|
||
|
||
20 MB
|
||
20 MB
|
||
20 MB
|
||
|
||
Same thing in reverse for data transfer out
|
||
USE
|
||
|
||
GET-OBJECT-ATTRIBUTES
|
||
|
||
OR
|
||
|
||
HEAD-OBJECT
|
||
|
||
TO
|
||
|
||
DERIVE
|
||
|
||
RANGES
|
||
|
||
get-object-attributes
|
||
…
|
||
"Parts": [
|
||
{
|
||
"PartNumber": 1,
|
||
"Size": 17179870,
|
||
"ChecksumCRC32": "fay0Cw=="
|
||
},
|
||
{
|
||
|
||
"PartNumber": 2,
|
||
"Size": 17179870,
|
||
"ChecksumCRC32": "4qs8xw=="
|
||
},
|
||
{
|
||
|
||
Calculate offsets
|
||
or
|
||
just use part numbers
|
||
|
||
"PartNumber": 3,
|
||
"Size": 14692009,
|
||
"ChecksumCRC32": "Ajz83g=="
|
||
}
|
||
|
||
]
|
||
…
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
S3API CLI docs:
|
||
|
||
Same thing in reverse for data transfer out
|
||
USING
|
||
|
||
RANGE
|
||
|
||
GET
|
||
|
||
REQUESTS
|
||
|
||
get-object
|
||
[--range <value>]
|
||
[--part-number <value>]
|
||
|
||
n times
|
||
|
||
--key <value>
|
||
--bucket <value>
|
||
<OUTFILE>
|
||
|
||
… stitch back together and write to storage in code
|
||
Range GET RFC:
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
S3API CLI docs:
|
||
|
||
Spreading requests across the fleet
|
||
IT’S
|
||
|
||
ALWAYS
|
||
|
||
DNS
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Spreading requests across the fleet
|
||
IT’S
|
||
|
||
ALWAYS
|
||
|
||
DNS
|
||
# nslookup s3.amazonaws.com
|
||
Server: 192.0.2.3
|
||
Address: 192.0.2.3#53
|
||
|
||
PUT
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Spreading requests across the fleet
|
||
IT’S
|
||
|
||
ALWAYS
|
||
|
||
DNS
|
||
# nslookup s3.amazonaws.com
|
||
Server: 192.0.2.3
|
||
Address: 192.0.2.3#53
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Spreading requests across the fleet
|
||
IT’S
|
||
|
||
ALWAYS
|
||
|
||
DNS
|
||
# nslookup s3.amazonaws.com
|
||
Server: 192.0.2.3
|
||
Address: 192.0.2.3#53
|
||
|
||
PUT
|
||
|
||
Non-authoritative answer:
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.1
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.4
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.5
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.9
|
||
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.2
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.6
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.8
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.7
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Multi-value answers
|
||
NEW
|
||
|
||
Common Runtime (CRT)
|
||
BEST
|
||
|
||
PRACTICES
|
||
|
||
IMPLEMENTED
|
||
|
||
IN
|
||
|
||
CODE
|
||
|
||
Open source component in the AWS SDK
|
||
• Automated multipart uploads
|
||
|
||
• Parallelization of range GETs for downloads
|
||
• Built-in retry logic using multiple IPs
|
||
|
||
• And more
|
||
|
||
S3AsyncClient.crtCreate();
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
CRT info:
|
||
|
||
NEW
|
||
|
||
Mountpoint
|
||
|
||
Mountpoint with
|
||
Local Cache
|
||
|
||
NEW
|
||
|
||
Mountpoint for
|
||
Containers
|
||
|
||
The Mountpoint for Amazon S3 Family
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
NEW
|
||
|
||
Mountpoint for
|
||
S3 Express
|
||
|
||
Mountpoint in action
|
||
CMP214
|
||
|
||
HPC on AWS for semiconductors and healthcare life sciences
|
||
With access to scalable, cost-effective supercomputing capabilities in
|
||
the cloud, organizations can accelerate discoveries in semiconductors,
|
||
biopharma, and medicine
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Clients
|
||
|
||
Front end
|
||
Web servers, DNS, and network
|
||
|
||
Index
|
||
Key/value mapping to storage
|
||
|
||
Storage
|
||
Data durably stored on devices
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The S3 index is big
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The S3 index is big
|
||
350 trillion objects
|
||
100+ million requests per second
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Usage
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Capacity
|
||
|
||
A-F
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
G-M
|
||
|
||
N-S
|
||
|
||
T-Z
|
||
|
||
A-F
|
||
|
||
G-M
|
||
|
||
Q-S
|
||
|
||
N-P
|
||
SPLIT
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
T-Z
|
||
|
||
reinvent-bucket
|
||
|
||
What’s a prefix?
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
reinvent-bucket
|
||
|
||
What’s a prefix?
|
||
Any string of characters
|
||
after the bucket name
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
What’s a prefix?
|
||
reinvent-bucket/p
|
||
reinvent-bucket/prefix
|
||
reinvent-bucket/prefix1/
|
||
reinvent-bucket/prefix2/
|
||
reinvent-bucket/prefix1/data/other
|
||
reinvent-bucket/prefix1/data/otherstuff
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
3,500 PUT requests
|
||
5,500 GET requests
|
||
per prefix
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
reinvent-bucket/prefix1/a
|
||
|
||
reinvent-bucket/prefix1
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinventbucket/prefix1/b
|
||
5,500 GETs/second
|
||
|
||
SPLIT
|
||
|
||
reinvent-bucket/prefix2/a
|
||
|
||
reinvent-bucket/prefix2
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinventbucket/prefix2/b
|
||
5,500 GETs/second
|
||
|
||
Total TPS to reinvent-bucket is 5,500 x 4 = 22,000 TPS
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Usage
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Capacity
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Maximizing TPS with good key naming
|
||
Mechanisms for mitigation
|
||
|
||
Keep cardinality to
|
||
the left in key
|
||
names
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Keep dates to
|
||
the right in
|
||
key names
|
||
|
||
Key naming example (switch around day/prefix)
|
||
reinvent-bucket/day1/prefix1/a
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/day1/prefix1
|
||
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix1/b
|
||
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/day1
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix2/a
|
||
|
||
reinvent-bucket/day1/prefix2
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix2/b
|
||
5,500 GETs/second
|
||
|
||
Date far to the left in key name!!
|
||
|
||
Total TPS to reinvent-bucket/day1 is 5,500 x 4 = 22,000 TPS
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Key naming example
|
||
reinvent-bucket/day1/prefix1/a
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/day1/prefix1
|
||
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix1/b
|
||
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/day1
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix2/a
|
||
|
||
reinvent-bucket/day1/prefix2
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinvent-bucket/day1/prefix2/b
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/day2
|
||
5,500 GETs/second
|
||
|
||
• The partitions from day1 are now unused
|
||
• We will need to split day2 as it sees sustained load
|
||
• Likely throttling as that process occurs
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Key naming example
|
||
Date far to the right in key name!!
|
||
reinvent-bucket/prefix1/a/day1
|
||
reinvent-bucket/prefix1/a/day2
|
||
SPLIT
|
||
|
||
reinvent-bucket/prefix1
|
||
|
||
reinvent-bucket/prefix1/b/day1
|
||
reinvent-bucket/prefix1/b/day2
|
||
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/prefix
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
|
||
5,500 GETs/second
|
||
SPLIT
|
||
|
||
reinvent-bucket/prefix2/a/day1
|
||
reinvent-bucket/prefix2/a/day2
|
||
|
||
reinvent-bucket/prefix2
|
||
5,500 GETs/second
|
||
|
||
SPLIT
|
||
|
||
5,500 GETs/second
|
||
|
||
reinvent-bucket/prefix2/b/day1
|
||
reinvent-bucket/prefix2/b/day2
|
||
5,500 GETs/second
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Clients
|
||
|
||
Front end
|
||
Web servers, DNS, and network
|
||
|
||
Index
|
||
Key/value mapping to storage
|
||
|
||
Storage
|
||
Data durably stored on devices
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
We store a lot of data
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
We store a lot of data
|
||
Millions of hard drives
|
||
Exabytes of data
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
99.999999999%
|
||
DATA DURABILITY
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Threat: Device failure
|
||
|
||
We must protect data stored on drives that can fail or
|
||
corrupt bits at rest
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Achieving 11 9s of durability
|
||
|
||
End-to-end integrity
|
||
checking of requests
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Data always stored on
|
||
redundant devices
|
||
|
||
Periodic durability
|
||
auditing for data at rest
|
||
|
||
End-to-end integrity checking of requests
|
||
Data redundantly stored
|
||
and checksums taken in storage
|
||
Checksums taken in transit
|
||
|
||
device
|
||
device
|
||
|
||
S3:PutObject
|
||
200:SUCCESS
|
||
|
||
device
|
||
device
|
||
|
||
Data stored compared to data uploaded
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Data always stored on redundant devices
|
||
5 devices, each with different data
|
||
|
||
A
|
||
X
|
||
B
|
||
D
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
A
|
||
C
|
||
D
|
||
|
||
A
|
||
C
|
||
E
|
||
|
||
B
|
||
C
|
||
E
|
||
|
||
B
|
||
D
|
||
E
|
||
|
||
Data always stored on redundant devices
|
||
5 devices, each with different data
|
||
|
||
A
|
||
X
|
||
B
|
||
D
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
A
|
||
B
|
||
D
|
||
|
||
A
|
||
C
|
||
D
|
||
|
||
A
|
||
C
|
||
E
|
||
|
||
B
|
||
C
|
||
E
|
||
|
||
B
|
||
D
|
||
E
|
||
|
||
Data always stored on redundant devices
|
||
|
||
Now do it at scale
|
||
• Adequate spare capacity always on hand
|
||
• Aggressive monitoring for failure
|
||
• Redundancy maintained when hardware
|
||
inevitably fails
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Periodic durability auditing for data at rest
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
11 9s of durability
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
λt
|
||
|
||
-2
|
||
µt βt(-2)
|
||
|
||
λt
|
||
0
|
||
|
||
-1
|
||
µt βt(-1)
|
||
|
||
µt βt(0)
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Λtσt(C)
|
||
|
||
λt
|
||
|
||
λt
|
||
|
||
1
|
||
µt
|
||
|
||
C
|
||
µt
|
||
|
||
Λtσt(C+1) Λtσt(C+2)
|
||
C+2
|
||
|
||
C+1
|
||
|
||
µt
|
||
|
||
µt
|
||
|
||
Threat model: AZ loss
|
||
|
||
We must protect data we store against the unexpected total or partial
|
||
loss of a zone. We assume that any single facility may fail at any time
|
||
and that we must protect the durability of data that is stored within it.
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Multi-AZ by default
|
||
Multi-AZ by default
|
||
AZ1
|
||
|
||
AZ3
|
||
|
||
AZ2
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Multi-AZ by default
|
||
|
||
200:SUCCESS
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
AZ1
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
AZ2
|
||
|
||
AZ3
|
||
|
||
Amazon S3 Express One Zone
|
||
|
||
Full or partial loss of an Availability Zone may lose my data in
|
||
S3 Express One Zone
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
99.999999999% data durability
|
||
WITHIN
|
||
|
||
A
|
||
|
||
SINGLE
|
||
|
||
AVAILABILITY
|
||
|
||
ZONE
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
AZ1
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
AZ2
|
||
|
||
AZ3
|
||
|
||
99.999999999% data durability
|
||
WITHIN
|
||
|
||
A
|
||
|
||
SINGLE
|
||
|
||
AVAILABILITY
|
||
|
||
ZONE
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
AZ2
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
99.999999999% data durability
|
||
WITHIN
|
||
|
||
A
|
||
|
||
SINGLE
|
||
|
||
AVAILABILITY
|
||
|
||
ZONE
|
||
|
||
X
|
||
|
||
End-to-end integrity
|
||
checking of requests
|
||
|
||
Data always stored on
|
||
redundant devices
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Periodic durability
|
||
auditing for data at rest
|
||
|
||
Loss or damage to all or
|
||
part of an AZ
|
||
|
||
Accidental delete
|
||
WHEN
|
||
|
||
YOU
|
||
|
||
TELL
|
||
|
||
US
|
||
|
||
TO
|
||
|
||
DELETE
|
||
|
||
SOMETHING,
|
||
|
||
WE
|
||
|
||
DO
|
||
|
||
IT
|
||
|
||
We never want accidental deletion of data in Amazon S3 buckets,
|
||
especially due to bulk deletion operations
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Accidental delete
|
||
|
||
Mechanisms for mitigation
|
||
|
||
S3 Versioning
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
S3 Replication
|
||
|
||
S3 Object Lock
|
||
|
||
Backups
|
||
|
||
We are a global service
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
We are a global service
|
||
Operating in all AWS Regions
|
||
Regional storage classes span 3+ AZs
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
The power to touch is the power to destroy
|
||
|
||
The availability of one AWS Region can never affect the
|
||
availability of another
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Regional isolation
|
||
A
|
||
|
||
LEARNED
|
||
|
||
TENET
|
||
|
||
Amazon S3: 2006 - 2010
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
AZ failure threat model
|
||
|
||
We must remain available through the unexpected loss of an entire
|
||
zone. We assume that any single facility may fail at any time and that
|
||
we must continue to serve requests when an entire zone is offline.
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Availability during zone failure
|
||
|
||
200:SUCCESS
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
A
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
A
|
||
|
||
B
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
B
|
||
|
||
C
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
C
|
||
|
||
C
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
D
|
||
|
||
D
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
E
|
||
|
||
AZ1
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
AZ2
|
||
|
||
AZ3
|
||
|
||
Availability during zone failure
|
||
IT’S
|
||
|
||
ALWAYS
|
||
|
||
DNS
|
||
# nslookup s3.amazonaws.com
|
||
Server: 192.0.2.3
|
||
Address: 192.0.2.3 #53
|
||
Non-authoritative answer:
|
||
|
||
PUT
|
||
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.4
|
||
|
||
X
|
||
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.5
|
||
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.6
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.7
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.8
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.117
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.232
|
||
Name: s3.amazonaws.com
|
||
Address: 192.0.2.64
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
AZ fault tolerance: Not just for AZ faults!
|
||
|
||
• Software deployments
|
||
• New hardware adoption
|
||
• Configuration changes
|
||
• More
|
||
Separate fault domains
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Defense in depth
|
||
Building good guardrails
|
||
• Build for correctness
|
||
• Threats and mitigations
|
||
• But assume incorrectness
|
||
• Guardrails
|
||
|
||
Examples
|
||
|
||
• Shadow mode
|
||
• Control plane limits
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
To recap
|
||
|
||
No compression algorithm for
|
||
experience
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Thank you!
|
||
Amy Therrien
|
||
|
||
Seth Markle
|
||
|
||
atherrie@amazon.com
|
||
|
||
smarkle@amazon.com
|
||
|
||
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
|
||
|
||
Please complete the session
|
||
survey in the mobile app
|