AWS Simple Storage Service(S3) Bucket Types
What is a S3 Bucket and its Types
What is a S3 bucket?#
Amazon Simple Storage Service (Amazon S3) is a distributed storage service that offers industry-leading scalability, data availability, security, and performance. There are other types of storage services offered by AWS like Elastic Block Service(EBS), Elastic File System(EFS) etc.
S3 is used to store any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics.
To store your data in Amazon S3, you first create a bucket which is a container for storing your data.
Originally S3 offered only general purpose buckets storing any type of data, over time S3 started to support other types of buckets as the use cases evolved.
Types#
Currently four types of buckets are supported - General Purpose, Directory, Table and Vector. Each type of bucket provides a unique set of features for different use cases.
General Purpose Buckets#
A general purpose bucket is a container for objects stored in Amazon S3, and you can store any number of objects in a bucket.
An object is a file and any metadata that describes that file. Each object can be up to 5 TB in size.
Directory Buckets#
Directory buckets organize data hierarchically into directories as opposed to the flat storage structure of general purpose buckets. There aren’t prefix limits for directory buckets, and individual directories can scale horizontally.
You can create up to 100 directory buckets in each of your AWS accounts, with no limit on the number of objects that you can store in a bucket. Your bucket quota is applied to each Region in your AWS account.
Directory Buckets support low latency and data-residency use cases.
With low latency use case directory buckets support the S3 Express One Zone storage class. A storage class is a category of storage based on the performance, data access, resiliency, and cost requirements of your workloads.With S3 Express One Zone storgae class, your data is redundantly stored on multiple devices within a single Availability Zone.
With data-residency use case you can create a directory bucket in a single AWS Dedicated Local Zone (DLZ) to store data. In Dedicated Local Zones, you can create S3 directory buckets to store data in a specific data perimeter, which helps support your data residency and isolation use cases. Directory buckets in Local Zones support the S3 One Zone-Infrequent Access storage class.S3 One Zone-IA storage class is used for data that is accessed less frequently, but requires rapid access when needed.
Table Buckets#
Table buckets are a new type of bucket and are used to store structured tabular data in S3 Tables, such as daily purchase transactions, streaming sensor data, or ad impressions. S3 Tables provide S3 storage that’s optimized for analytics workloads by storing data in the Apache Iceberg format.
Vector Buckets#
S3 Vector Buckets offer purpose-built storage in the cloud to store and query vectors. Vectors are used to store embeddings generated by machine learning models.An embedding is a low-dimensional numerical representation of unstructured data, like text, images, or audio, that captures its underlying meaning and relationships. Within a vector bucket, you can organize your vector data with vector indexes. You can perform similarity queries on your vector data within vector indexes.