Amazon S3 Explained

Amazon S3
Type:Cloud storage
Language:English
Current Status:Active
Ipv6:Yes
Commercial:Yes
Registration:Required (included in free tier layer)
Owner:Amazon.com

Amazon Simple Storage Service (S3) is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface.[1] Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its e-commerce network.[2] Amazon S3 can store any type of object, which allows uses like storage for Internet applications, backups, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage. AWS launched Amazon S3 in the United States on March 14, 2006,[3] [4] then in Europe in November 2007.[5]

Technical details

Design

Amazon S3 manages data with an object storage architecture[6] which aims to provide scalability, high availability, and low latency with high durability. The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key.[7] Buckets can be managed using the console provided by Amazon S3, programmatically with the AWS SDK, or the REST application programming interface. Objects can be up to five terabytes in size.[8] [9] Requests are authorized using an access control list associated with each object bucket and support versioning[10] which is disabled by default.[11] Since buckets are typically the size of an entire file system mount in other systems, this access control scheme is very coarse-grained. In other words, unique access controls cannot be associated with individual files. Amazon S3 can be used to replace static web-hosting infrastructure with HTTP client-accessible objects,[12] index document support, and error document support.[13] The Amazon AWS authentication mechanism allows the creation of authenticated URLs, valid for a specified amount of time. Every item in a bucket can also be served as a BitTorrent feed. The Amazon S3 store can act as a seed host for a torrent and any BitTorrent client can retrieve the file. This can drastically reduce the bandwidth cost for the download of popular objects. A bucket can be configured to save HTTP log information to a sibling bucket; this can be used in data mining operations.[14] There are various User Mode File System (FUSE)–based file systems for Unix-like operating systems (for example, Linux) that can be used to mount an S3 bucket as a file system. The semantics of the Amazon S3 file system are not that of a POSIX file system, so the file system may not behave entirely as expected.[15]

Amazon S3 storage classes

Amazon S3 offers nine different storage classes with different levels of durability, availability, and performance requirements.[16]

The Amazon S3 Glacier storage classes above are distinct from Amazon Glacier, which is a separate product with its own APIs.

File size limits

An object in S3 can be between 0 bytes and 5TB. If an object is larger than 5TB, it must be divided into chunks prior to uploading. When uploading, Amazon S3 allows a maximum of 5GB in a single upload operation; hence, objects larger than 5GB must be uploaded via the S3 multipart upload API.[18]

Uses

Notable users

S3 API and competing services

The broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3 API. These services use the standard programming interface but are differentiated by their underlying technologies and business models.[29] A standard interface enables better competition from rival providers and allows economies of scale in implementation, among other benefits.[30]

History

Amazon Web Services introduced Amazon S3 in 2006.[31] [32]

DateNumber of Items Stored
October 200710 billion[33]
January 200814 billion
October 200829 billion[34]
March 200952 billion[35]
August 200964 billion[36]
March 2010102 billion[37]
April 20132 trillion[38]
March 2021100 trillion[39]
March 2023280 trillion[40]

In November 2017 AWS added default encryption capabilities at bucket level.[41]

See also

References

Sources

Notes and References

  1. Book: Mobile Cloud Computing: Foundations and Service Models. Huang. Dijiang. Wu. Huijun. 2017-09-08. Morgan Kaufmann. 9780128096444. 67. en. 2018-11-15. https://web.archive.org/web/20181115112855/https://books.google.ca/books?id=dupGDgAAQBAJ&source=gbs_navlinks_s. 2018-11-15. live.
  2. Web site: Cloud Object Storage - Store & Retrieve Data Anywhere - Amazon Simple Storage Service. Amazon Web Services, Inc.. 2018-05-17. https://web.archive.org/web/20180517083922/https://aws.amazon.com/s3/. 2018-05-17. live.
  3. 2006-03-14. Amazon Web Services Launches "Amazon S3". 2018-11-14. https://web.archive.org/web/20181115112821/https://press.aboutamazon.com/news-releases/news-release-details/amazon-web-services-launches-amazon-s3-simple-storage-service. 2018-11-15. live.
  4. News: 5 Key Events in the history of Cloud Computing - DZone Cloud. dzone.com. 2018-09-28. en. https://web.archive.org/web/20180929000541/https://dzone.com/articles/5-key-events-history-cloud. 2018-09-29. live.
  5. Amazon Web Services Offers European Storage for Amazon S3 . 2007-11-06 . 2018-11-14 . https://web.archive.org/web/20181115112804/https://press.aboutamazon.com/news-releases/news-release-details/amazon-web-services-offers-european-storage-amazon-s3 . 2018-11-15 . live.
  6. Web site: What is Cloud Object Storage? – AWS . 2019-10-16 . Amazon Web Services, Inc. . en-US . live . https://web.archive.org/web/20180920181620/https://aws.amazon.com/what-is-cloud-object-storage/ . 2018-09-20 . 2018-07-09.
  7. Web site: Tech Blog » Starting Websphere in Cloud and saving the data in S3. techblog.aasisvinayak.com. https://web.archive.org/web/20100312080332/http://techblog.aasisvinayak.com/starting-websphere-in-cloud-and-saving-the-data-in-s3. 2010-03-12. dead.
  8. Web site: open-guides/og-aws. GitHub. 2018-05-17. https://web.archive.org/web/20180103123141/https://github.com/open-guides/og-aws/blob/master/README.md#s3#s3. 2018-01-03. live.
  9. Web site: Error Responses - Amazon Simple Storage Service. docs.aws.amazon.com. 2018-05-21. https://web.archive.org/web/20171224010324/http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html. 2017-12-24. live.
  10. Web site: Using versioning in S3 buckets - Amazon Simple Storage Service .
  11. Web site: Introduction to Amazon S3 - Amazon Simple Storage Service. docs.aws.amazon.com. 2018-05-17. https://web.archive.org/web/20180512235723/https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html. 2018-05-12. live.
  12. Web site: How to use Amazon S3 for Web Hosting. bucketexplorer.com. 2008-05-06. https://web.archive.org/web/20080408024708/http://www.bucketexplorer.com/documentation/amazon-s3--how-to-use-Amazon-s3-for-web-hosting.html. 2008-04-08. dead.
  13. http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?WebsiteHosting.html Amazon Simple Storage Service
  14. http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html Server Access Logging
  15. Web site: Comparison of S3QL and other S3 file systems . 2012-06-29 . https://web.archive.org/web/20120805030057/http://code.google.com/p/s3ql/wiki/other_s3_filesystems . 2012-08-05 . live.
  16. Web site: Cloud Storage Classes – Amazon Simple Storage Service (S3) – AWS. Amazon Web Services, Inc.. 2018-05-17. https://web.archive.org/web/20180613200647/https://aws.amazon.com/s3/storage-classes/. 2018-06-13. live.
  17. Web site: 2023-11-28 . Announcing the new Amazon S3 Express One Zone high performance storage class AWS News Blog . 2023-12-01 . aws.amazon.com . en-US.
  18. Web site: How to Upload Large Files to S3 . June 21, 2022 . June 22, 2022 . October 1, 2022 . https://web.archive.org/web/20221001213248/https://riyanchristy.goseeq.net/how-to-upload-large-files-to-s3-efficiently/ . dead .
  19. Web site: Amazon S3: Show Me the Money. November 10, 2006. SmugMug Blog. SmugMug. 2017-03-03. https://web.archive.org/web/20170303130206/https://donmacaskill.wordpress.com/2006/11/10/amazon-s3-show-me-the-money/. 2017-03-03. live.
  20. Web site: S3mper: Consistency in the Cloud. 2016-05-01. https://web.archive.org/web/20160424205437/http://techblog.netflix.com/2014/01/s3mper-consistency-in-cloud.html. 2016-04-24. live.
  21. Web site: Introduction to Amazon S3. Amazon. 28 December 2017. https://web.archive.org/web/20171225230323/http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html. 2017-12-25. live.
  22. Web site: Amazon Web Services: the secret to the online retailer's future success. Hern. Alex. 2017-02-02. the Guardian. en. 2018-04-23. https://web.archive.org/web/20180502223750/https://www.theguardian.com/technology/2017/feb/02/amazon-web-services-the-secret-to-the-online-retailers-future-success. 2018-05-02. live.
  23. Web site: AWS Case Study: reddit . aws.amazon.com . 2015 . March 18, 2015 . https://web.archive.org/web/20150317005440/http://aws.amazon.com/solutions/case-studies/reddit/ . 2015-03-17 . live .
  24. Web site: Bitcasa Legal. May 16, 2013. 2013-05-16. https://archive.today/20130628192701/https://www.bitcasa.com/legal. 2013-06-28. dead.
  25. Web site: What is Tahoe-LAFS-on-S3?. August 21, 2012. 2012-08-21. https://web.archive.org/web/20130506021742/https://leastauthority.com/products. 2013-05-06. live.
  26. The Epic Story of Dropbox's Exodus From the Amazon Cloud Empire. WIRED. 2018-04-23. en-US. https://web.archive.org/web/20180125195150/https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/. 2018-01-25. live.
  27. News: Dropbox saved almost $75 million over two years by building its own tech infrastructure. 2018-02-23. GeekWire. 2018-04-23. en-US. https://web.archive.org/web/20180423170423/https://www.geekwire.com/2018/dropbox-saved-almost-75-million-two-years-building-tech-infrastructure/. 2018-04-23. live.
  28. Web site: Swiftype Explains Their Cloud Stack. July 1, 2013. 2014-12-08. https://web.archive.org/web/20141208083021/http://stackshare.io/posts/swiftype-explains-their-cloud-stack. 2014-12-08. live.
  29. Web site: Watters. Audrey. Cloud Community Debates, Is Amazon S3's API the Standard? (And Should It Be?). 12 July 2010. SAY Media, Inc.. 19 December 2012. https://web.archive.org/web/20130217035941/http://readwrite.com/2010/07/12/cloud-community-debates-is-ama. 2013-02-17. live.
  30. Book: Committee on Standards Workshop Planning, Board on Telecommunications and Computer Applications, Commission on Engineering and Technical Systems, National Research Council. Crossroads of Information Technology Standards. 1990. The National Academies Press, 1990.. Washington, DC. 36–37. 10.17226/10440. 978-0-309-58171-4. 2014-03-25. https://web.archive.org/web/20140325160847/http://www.nap.edu/openbook.php?record_id=10440&page=36. 2014-03-25. live.
  31. Overview of Amazon Web Services, 2018, https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.html
  32. Garfinkel, Simson L. 2007. An Evaluation of Amazon's Grid Computing Services: EC2, S3, and SQS. Harvard Computer Science Group Technical Report TR-08-07. https://dash.harvard.edu/bitstream/handle/1/24829568/tr-08-07.pdf?sequence=1
  33. Web site: Happy Birthday, Amazon S3!. 2008-03-19. Vogels, Werner. All Things Distributed. 2008-05-23. https://web.archive.org/web/20080509070955/http://www.allthingsdistributed.com/2008/03/happy_birthday_amazon_s3.html. 2008-05-09. live.
  34. Web site: 2008-10-08 . Amazon S3 - Busier Than Ever . 2008-10-09 . https://web.archive.org/web/20081011012039/http://aws.typepad.com/aws/2008/10/amazon-s3---now.html . 2008-10-11 . dead.
  35. Web site: Celebrating S3's Third Birthday With Special Anniversary Pricing - Amazon Web Services. 31 March 2009. typepad.com. 2009-04-01. https://web.archive.org/web/20110707102355/http://aws.typepad.com/aws/2009/03/celebrating-s3s-third-birthday-with-an-upload-promotion.html. 2011-07-07. live.
  36. Web site: Amazon's Head Start in the Cloud Pays Off. eweek.com.
  37. Web site: Amazon S3 Now Hosts 100 Billion Objects. 9 March 2010. datacenterknowledge.com. 2010-03-09. https://web.archive.org/web/20100312102850/http://www.datacenterknowledge.com/archives/2010/03/09/amazon-s3-now-hosts-100-billion-objects/. 2010-03-12. live.
  38. Web site: Amazon S3 – Two Trillion Objects, 1.1 Million Requests / Second - Amazon Web Services. 18 April 2013. typepad.com.
  39. Web site: Celebrate 15 Years of Amazon S3 with 'Pi Week' Livestream Events. 14 March 2021. amazon.com.
  40. Web site: Celebrate Amazon S3’s 17th birthday at AWS Pi Day 2023. 14 March 2023. amazon.com.
  41. Web site: New Amazon S3 Encryption & Security Features. 6 November 2017.

This article is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article "Amazon S3".

Except where otherwise indicated, Everything.Explained.Today is © Copyright 2009-2024, A B Cryer, All Rights Reserved. Cookie policy.