Append-only explained

Append-only is a property of computer data storage such that new data can be appended to the storage, but where existing data is immutable.

Access control

Many file systems' Access Control Lists implement an "append-only" permission:

• chattr in Linux can be used to set the append-only flag to files and directories. This corresponds to the flag in .
• NTFS ACL has a control for "Create Folders / Append Data", but it does not seem to keep data immutable.^[1]

Many cloud storage providers provide the ability to limit access as append-only.^[2] This feature is especially important to mitigate the risk of data loss for backup policies in the event that the computer being backed-up becomes infected with ransomware capable of deleting or encrypting the computer's backups.^{[3] [4]}

Data structures

Many data structures and databases implement immutable objects, effectively making their data structures append-only. Implementing an append-only data structure has many benefits, such as ensuring data consistency, improving performance,^[5] and permitting rollbacks.^{[6] [7]}

The prototypical append-only data structure is the log file. Log-structured data structures found in Log-structured file systems and databases work in a similar way: every change (transaction) that happens to the data is logged by the program, and on retrieval the program must combine the pieces of data found in this log file. Blockchains add cryptography to the logs so that every transaction is verifiable.

Append-only data structures may also be mandated by the hardware or software environment:

• All objects are immutable in purely functional programming languages, where every function is pure and global states do not exist.^[8]
• Flash storage cells can only be written to once before erasing. Erasing on a flash drive works on the level of pages which cover many cells at once, so each page is treated as an append-only set of cells until it fills up.^{[9] [10]}
• Hard drives that use shingled magnetic recording cannot be written to randomly because writing on a track would clobber a neighboring, usually later, track. As a result, each "zone" on the drive is append-only.^{[11] [5]}

Append-only data structures grow over time, with more and more space dedicated to "stale" data found only in the history and more time wasted on parsing these data. A number of append-only systems implement rewriting (copying garbage collection), so that a new structure is created only containing the current version and optionally a few older ones.^{[6] [12]}

See also

• Access control list
• Cloud storage
• Comparison of file hosting services
• Data structure
• Purely-functional data structure
• Log-structured merge-tree
• Certificate Transparency
• Write once read many

Notes and References

1. Web site: powershell - How to give "only append" access to user in windows, for logging purposes . Server Fault.
2. Web site: Jim Donovan. Wasabi. Why Use Immutable Storage?. September 11, 2018.
3. Eugene Kolodenker . William Koch . Gianluca Stringhini . Manuel Egele . PayBreak: Defense Against Cryptographic Ransomware . Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security . 599–611 . 10.1145/3052973.3053035 . April 2017. free . Due to the threat of ransomware targeting the key vault, our implementation stores the harvested key material into an append-only file protected with Administrator privileges..
4. Book: Pont . Jamie . Abu Oun . Osama . Brierley . Calvin . Arief . Budi . Hernandez-Castro . Julio . Secure IT Systems, Proceedings of 24th Nordic Conference, NordSec 2019 . 2019 . Springer International Publishing . 978-3-030-35055-0 . 137–154 . en . A Roadmap for Improving the Impact of Anti-ransomware Research.
5. Web site: Extending Magic Pocket Innovation with the first petabyte scale SMR drive deployment. Magic Pocket Hardware Engineering Teams. dropbox.tech.
6. Web site: Redis Persistence. Redis.
7. Web site: Additional Notes. Borg Deduplicating Archiver 1.1.11 documentation.
8. Web site: Thirteen ways of looking at a turtle . F# for fun and profit. en. 2018-11-13.
9. Reid . Colin . Bernstein . Phil . Implementing an Append-Only Interface for Semiconductor Storage . IEEE Data Eng. Bull. . 1 January 2010 . 33 . 14–20 .
10. Web site: NVMe Zoned Namespace . ZonedStorage.io . The internals of Solid State Drives are such that they implement a log-structured data structure, where data is written sequentially to the media. . 2020-04-25 . 2020-01-29 . https://web.archive.org/web/20200129163241/https://zonedstorage.io/introduction/zns/ . dead .
11. Web site: Support for shingled magnetic recording devices . Jake Edge . . March 26, 2014 . December 14, 2014.
12. Brewer . Eric . Ying . Lawrence . Greenfield . Lawrence . Cypher . Robert . T'so . Theodore . Disks for Data Centers . Proceedings of USENIX FAST 2016 . 2016 . en . Because of the write restrictions imposed by SMR, when data is deleted, that deleted capacity can not be reused until the system copies the remaining live data in that SMR zone to another part of the disk, a form of garbage collection (GC)..