| Quantcast File System (QFS) | |
| Developer: | Sriram Rao, Michael Ovsiannikov, Quantcast |
| Latest Release Version: | 2.2.6 |
| Latest Release Date: | [1] |
| Programming Language: | C++ |
| Genre: | Distributed File System |
| License: | Apache License 2.0 |
Quantcast File System (QFS) is an open-source distributed file system software package for large-scale MapReduce or other batch-processing workloads. It was designed as an alternative to the Apache Hadoop Distributed File System (HDFS), intended to deliver better performance and cost-efficiency for large-scale processing clusters.
QFS is software that runs on a cluster of hundreds or thousands of commodity Linux servers and allows other software layers to interact with them as if they were one giant hard drive. It has three components:
In a cluster of hundreds or thousands of machines, the odds are low that all will be running and reachable at any given moment, so fault tolerance is the central design challenge. QFS meets it with Reed–Solomon error correction. The form of Reed–Solomon encoding used in QFS stores redundant data in nine places and can reconstruct the file from any six of these stripes.[2] When it writes a file, it by default stripes it across nine physically different machines — six holding the data, three holding parity information. Any three of those can become unavailable. If any six remain readable, QFS can reconstruct the original data. The result is fault tolerance at a cost of a 50% expansion of data.
QFS is written in the programming language C++, operates within a fixed memory footprint, and uses direct input and output (I/O).
QFS evolved from the Kosmos File System (KFS), an open source project started by Kosmix in 2005. Quantcast adopted KFS in 2007, built its own improvements on it over the next several years, and released QFS 1.0 as an open source project in September, 2012.[3]