DuckDB explained

DuckDB
Developer:DuckDB Labs
Latest Release Version:v1.0.0
Programming Language:C++
Operating System:Cross-platform
Genre:Column-oriented DBMS
RDBMS
License:MIT License

DuckDB is an open-source column-oriented relational database management system (RDBMS) originally developed by Mark Raasveldt and Hannes Mühleisen at the Centrum Wiskunde & Informatica (CWI) in the Netherlands[1] and first released in 2019.[2] The project has over 6 million downloads per month.[3] [4] [5] It is designed to provide high performance on complex queries against large databases in embedded configuration,[1] such as combining tables with hundreds of columns and billions of rows. Unlike other embedded databases (for example, SQLite) DuckDB is not focusing on transactional (OLTP) applications and instead is specialized for online analytical processing (OLAP) workloads.[6]

DuckDB in its OLAP niche does not compete with the traditional DBMS like MSSQL, PostgreSQL and Oracle database. While using SQL for queries, DuckDB targets the serverless applications and provides extremely fast responses using Apache Parquet files for storage. These attributes make it a popular choice for large dataset analysis in interactive mode, but match poorly the requirements of the enterprise data storage.[7]

DuckDB uses a vectorized query processing engine. DuckDB is special amongst database management systems because it does not have any external dependencies and can build with just a C++11 compiler.[8] DuckDB also deviates from the traditional client–server model by running inside a host process (it has bindings, for example, for a Python interpreter with the ability to directly place data into NumPy arrays[1]).

Commercial use

DuckDB is used at Facebook, Google, and Airbnb.[9]

DuckDB co-author Mühleisen also runs a support and consultancy firm for the software, DuckDB Labs.[2] The company has chosen not to take venture capital funding, stating "We feel investment would force the project direction towards monetization, and we would much prefer keeping DuckDB open and available for as many people as possible".[5] Another company, MotherDuck, has received $100m funding for its data platform based on DuckDB, with investors including Andreessen Horowitz.[10]

Further reading

External links

Notes and References

  1. Book: Kamphuis, Chris . Advances in Information Retrieval . Graph Databases for Information Retrieval . Springer International Publishing . Cham . 12036 . 2020 . 978-3-030-45441-8 . 7148032 . 10.1007/978-3-030-45442-5_79 . 608–612.
  2. Web site: Clark . Lindsay . DuckDB reaches version 0.5.0 . 2024-03-23 . www.theregister.com . en . 2024-03-07 . https://web.archive.org/web/20240307163220/https://www.theregister.com/2022/09/09/duckdb_0_5_0/ . live .
  3. Web site: PyPi Download Stats . 2024-08-13 . www.pypistats.org . en . 2024-08-13 . https://web.archive.org/web/20240813165631/https://pypistats.org/packages/duckdb . live .
  4. Web site: DuckDB Python Downloads Dashboard . 2024-08-13 . duckdbstats.com . en . 2024-08-13 . https://web.archive.org/web/20240813165159/https://duckdbstats.com/ . live .
  5. Web site: Clark . Lindsay . DuckDB Labs puts limit on free support, rules out VC funding . 2024-03-23 . www.theregister.com . en . 2024-03-23 . https://web.archive.org/web/20240323064605/https://www.theregister.com/2023/10/05/duckdb_labs_puts_limit_on_vc_funds/ . live .
  6. Raasveldt . Mark . Mühleisen . Hannes . DuckDB: an Embeddable Analytical Database . ACM . 2019-06-25 . 978-1-4503-5643-5 . 10.1145/3299869.3320212 . 1981–1984.
  7. Book: Bannert, M. . Research Software Engineering: A Guide to the Open Source Ecosystem . CRC Press . Chapman & Hall/CRC Data Science Series . 2024 . 978-1-04-000513-2 . 2024-03-23 . 25 . 2024-03-23 . https://web.archive.org/web/20240323010627/https://books.google.com/books?id=yWL7EAAAQBAJ&pg=PT25 . live .
  8. Web site: DuckDB Building Instructions . 2024-08-16 .
  9. Web site: Clark . Lindsay . Scale-up database wrangler MotherDuck scores $47.5 million . 2024-03-23 . www.theregister.com . en . 2024-03-23 . https://web.archive.org/web/20240323064604/https://www.theregister.com/2022/11/17/475_million_says_scaleup_databases/ . live .
  10. Web site: Clark . Lindsay . MotherDuck serverless analytics platform wins $52.5M funding . 2024-03-23 . www.theregister.com . en . 2024-03-23 . https://web.archive.org/web/20240323064604/https://www.theregister.com/2023/09/21/motherduck_funding/ . live .