BigQuery explained

BigQuery
Type:	Platform as a service data warehouse
Language:	English
Current Status:	Active
Registration:	Required
Owner:	Google

BigQuery is a managed, serverless data warehouse product by Google, offering scalable analysis over large quantities of data. It is a Platform as a Service (PaaS) that supports querying using a dialect of SQL. It also has built-in machine learning capabilities. BigQuery was announced in May 2010 and made generally available in November 2011.^[1]

Design

BigQuery provides external access to Google's Dremel technology,^{[2] [3]} a scalable, interactive ad hoc query system for analysis of nested data. BigQuery requires all requests to be authenticated, supporting a number of Google-proprietary mechanisms as well as OAuth.

Features

• Managing data - Create and delete objects such as tables, views, and user defined functions. Import data from Google Storage in formats such as CSV, Parquet, Avro or JSON.
• Query - Queries are expressed in a SQL dialect^[4] and the results are returned in JSON with a maximum reply length of approximately 128 MB, or an unlimited size when large query results are enabled.^[5]
• Integration - BigQuery can be used from Google Apps Script^[6] (e.g. as a bound script in Google Docs), or any language that can work with its REST API or client libraries.^[7]
• Access control - Share datasets with arbitrary individuals, groups, or the world.
• Machine learning - Create and execute machine learning models using SQL queries.
• Cross-cloud analytics - Analyze data across Google Cloud, Amazon Web Services, and Microsoft Azure^{[8] [9]}
• Data sharing - Exchange data and analytics assets across organizational boundaries.^[10]
• In-Memory analysis service - BI Engine built into BigQuery that enables users to analyze large and complex datasets interactively with sub-second query response time and high concurrency.^{[11] [12]}
• Business intelligence - Visualize data from BigQuery by importing into Data Studio, a data visualization tool ^[13]

Pricing

The two main components of BigQuery pricing are the cost to process queries and the cost to store data. BigQuery offers two types of pricing - on demand pricing which charges for the number of petabytes processed for each query and flat-rate pricing which charges for slots or virtual CPUs.^[14]

Partnerships & integrations

BigQuery partners and natively integrates with several tools:^[15]

• • BI and data visualization: Tableau, Microstrategy, ThoughtSpot, SAS, Qlik Neo4j and Dataiku Connectors and developer tools: CData, Progress, Magnitude, KingswaySoft, ZapppySys

Adoption

Customers of BigQuery include 20th Century Fox, American Eagle Outfitters, HSBC, CNA Insurance, Asahi Group, ATB Financial, Athena, The Home Depot, Wayfair, Carrefour, Oscar Health, and several others.^[16] Gartner named Google as a Leader in the 2021 Magic Quadrant™ for Cloud Database Management Systems.^[17] BigQuery is also named a Leader in The 2021 Forrester Wave: Cloud Data Warehouse.^[18] According to a study by Enterprise Strategy Group, BigQuery saves up to 27% in total cost of ownership over three years compared to other cloud data warehousing solutions.^[19]

Notes and References

1. Web site: Google opens BigQuery for cloud analytics: Dangles free trial to lure doubters . Iain Thomson . . November 14, 2011 . August 26, 2016 .
2. Web site: Dremel: Interactive Analysis of Web-Scale Datasets. Sergey Melnik . Andrey Gubarev . Jing Jing Long . Geoffrey Romer . Shiva Shivakumar . Matt Tolton . Theo Vassilakis . Proc. of the 36th International Conference on Very Large Data Bases (VLDB). 2010.
3. Web site: An Inside Look at Google BigQuery . Kazunori Sato . 2012 . August 26, 2016 .
4. Web site: SQL Reference. 26 June 2017.
5. Web site: Quota Policy. 26 June 2017.
6. Web site: BigQuery Service | Apps Script | Google Developers . March 15, 2018 . April 23, 2018 .
7. Web site: BigQuery Client Libraries. 26 June 2017.
8. Web site: bigquery .
9. Web site: Google Clouds BiqQuery Omni Now Generally Available. 12 October 2021 .
10. Web site: Analytics Hub.
11. Web site: BI Engine.
12. Web site: With Many Updates in BigQuery. 2 July 2022.
13. Web site: with Many Updates in BigQuery. 2 July 2022.
14. Web site: BigQuery Costs. 13 July 2023 .
15. Web site: BigQuery Section.
16. Web site: Customers for Data Analytics.
17. Web site: Whats Changed 2021 Gartner Magic Quadrant for Cloud Database Management Systems. 13 January 2022 .
18. Web site: BigQuery named leader in forrester wave cloud data warehouse. 30 March 2021 .
19. Web site: Economic Validation Google BigQuery va. Cloud Based EDWS.