Project Jupyter Explained

Project Jupyter
Abbreviation:Jupyter
Type:nonprofit organization
Purpose:Interactive data science and scientific computing
Region Served:Worldwide
Language:English

Project Jupyter is a project to develop open-source software, open standards, and services for interactive computing across multiple programming languages.

It was spun off from IPython in 2014 by Fernando Pérez and Brian Granger. Project Jupyter's name is a reference to the three core programming languages supported by Jupyter, which are Julia, Python and R. Its name and logo are an homage to Galileo's discovery of the moons of Jupiter, as documented in notebooks attributed to Galileo.

Jupyter is financially sponsored by NumFOCUS.[1]

History

The first version of Notebooks for IPython was released in 2011 by a team including Fernando Pérez, Brian Granger, and Min Ragan-Kelley.[2] In 2014, Pérez announced a spin-off project from IPython called Project Jupyter.[3] IPython continues to exist as a Python shell and a kernel for Jupyter, while the notebook and other language-agnostic parts of IPython moved under the Jupyter name.[4] [5] Jupyter supports execution environments (called "kernels") in several dozen languages, including Julia, R, Haskell, Ruby, and Python (via the IPython kernel).

In 2015, about 200,000 Jupyter notebooks were available on GitHub. By 2018, about 2.5 million were available.[6] In January 2021, nearly 10 million were available, including notebooks about the first observation of gravitational waves[7] and about the 2019 discovery of a supermassive black hole.

Major cloud computing providers have adopted the Jupyter Notebook or derivative tools as a frontend interface for cloud users. Examples include Amazon SageMaker Notebooks,[8] Google's Colab,[9] [10] and Microsoft's Azure Notebook.[11]

Visual Studio Code supports local development of Jupyter notebooks. As of July 2022, the Jupyter extension for VS Code has been downloaded over 40 million times, making it the second-most popular extension in the VS Code Marketplace.[12]

The steering committee of Project Jupyter received the 2017 ACM Software System Award, an annual award that honors people or an organization "for developing a software system that has had a lasting influence, reflected in contributions to concepts, in commercial acceptance, or both".[13]

The Atlantic published an article entitled "The Scientific Paper Is Obsolete" in 2018, discussing the role of Jupyter Notebook and the Mathematica notebook in the future of scientific publishing. Economist Paul Romer, in response, published a blog post in which he reflected on his experiences using Mathematica and Jupyter for research, concluding in part that Jupyter "does a better job of delivering what Theodore Gray had in mind when he designed the Mathematica notebook."[14]

In 2021, Nature named Jupyter as one of ten computing projects that transformed science.[15]

Jupyter Notebook

Jupyter Notebook can colloquially refer to two different concepts, either the user-facing application to edit code and text, or the underlying file format which is interoperable across many implementations.

Applications

Jupyter Notebook (formerly IPython Notebook) is a web-based interactive computational environment for creating notebook documents. Jupyter Notebook is built using several open-source libraries, including IPython, ZeroMQ, Tornado, jQuery, Bootstrap, and MathJax. A Jupyter Notebook application is a browser-based REPL containing an ordered list of input/output cells which can contain code, text (using Github Flavored Markdown), mathematics, plots and rich media.

Jupyter Notebook is similar to the notebook interface of other programs such as Maple, Mathematica, and SageMath, a computational interface style that originated with Mathematica in the 1980s. Jupyter interest overtook the popularity of the Mathematica notebook interface in early 2018.[16]

JupyterLab is a newer user interface for Project Jupyter, offering a flexible user interface and more features than the classic notebook UI. The first stable release was announced on February 20, 2018.[17] [18] In 2015, a joint $6 million grant from The Leona M. and Harry B. Helmsley Charitable Trust, The Gordon and Betty Moore Foundation, and The Alfred P. Sloan Foundation funded work that led to expanded capabilities of the core Jupyter tools, as well as to the creation of JupyterLab.[19]

GitHub announced in November 2022 that JupyterLab would be available in its online Coding platform called Codespace.[20]

In August 2023, Jupyter AI, a Jupyter extension, was released. This extension incorporates generative artificial intelligence into Jupyter notebooks, enabling users to explain and generate code, rectify errors, summarize content, inquire about their local files, and generate complete notebooks based on natural language prompts. [21]

JupyterHub is a multi-user server for Jupyter Notebooks. It is designed to support many users by spawning, managing, and proxying many singular Jupyter Notebook servers.[22]

Documents

A Jupyter Notebook document is a JSON file, following a versioned schema, usually ending with the ".ipynb" extension.The main parts of the Jupyter Notebooks are: Metadata, Notebook format and list of cells. Metadata is a data Dictionary of definitions to set up and display the notebook. Notebook Format is a version number of the software. List of cells are different types of Cells for Markdown (display), Code (to execute), and output of the code type cells.[23]

While JSON is the most common format, it is possible to forgo some features (like storing images and metadata), and save notebooks as markdown documents using extensions like JupyText. Jupytext is often used in conjunction with version control to make diffing and merging of notebook simpler.

See also

External links

Notes and References

  1. Web site: NumFOCUS Sponsored Projects. NumFOCUS. 2021-10-25.
  2. News: Vu . Linda . Project Jupyter: A Computer Code that Transformed Science . August 15, 2022 . Berkeley Lab Computing Sciences . June 14, 2021 . en.
  3. Web site: Project Jupyter // Speaker Deck.
  4. Web site: The Notebook, Qt console and a number of other pieces are now parts of Jupyter. GitHub. 29 May 2021.
  5. Web site: The Big Split™. 28 August 2017.
  6. Perkel . Jeffrey M. . Why Jupyter is data scientists' computational notebook of choice . August 15, 2022 . Nature . October 30, 2018 . 563 . 7729 . 145–146 . en . 10.1038/d41586-018-07196-1. 30375502 . 2018Natur.563..145P . 256770398 .
  7. LIGO Open Science Center. losc.ligo.org. 2016 . 10.7935/K5MW2F23 . en. 2018-05-04. LIGO Scientific Collaboration .
  8. News: Gallagher . Sean . Machine learning, concluded: Did the "no-code" tools beat manual analysis? . August 15, 2022 . Ars Technica . August 15, 2022 . en-us.
  9. News: Sherrer . Kara . Google Colab vs Jupyter Notebook: Compare data science software . August 15, 2022 . TechRepublic . May 25, 2022.
  10. News: Nerds rejoice: Google just released its internal tool to collaborate on AI. Quartz. 2018-09-06. en-US.
  11. News: Wayner . Peter . Essential data science tools for elevating your analytics operations . August 15, 2022 . CIO . May 5, 2022.
  12. News: Ramel . David . VS Code and Python: A Natural Fit for Data Science - . August 15, 2022 . Visual Studio Magazine . July 12, 2022.
  13. Web site: Software System Award . ACM Awards . . . 2016-05-05 . https://web.archive.org/web/20160505152017/http://awards.acm.org/software_system/year.cfm . dead .
  14. Web site: Jupyter, Mathematica, and the Future of the Research Paper . Paul . Romer. paulromer.net. en-US. 2018-04-15.
  15. Perkel . Jeffrey M. . Ten computer codes that transformed science . August 15, 2022 . Nature . January 20, 2021 . 589 . 7842 . 344–348 . en . 10.1038/d41586-021-00075-2. 33473232 . 2021Natur.589..344P . 231663425 .
  16. News: The Scientific Paper Is Obsolete. Somers. James. The Atlantic. 2018-04-10. en-US.
  17. News: JupyterLab is Ready for Users . 2018-02-20. Jupyter Blog. 2018-05-04.
  18. News: Brust . Andrew . Data science notebooks get real: JupyterLab releases to users . August 15, 2022 . ZDNet . February 24, 2018 . en.
  19. Web site: UC Berkeley and Cal Poly to Expand and Enhance Open-Source Software for Scientific Computing and Data Science Helmsley Charitable Trust. helmsleytrust.org. en. 2018-05-03. 2020-01-03. https://web.archive.org/web/20200103220158/https://helmsleytrust.org/news/uc-berkeley-and-cal-poly-expand-and-enhance-open-source-software-scientific-computing-and-data. dead.
  20. Web site: Using Codespaces with JupyterLab (Public Beta) GitHub Changelog . 2022-11-11 . The GitHub Blog . en-US.
  21. Web site: Weill . Jason . Generative AI in Jupyter . live . https://archive.today/20230814170112/https://blog.jupyter.org/generative-ai-in-jupyter-3f7174824862 . August 14, 2023 . August 19, 2023 . Jupyter.
  22. News: Lahoti . Sugandha . JupyterHub 1.0 releases with named servers, support for TLS encryption and more . August 15, 2022 . Packt Hub . May 6, 2019.
  23. Book: Toomey, Dan . Learning Jupyter . Packt . 2016 . 978-1-78588-487-0 . 1st . Birmingham - Mumbai . 21.