OpenVINO explained

OpenVINO
Logo Alt:OpenVINO logo
Developer:Intel Corporation
Latest Release Version:2024.3
Latest Release Date:July 2024.[1]
Programming Language:C++
Operating System:Cross-platform
License:Apache License 2.0
Asof:July 2024

OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. It enables programmers to develop scalable and efficient AI solutions with relatively few lines of code. It supports several popular model formats and categories, such as large language models, computer vision, and generative AI.

Actively developed by Intel, it prioritizes high-performance inference on Intel hardware but also supports ARM/ARM64 processors[2] and encourages contributors to add new devices to the portfolio.

Based in C++, it offers the following APIs: C/C++, Python, and Node.js (an early preview).

OpenVINO is cross-platform and free for use under Apache License 2.0.[3]

Workflow

The simplest OpenVINO usage involves obtaining a model and running it as is. Yet for the best results, a more complete workflow is suggested:[4]

OpenVINO model format

OpenVINO IR[5] is the default format used to run inference. It is saved as a set of two files, *.bin and *.xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application's API or a dedicated converter.

Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference. Some pre-converted models can be found in the Hugging Face repository.[6]

The supported model formats are:[7]

OS support

OpenVINO runs on Windows, Linux and MacOS.[8]

See also

References

Notes and References

  1. Web site: Release Notes for Intel Distribution of OpenVINO toolkit 2024.3. July 2024.
  2. Web site: 24 Jan 2024 . OpenVINO Compatibility and Support . OpenVINO Documentation.
  3. Web site: 16 Oct 2018 . License . OpenVINO repository.
  4. Web site: 25 April 2024 . OpenVINO Workflow . OpenVINO Documentation.
  5. Web site: 2 Feb 2024 . OpenVINO IR . www.docs.openvino.ai.
  6. Web site: Hugging Face OpenVINO Space . Hugging Face.
  7. Web site: 24 Jan 2024 . OpenVINO Model Preparation . OpenVINO Documentation.
  8. Web site: February 2024 . System Requirements . OpenVINO Documentation.