Advanced Matrix Extensions Explained

Advanced Matrix Extensions (AMX), also known as Intel Advanced Matrix Extensions (Intel AMX), are extensions to the x86 instruction set architecture (ISA) for microprocessors from Intel designed to work on matrices to accelerate artificial intelligence (AI) and machine learning (ML) workloads.^[1]

Extensions

AMX was introduced by Intel in June 2020 and first supported by Intel with the Sapphire Rapids microarchitecture for Xeon servers, released in January 2023.^{[2] [3]} It introduced 2-dimensional registers called tiles upon which accelerators can perform operations. It is intended as an extensible architecture; the first accelerator implemented is called tile matrix multiply unit (TMUL).^{[4] [5]}

In Intel Architecture Instruction Set Extensions and Future Features revision 46, published in September 2022, a new AMX-FP16 extension was documented. This extension adds support for half-precision floating-point numbers. In revision 48 from March 2023, AMX-COMPLEX was documented, adding support for half-precision floating-point complex numbers. Both extensions are planned for inclusion in the future Granite Rapids processors (AMX-COMPLEX - only in Granite Rapids-D^[6]).

Tile matrix multiply unit

TMUL unit supports BF16 and INT8 input types.^[7] AMX-FP16 also adds support for real and complex FP16 numbers. The register file consists of 8 tiles, each with 16 rows of size of 64 bytes (32 BF16/FP16 or 64 INT8 elements). The only supported operation is matrix multiplication $C\_\; +=\; \backslash sum\_^J\; A\_B\_.$

4th Gen Intel Xeon Scalable processor can perform 2048 INT8 or 1024 BF16 operations per cycle:^{[8] [9]} the maximal input sizes are $16\; \backslash times\; J$ for and $J\; \backslash times\; 16$ for, where is 64 for INT8 and 32 BF16. The matrix multiplication requires $256J$ multiplication and $256J$ additions, thus performing $512J$ operations in 16 cycles.

Software support

• Compiler and assembler support
  • LLVM 13^{[10] [11] [12] [13]}
  • GCC 11^{[14] [15] [16]}
  • GNU Assembler (GAS) initial support committed at 25 June 2020^{[17] [13]}
• Operating system support
  • glibc support for detecting AMX feature in CPUs committed at 25 June 2020^[18]
  • Linux kernel support released in version 5.16^[19]
  • VMware vSphere support for AMX in Virtual Machines (VMs) released in ESXi version 8.0u1 for VMs using Hardware Version 20^[20]

External links

• Intel Intrinsics Guide
• Wikichip: Advanced Matrix Extension (AMX) - x86

Notes and References

1. Web site: With AMX, Intel Adds AI/ML Sparkle to Sapphire Rapids. Nicole. Hemsoth. August 19, 2021. The Next Platform.
2. Web site: Intel AMX: Erste Informationen zur Advanced Matrix Extensions Architecture. heise. online. heise online. 28 June 2020 .
3. Web site: Intel Xeon Sapphire Rapids: How To Go Monolithic with Tiles. Ian. Cutress. AnandTech.
4. Web site: Intel® Architecture Instruction Set Extensions and Future Features .
5. Web site: The x86 Advanced Matrix Extension (AMX) Brings Matrix Operations; To Debut with Sapphire Rapids. David. Schor. June 29, 2020.
6. Web site: Intel Granite Rapids D Support Merged Into GCC 14 . Michael . Larabel . . July 12, 2023.
7. Web site: Advanced Matrix Extension (AMX) - x86 - WikiChip. en.wikichip.org.
8. Web site: Accelerate Artificial Intelligence (AI) Workloads with Intel Advanced Matrix Extensions (Intel AMX) . 2023-04-13 . Intel.
9. Web site: Intel® 64 and IA-32 Architectures Optimization Reference Manual Volume 1 . Intel.
10. Web site: What's New in LLVM for 4th Gen Intel® Xeon® & Max Series CPUs . 21 April 2023 .
11. Web site: Intel AMX Support Begins Landing In LLVM. Larabel . Michael. 2020-07-02. Phoronix. en-US. 2020-07-02.
12. Web site: [X86-64] Support Intel AMX instructions ]. . 2020-07-02. en-US. 2020-07-02.
13. Web site: Intel AMX Support Lands In The GNU Assembler. Larabel. Michael. 2020-07-02. Phoronix. en-US. 2020-07-02.
14. Web site: GCC 11 Release Series — Changes, New Features, and Fixes - GNU Project . 21 April 2023 .
15. Web site: [PATCH] Enable GCC support for AMX]. 2020-07-06. en-US. 2020-07-09.
16. Web site: Enable GCC support for AMX-TILE,AMX-INT8,AMX-BF16. · gcc-mirror/gcc@5c60984 . 2022-09-05 . GitHub . en.
17. Web site: commits with Intel AMX. 2020-07-02. en-US. 2020-07-02.
18. Web site: x86: Detect Intel Advanced Matrix Extensions. 2020-07-02. en-US. 2020-07-02.
19. Web site: Linux 5.16 Features Include FUTEX2, Intel AMX, Folios, DG2/Alchemist, More Apple Silicon Support. Phoronix.
20. Web site: Accessing Sapphire Rapids AMX instructions on vSphere. 2023-08-24. en-US. Earl C. Ruby III.