Recently, ChatGPT has become the focus of attention from all walks of life. From a technical perspective, the boom of ChatGPT is the result of the evolution, development and breakthrough in the field of deep learning, representing the rapid progress of large model technology based on the Transformer architecture. Therefore, how to efficiently deploy Transformer on the edge and end sides has become a core consideration for users when choosing a platform.

In March 2023, Axera launched its third-generation high computing power and high energy efficiency ratio SoC chip —— AX650N. Relying on its outstanding performance in high performance, high precision, easy deployment, low power consumption and other aspects, AX650N has been favored by an increasing number of users with large model deployment needs, and has taken the lead to become a landing platform for Transformer on the edge and end sides.

Transformer is the main architecture adopted by various current large models, and the popularity of ChatGPT has made people gradually realize that artificial intelligence has a higher upper limit and can exert huge potential in the field of computer vision. Compared with deploying large Transformer models on GPU in the cloud, the biggest challenge of deploying Transformer on the edge and end sides comes from power consumption. This also makes Axera's mixed-precision NPU with both high performance and low power consumption characteristics become the preferred platform for deploying Transformer on the edge and end sides, and its superior performance determines the operation effect of Transformer.

As a company engaged in the research and development of artificial intelligence visual perception chips and the construction of basic computing power platforms, Axera has always been committed to the implementation of more practical applications. The currently widely adopted Transformer network SwinT delivers an outstanding performance on Axera's AX650N platform: the high performance of 361 FPS, high precision of 80.45%, low power consumption of 199 FPS/W, and the extremely easy deployment capability with the original model and PTQ quantization, all make AX650N take a leading position in the implementation of Transformer.

Specifically, the high performance of 361 FPS is comparable to that of high-end domain controller SoC based on GPU in the field of automotive autonomous driving; the high precision of 80.45% is also higher than the market average; the speed of 199 FPS/W fully reflects the characteristics of low power consumption, with a several-fold advantage compared with the current high-end domain controller SoC based on GPU; more importantly, AX650N features convenient deployment —— the original models on GitHub can run efficiently on Axera's platform without modifying the models or retraining with QAT. At the same time, AX650N supports low-bit mixed precision. If users adopt INT4, they can greatly reduce the memory and bandwidth occupancy rate, achieving the goal of effectively controlling the deployment costs on the edge and end sides. All these features ensure that as an artificial intelligence computing power platform, AX650N delivers better and more user-friendly practical application effects, and greatly improves user efficiency.

At present, AX650N has been adapted to Transformer models including ViT/DeiT, Swin/SwinV2 and DETR, and has achieved an operation speed of more than 30 FPS on DINOv2, which also makes it more convenient for users to perform downstream operations such as detection, classification and segmentation. Products based on AX650N have currently played an important role in the core fields of computer vision such as smart city, smart education and intelligent manufacturing.

Going forward, Axera's AX650N will continue to optimize for the Transformer architecture and explore more large Transformer models, such as multimodal large models, to continuously achieve better implementation effects of Transformer on Axera's platform. Notably, Axera will also launch corresponding development boards to meet developers' needs for in-depth research on Transformer and explore richer product applications.

"Axera will continue to strive to build an edge and end-side artificial intelligence computing power platform based on chip + software, bring intelligence to real-life scenarios, and ultimately realize the corporate vision of inclusive AI creating a better life," said Ms. Qiu Xiaoxin, Founder and CEO of Axera. In the future, Axera will continue to explore the path of becoming an artificial intelligence computing power platform company, accelerate the implementation of large Transformer-based models on the edge and end sides, and make inclusive intelligence truly take root and bear fruit.