EdgeQ-GEMM: A Single-Engine Bit-Partitioned GEMM Accelerator With Integrated On-the-Fly Quantization for Edge AI Processors

  • Jung, Chaebin; 
  • Lee, Kyeongwon; 
  • Park, Sechan; 
  • Choi, Jongin; 
  • Lee, Jae-Jin; 
  • ... Lee, Woojoo
Citations

WEB OF SCIENCE

0
Citations

SCOPUS

0

초록

Continuous-sensor edge AI workloads, such as keyword spotting (KWS) and human-activity recognition, process streaming inputs under tight power and area constraints. Many integrated on-the-fly (OTF) edge accelerators couple high-precision activation conversion to a fixed-precision GEMM engine, whereas many precision-flexible accelerators operate on operands already represented in low-precision integer formats. EdgeQ-GEMM combines these two design domains by integrating FP32/INT32-to-A8 activation conversion with a precision-flexible, bit-partitioned GEMM datapath. Within the tile-streaming datapath, the GEMM engine directly consumes either the full 8-bit activation code or its MSB-derived 4-bit slice, while the dequantization path receives the associated scale and compensation metadata without software-side quantization preprocessing or intermediate quantized-activation SRAM materialization. A single fixed-throughput engine thereby supports W8 & times; A8, MSB-derived W8 & times; A4-slice, and W4 & times; A8 execution without duplicated precision-specific GEMM engines or widened external interfaces. The MSB-derived A4-slice mode uses the scale and offset/compensation parameters associated with the A8 code and is not an independently quantized INT4 activation mode. The same architecture supports an adaptive streaming mode that selects between A8 and the MSB-derived A4 slice, as well as a layerwise mixed-precision quantization (MPQ) mode using W4/W8 weights and OTF-quantized A8 activations. Implemented in a Rocket-based edge processor and evaluated through FPGA prototyping and 45-nm synthesis, EdgeQ-GEMM reduces OTF-GEMM accelerator energy by 14.8%-27.0% at the energy-oriented adaptive setting. At the accuracy-oriented setting, the accuracy loss remains at or below 1.0 percentage point across the evaluated KWS and human-activity recognition (HAR) workloads. For ResNet-50/ImageNet layerwise MPQ, it reduces OTF-GEMM accelerator energy by up to 11.4% relative to a Dual-OTF-GEMM baseline while preserving the accuracy of the same MPQ assignment

키워드

Quantization (signal); Modeling; Engines; Tiles; Energy; Timing; Accuracy; Codes; Program processors; Design methodology; Bit-partitioned datapath; edge AI processor; low-power DNN acceleration; mixed-precision GEMM accelerator; MP-GEMM accelerator; on-the-fly quantization; OTF quantization
제목
EdgeQ-GEMM: A Single-Engine Bit-Partitioned GEMM Accelerator With Integrated On-the-Fly Quantization for Edge AI Processors
저자
Jung, Chaebin; Lee, Kyeongwon; Park, Sechan; Choi, Jongin; Lee, Jae-Jin; Lee, Woojoo
DOI
10.1109/TVLSI.2026.3735327
발행일
2026
유형
Article; Early Access
저널명
IEEE Transactions on Very Large Scale Integration (VLSI) Systems