Micro-Kernels for Portable and Efficient Matrix Multiplication in Deep Learning

Abstract We demonstrate that it is possible to rapidly assemble an amplevariety of high performance micro-kernels for the general matrix multiplica-tion (gemm) using vector intrinsics to exploit the SIMD (single instruction,multiple data) units in current general-purpose processors. For the particulartype of applications arising in deep learning, our experiments expose that theintrinsics-based micro-kernels can deliver efficiency on par with or even higherthan the conventional, carefully tuned implementations of gemm in currentlinear algebra libraries for ARM-based processors equipped with 128-bit SIMDunits and, to a lower extent, in processors with 512-bit SIMD units.