Benchmark Assessment for Optimization Andacceleration Libraries in Deep Learning-Based Models

Deep Learning (DL) models are widely used in machine learning due to their performance and abilityto deal with large datasets while producing high accuracy and performance metrics. The size ofsuch datasets and the complexity of DL models cause such models to be complex, consuming largeamount of resources and time to train. Many recent libraries and applications are introduced todeal with DL complexity and efficiency issues. Neural network-based models have utilized stateof the art optimization and acceleration libraries to scale up the size of training datasets and thenumber of parameters in the models themselves. In this paper, we evaluated one example, MicrosoftDeepSpeed library through classification tasks. DeepSpeed public sources reported classificationperformance metrics on the LeNet architecture. We extended this through evaluating the libraryon several modern neural network architectures, including convolutional neural networks (CNNs)and Vision Transformer (ViT). Results indicated that neural network models for imaging tasks areimproved through using the DeepSpeed library in general.