Enhancing Image Classification Capabilities in the Vision Transformer Network Model with Quaternion Algebra

Abstract

Abstract. Vision Transformer is a novel approach in artificial intelligence, focusing on image classification. Despite its potential, ViT's emphasis on global data processing presents accuracy challenges compared to local data processing methods like Convolutional Neural Networks (CNN). To address this, we propose two methods. The first integrates a portion of the Residual Network to replace token transformation layers, allowing for local data feature extraction and improved relationship learning between tokens. The second solution suggests transforming layers in the bottleneck component into types that process in the Quaternion hypercomplex domain, enhancing the multidimensional representation of data. Both solutions aim to leverage the strengths of CNN and ViT, thereby indirectly improving image classification accuracy.