With the further development of industry, strip steel occupies an important position in industrial production and is widely used in various manufacturing fields. It is especially important to monitor the quality of strip steel production. In order to improve the detection rate of defective strip steel for its complex and varied surface defects and other characteristics, this paper proposes a defect classification algorithm based on the SFN-VIT (Improved Shuffle Network Unite Vision Transformer) model to classify six types of defects in strip steel and compare it with other classification algorithms based on convolutional neural network. The experimental data show that the proposed SFN-VIT model outperforms the traditional machine learning algorithm model and achieves an average accuracy of 91.7% for defect classification on the NEU-CLS dataset (Tohoku University strip steel surface defect categories dataset), which is a 5.1% improvement compared to the traditional classification algorithm.