In the field of artificial intelligence and machine learning method, people always need a scheme for feature extraction on the high order tensor data reserving more information in image retrieval, object recognition, etc. The reserved information keeping the spatial relationships in different modes when extracting the feature of the data can make the classification more accurate and efficient. Traditional feature extraction on the high order tensor data must be previously expanded into high dimensional vectors such as Principle Component Analysis and Fisher Discriminant Analysis, causing the loss of spatial relation information residing in original high order tensor data. We propose a classification model combining High-Order Principal Component Analysis (HOPCA) and Small Kernel Bilateral Two-Dimensional Fisher Discriminant Analysis (SKB2DFDA) which transformed the feature extraction through HOPCA into the new feature extraction that annihilates some drawback of traditional feature extraction, reduces the computational time, and increases our classification model accuracy.