Weather classification is getting more and more attractive because it has many potential applications,such as visual systems and intelligent transportation,especially in transportation.However,the researches about it were based weather dataset are designed to be executed in clear weather conditions.And it is challenging tasks due to the diversity of weather cues and lack of discriminate feature.Therefore,in addition to the image features,real-time weather factors is needed to provide additional information.Most existing weather classification methods involve many pre-processing techniques,for instance sky detection and boundary detection,which makes the model highly rely on the performance of the aforementioned steps.Therefore,in this paper,we propose a novel method for multi-class weather classification.We newly constructed multi-class weather dataset contains 5K images which has more adverse weather conditions.To improve the discrimination of image representation and decrease the influence of pre-processing,our approach combines the real-time weather data with the image feature as the final feature vector to identify different weather.With experiments on our own dataset,we demonstrate that the proposed framework achieves superior performance compared to the state-of-the-art methods.