High frequency (HF) communication is widely spread due to some merits like easy deployment and wide communication coverage. Spectrum prediction is a promising technique to facilitate the working frequency selection and enhance the function of automatic link establishment. Most of the existing spectrum prediction algorithms focus on predicting spectrum values in a slot-by-slot manner and therefore are lack of timeliness. Deep leing based spectrum prediction is developed in this paper by simultaneously predicting multi-slot ahead states of multiple spectrum points within a period of time. Specifically, we first employ supervised leing and construct samples depending on longterm and short-term HF spectrum data. Then, advanced residual units are introduced to build multiple residual network modules to respectively capture characteristics in these data with diverse time scales. Further, convolution neural network fuses the outputs of residual network modules above for temporal-spectral prediction, which is combined with residual network modules to construct the deep temporal- spectral residual network. Experiments have demonstrated that the approach proposed in this paper has a significant advantage over the benchmark schemes.