A deep-learning-based method, called ConvLSTMP3, is developed to predict the sea surface heights (SSHs). ConvLSTMP3 is data-driven by treating the SSH prediction problem as the one of extracting the spatial-temporal features of SSHs, in which the spatial features are “learned” by convolutional operations while the temporal features are tracked by long short term memory (LSTM). Trained by a reanalysis dataset of the South China Sea (SCS), ConvLSTMP3 is applied to the SSH prediction in a region of the SCS east off Vietnam coast featured with eddied and offshore currents in summer. Experimental results show that ConvLSTMP3 achieves a good prediction skill with a mean RMSE of 0.057 m and accuracy of 93.4％ averaged over a 15-d prediction period. In particular, ConvLSTMP3 shows a better performance in predicting the temporal evolution of mesoscale eddies in the region than a full-dynamics ocean model. Given the much less computation in the prediction required by ConvLSTMP3, our study suggests that the deep learning technique is very useful and effective in the SSH prediction, and could be an alternative way in the operational prediction for ocean environments in the future.