Underwater acoustic applications depend critically on the prediction of sound propagation, which can be significantly affected by a rough surface, especially in shallow water. This paper aims to investigate how randomly fluctuating surface influences transmission loss (TL) in shallow water. The one-dimension wind-wave spectrum, Monterey–Miami parabolic equation (MMPE) model, Monte Carlo method, and parallel computing technology are combined to investigate the effects of different sea states on sound propagation. It is shown that TL distribution properties are related to the wind speed, frequency, range, and sound speed profile. In a homogenous waveguide, with wind speed increasing, the TLs are greater and more dispersive. For a negative thermocline waveguide, when the source is above the thermocline and the receiver is below that, the effects of the rough surface are the same and more significant. When the source and receiver are both below the thermocline, the TL distributions are nearly the same for different wind speeds. The mechanism of the different TL distribution properties in the thermocline environment is explained by using ray theory. In conclusion, the statistical characteristics of TL are affected by the relative roughness of the surface, the interaction strength of the sound field with the surface, and the changes of propagating angle due to refraction.