The classical version of Mandelstam-Tamm speed limit based on the Wigner function in phase space was reported by Shanahan et al. [Phys. Rev. Lett. 120070401 (2018)]. We present the Margolus-Levitin speed limit across the quantum-to-classical transition in phase space based on the trace distance. The Margolus-Levitin speed limit is set by the Schatten L1 norm of the generator of time-dependent evolution for both the quantum and classical domains. As an example, the time-dependent harmonic oscillator is considered to illustrate the result.