Order analysis is one of the most important technique means of condition monitoring and fault diagnosis for rotary machinery. The traditional order analyses usually employ the Vold-Kalman filtering, however this method is confined to the expensive hardware equipments. This paper starts from Gabor transform and applies the Gabor time-frequency filtering to vibration signal. The order component’s time-frequency coefficients are extracted by mask operation. The order component is reconstructed from the obtained coefficients. The following four key technologies, such as smoothing rotary speed curve, defining filtering band width, constructing the mask operation matrix and reconstructing signal component, are also deeply discussed. Moreover, the technique to smooth the rotary speed curve based on polynomial approximation, the method to determine filtering band width, the arithmetic to constitute mask array and the iterative algorithm to reconstruct signal based on minimum mean square error are specifically analyzed. The 4th order component is successfully gained by using the methods that Gabor time-frequency filter, and the validity and feasibility of this method are approved. This method can solve the problem of order tracking filter technologies which used to depend on hardware and efficiently improve the accuracy of order analysis.