The numerical method of modes analysis and decomposition of the output signal in 3D electromagnetic particle-in-cell simulation is presented. By the method, multiple modes can be resolved at one time using a set of diagnostic data, the amplitudes and the phases of the specified modes can all be given separately. Based on the method, the output signals of one X-band tri-bend mode converter used for one high power microwave device, with ionization process in the device due to the strong normal electric field, are analyzed and decomposed.