Temporally and Spatially Modulated Fourier Transform Imaging Spectrometer(TSMFTIS)is a new imaging spectrometer without moving mirrors and slits.As applied in remote sensing,TSMFTIS needs to rely on push-broom of the flying platform to obtain the interferogram of the target detected,and if the moving state of the flying platform changed during the imaging process,the target interferogram picked up from the remote sensing image sequence will deviate from the ideal interferogram,then the target spectrum recovered shall not reflect the real characteristic of the ground target object.Therefore,in order to achieve a high precision spectrum recovery of the target detected,the geometry position of the target point on the TSMFTIS image surface can be calculated in accordance with the sub-pixel image registration method,and the real point interferogram of the target can be obtained with image interpolation method.The core idea of the interpolation methods(nearest,bilinear and cubic etc)are to obtain the grey value of the point to be interpolated by weighting the grey value of the pixel around and with the kernel function constructed by the distance between the pixel around and the point to be interpolated.This paper adopts the gauss-based kernel regression mode,present a kernel function that consists of the grey information making use of the relative deviation and the distance information,then the kernel function is controlled by the deviation degree between the grey value of the pixel around and the means value so as to adjust weights self-adaptively.The simulation adopts the partial spectrum data obtained by the pushbroom hyperspectral imager(PHI)as the spectrum of the target,obtains the successively push-broomed motion error image in combination with the related parameter of the actual aviation platform; then obtains the interferogram of the target point with the above interpolation method; finally,recovers spectrogram with the nonuniform fast Fourier transform algorithm.Compared with the accurate spectrogram,the spectrogram recovered with the relative deviation-based kernel regression interpolation method has remarkable improvement over the previous methods.