A Crank-Nicolson scheme based on nonconforming finite element with moving grids is discussed for a class of parabolic integro-differential equations under anisotropic meshes.The corresponding convergence analysis is presented and the error estimates are obtained by using the interpolation operator instead of the conventional elliptic projection which is an indispensable tool in the convergence analysis of traditional finite element methods in previous literature.