A double-layer harmonic diffractive element (HDE) structure is investigated and the optimization procedure is given based on the equation of diffraction efficiency of the dou ble-layer diffractive optical element. A new infrared double-layer inverted telephoto-optical system with an HDE is designed, which can work in the mid- and farinfrared wavebands. The diffraction efficiency of the system at each wavelength in the designed two wavebands is larger than 90%, which improves the image contrast and the imaging quality significantly.