This paper studies the roughness effect combining with effects of rarefaction and compressibility by a lattice Boltzmann model for rarefied gas flows at high Knudsen numbers. By discussing the effect of the tangential momentum accommodation coefficient on the rough boundary condition, the lattice Boltzmann simulations of nitrogen and helium flows are performed in a two-dimensional microchannel with rough boundaries. The surface roughness effects in the microchannel on the velocity field, the mass flow rate and the friction coefficient are studied and analysed. Numerical results for the two gases in micro scale show different characteristics from macroscopic flows and demonstrate the feasibility of the lattice Boltzmann model in rarefied gas dynamics.