Nanoindentation measurement was performed to evaluate the local hardness distribution for quenched and tempered martensitic steel with ferrite-cementite constituent. The ratio of standard deviation to the average nanohardness was 15.4% when nanoindentation tests were carried out at the peak load of 1000 μN, while that of the Vickers hardness was only 1.5% at 9.8 N. Electron backscatter diffraction analysis and SEM observation all showed that the large scattering of nanohardness did not depend on the crystallographic orientation of each grain but from the inhomogeneous microstructure in the sub-micron scale such as cementite distribution. Compared with the results on tungsten single crystals with different surface orientations as well as another martensite processed by modified-ausforming with the same chemical composition but more homogeneous cementite distribution, the nanohardness showed smaller scattering, supporting the conclusion.