With a continuously increasing population and better food consumption levels, improving the efficiency of arable land use and increasing its productivity have become fundamental strategies to meet the growing food security needs in China. A spatial distribution map of medium- and low-yield cropland is necessary to implement plans for cropland improvement. In this study, we developed a new method to identify high-, medium-, and low-yield cropland from Moderate Resolution Imaging Spectroradiometer(MODIS) data at a spatial resolution of 500 m. The method could be used to reflect the regional heterogeneity of cropland productivity because the classification standard was based on the regionalization of cropping systems in China. The results showed that the proportion of high-, medium-, and low-yield cropland in China was 21%, 39%, and 40%, respectively. About 75% of the low-yield cropland was located in hilly and mountainous areas, and about 53% of the high-yield cropland was located in plain areas. The five provinces with the largest area of high-yield cropland were all located in the Huang-Huai-Hai region, and the area amounted to 42% of the national high-yield cropland area. Meanwhile, the proportion of high-yield cropland was lower than 15% in Heilongjiang, Sichuan, and Inner Mongolia, which had the largest area allocated to cropland in China. If all the medium-yield cropland could be improved to the productive level of high-yield cropland and the low-yield cropland could be improved to the level of medium-yield cropland, the total productivity of the land would increase 19% and 24%, respectively.