In order to investigate the influence of complex conditions of in-situ surrounding rocks on the settlement behavior of nubbly coal mine waste subjected to high gravity pressure, four kinds of loading chambers made of different similar materials with different elastic moduli in experiments were used to simulate the deformation features of in-site rocks, including soft, moderate hardness, hard and extra-hard rocks. The results show that all the settlement-axial load (or axial strain-stress) curves obtained under four different surrounding rock conditions present power-exponential function feature. The final settlement of coal mine waste under the same axial load is closely related to the lumpiness gradations and the deformation behavior of chamber materials used to simulate behaviors of different in-situ surrounding rocks. In the same surrounding rock condition, the final settlement under the same maximum axial load decreases with the decrease of the proportion of larger gradation of coal mine waste. While for the same lumpiness gradation case, the settlement increases with the decrease of elastic modulus of simulated surrounding rocks and the lateral pressure induced by axial load increases with the increase of elastic modulus of loading chambers that are used to simulate different surrounding rocks. The test results also reveal that both the compaction curve and lateral pressure curve show a three-stage behavior, and the duration of each stage, which is closely related to gradations and the deformation feature of loading chamber materials, decreases with the increase of the proportion of the small size of coal mine waste and elastic modulus of the simulated rock materials.