This work focuses on the production of a new composite material using Yellow River sediment and coal slime ash via alkali-activating method. XRD, FTIR and SEM/EDS were used to characterize the alkali-activated products and microstructure of the composite material. Compressive strength was tested to characterize the mechanical property of the composite material. It is found that the compressive strength of the Yellow River sediment-coal slime ash composites increases as the added Ca(OH)2 content grows. The compressive strength increases fast in the early stage but slowly after 28 days. The strength of the composites can be significantly improved via the addition of small amount of NaOH and gypsum. The products (C-S-H, ettringite and CaCO3), especially C-S-H, make much contribution to the enhancement of strength. The highest strength of the composites can reach 14.4 MPa after 90 days curing with 5% Ca(OH)2, 0.2% NaOH and 7.5%gypsum. The improved properties of the composites show great potential of utilizing Yellow River sediment for inexpensive construction materials.