Silicon photonics is a promising technology to address the demand for dense and integrated next-generation optical interconnections due to its complementary-metal-oxide-semiconductor (CMOS) compatibility. However, one of the key building blocks, the silicon modulator, suffers from several drawbacks, including a limited bandwidth, a relatively large footprint, and high power consumption. The graphene-based silicon modulator, which benefits from the excellent optical properties of the two-dimensional graphene material with its unique band structure, has significantly advanced the above critical figures of merit. In this work, we review the state-of-the-art graphene-based silicon modulators operating in various mechanisms, i.e., thermal-optical, electro-optical, and plasmonic. It is shown that graphene-based silicon modulators possess the potential to have satisfactory characteristics in intra- and inter-chip connections.