Exhausted T cells are a group of dysfunctional T cells, which are present in chronic infections or tumors. The most significant characteristics of exhausted T cells are attenuated effector cytotoxicity, reduced cytokine production, and upregulation of multiple inhibitory molecular receptors (e.g., PD-1, TIM-3, and LAG-3). The intracellular metabolic changes, altered expression of transcription factors, and a unique epigenetic landscape constitute the exhaustion program. Recently, researchers have made progress in understanding exhausted T cells, with the definition and identification of exhausted T cells changing from phenotype-based to being classified at the transcriptional and epigenetic levels. Recent studies have revealed that exhausted T cells can be separated into two subgroups, namely TCF1+PD-1+ progenitor-like precursor exhausted cells and TCF1-PD-1+ terminally differentiated exhausted T cells. Moreover, the progenitor-like precursor cell population may be a subset of T cells that can respond to immunotherapy. Studies have also found that TOX initiates and dominates the development of exhausted T cells at the transcriptional and epigenetic levels. TOX also maintains T cell survival and may affect decisions regarding treatment strategies. In this review, we discuss the latest developments in T cell exhaustion in regards to definitions, subpopulations, development mechanisms, differences in diverse diseases, and treatment prospects for exhausted T cells. Furthermore, we hypothesize that the epigenetic state regulated by TOX might be the key point, which can determine the reversibility of exhaustion and the efficacy of immunotherapy.