Complex-valued neural networks（CVNNs） have shown their excellent efficiency compared to their real counterparts in speech enhancement, image and signal processing.Researchers throughout the years have made many efforts to improve the learning algorithms and activation functions of CVNNs. Since CVNNs have proven to have better performance in handling the naturally complex-valued data and signals, this area of study will grow and expect the arrival of some effective improvements in the future. Therefore, there exists an obvious reason to provide a comprehensive survey paper that systematically collects and categorizes the advancement of CVNNs. In this paper, we discuss and summarize the recent advances based on their learning algorithms, activation functions, which is the most challenging part of building a CVNN, and applications. Besides,we outline the structure and applications of complex-valued convolutional, residual and recurrent neural networks. Finally,we also present some challenges and future research directions to facilitate the exploration of the ability of CVNNs.