Advanced technologies are required in future mobile wireless networks to support services with highly diverse requirements in terms of high data rate and reliability,low latency,and massive access.Deep Learning (DL),one of the most exciting developments in machine learning and big data,has recently shown great potential in the study of wireless communications.In this article,we provide a literature review on the applications of DL in the physical layer.First,we analyze the limitations of existing signal processing techniques in terms of model ac-curacy,global optimality,and computational scalability.Next,we provide a brief review of classical DL frame-works.Subsequently,we discuss recent DL-based physical layer technologies,including both DL-based signal processing modules and end-to-end systems.Deep neural networks are used to replace a single or several con-ventional functional modules,whereas the objective of the latter is to replace the entire transceiver structure.Lastly,we discuss the open issues and research directions of the DL-based physical layer in terms of model complexity,data quality,data representation,and algorithm reliability.