To support dramatically increased traffic loads, communication networks become ultra-dense. Traditional cell association (CA) schemes are time-consuming, forcing researchers to seek fast schemes. This paper proposes a deep Q-learning based scheme, whose main idea is to train a deep neural network (DNN) to calculate the Q values of all the state-action pairs and the cell holding the maximum Q value is as-sociated. In the training stage, the intelligent agent continuously generates samples through the trial-and-error method to train the DNN until convergence. In the application stage, state vectors of all the users are inputted to the trained DNN to quickly obtain a sat-isfied CA result of a scenario with the same BS loca-tions and user distribution. Simulations demonstrate that the proposed scheme provides satisfied CA results in a computational time several orders of magnitudes shorter than traditional schemes. Meanwhile, perfor-mance metrics, such as capacity and fairness, can be guaranteed.