The application of machine learning to the field of ultrafast photonics is becoming more and more extensive. In this paper, for the automatic mode-locked operation in a saturable absorber-based ultrafast fiber laser (UFL), a deep-reinforcement learning algorithm with low latency is proposed and implemented. The algorithm contains two actor neural networks providing strategies to modify the intracavity lasing polarization state and two critic neural networks evaluating the effect of the actor networks. With this algorithm, a stable fundamental mode-locked (FML) state of the UFL is demonstrated. To guarantee its effectiveness and robustness, two experiments are put forward. As for effectiveness, one experiment verifies the performance of the trained network model by applying it to recover the mode-locked state with environmental vibrations, which mimics the condition that the UFL loses the mode-locked state quickly. As for robustness, the other experiment, at first, builds a database with UFL at different temperatures. It then trains the model and tests its performance. The results show that the average mode-locked recovery time of the trained network model is 1.948 s. As far as we know, it is 62.8% of the fastest average mode-locked recovery time in the existing work. At different temperatures, the trained network model can also recover the mode-locked state of the UFL in a short time. Remote algorithm training and automatic mode-locked control are proved in this work, laying the foundation for long-distance maintenance and centralized control of UFLs.