将连续状态空间下的Q学习构建为最小二乘支持向量机的回归估计问题,利用最小二乘支持向量机良好的泛化以及非线性逼近性能实现由系统状态-动作对到Q值函数的映射。为了保证计算速度以及适应Q学习系统在线学习的需要,最小二乘支持向量机的训练样本是窗式移动的,即在Q学习系统学习的同时获取样本数据并进行最小二乘支持向量机的训练。小车爬山控制问题的仿真结果表明该方法学习效率高,能够有效解决强化学习系统连续状态空间的泛化问题。 The Q learning under continuous state space is constructed as the regression estimation problem of least square support vector machine, and the mapping of system state-action to Q-value function is achieved by the good generalization and nonlinear approximation performance of LS-SVM . In order to ensure the computing speed and meet the need of online learning of Q learning system, the training samples of least square support vector machine are windowed, that is to obtain the sample data while learning the Q learning system and train the least square support vector machine . Simulation results show that this method has high learning efficiency and can effectively solve the problem of generalized continuous state space of reinforcement learning system.