In this work, we combined the model based rein-forcement leing (MBRL) and model free reinforcement le-ing (MFRL) to stabilize a biped robot (NAO robot) on a rotating platform, where the angular velocity of the platform is unknown for the proposed leing algorithm and treated as the extal disturbance. Nonparametric Gaussian processes normally re-quire a large number of training data points to deal with the dis-continuity of the estimated model. Although some improved method such as probabilistic inference for leing control (PILCO) does not require an explicit global model as the actions are obtained by directly searching the policy space, the overfit-ting and lack of model complexity may still result in a large devi-ation between the prediction and the real system. Besides, none of these approaches consider the data error and measurement noise during the training process and test process, respectively. We propose a hierarchical Gaussian processes (GP) models, contain-ing two layers of independent GPs, where the physically continu-ous probability transition model of the robot is obtained. Due to the physically continuous estimation, the algorithm overcomes the overfitting problem with a guaranteed model complexity, and the number of training data is also reduced. The policy for any given initial state is generated automatically by minimizing the expec-ted cost according to the predefined cost function and the ob-tained probability distribution of the state. Furthermore, a novel Q(λ) based MFRL method scheme is employed to improve the policy. Simulation results show that the proposed RL algorithm is able to balance NAO robot on a rotating platform, and it is cap-able of adapting to the platform with varying angular velocity.