In order to solve the combinative explosion problems in a continuous and high dimensional state space, a function approximation approach is usually used to represent the state space. The normalized radial basis function (NRBF) was adopted as the local function approximator and a kind of adaptive state space construction strategy based on the NRBF (ASC-NRBF) was proposed, which enables the system to allocate appropriate number and size of the basis functions automatically. Combined with the reinforcement learning method, the proposed ASC-NRBF method was applied to the robot navigation problem. Simulation results illustrate the performance of the proposed method.