For the Boltzmann equation with an external force in the form of the gradient of a potential function in space variable, the stability of its stationary solutions as local Maxwellians was studied by S. Ukai et al. (2005) through the energy method. Based on this stability analysis and some techniques on analyzing the convergence rates to stationary solutions for the compressible Navier-Stokes equations, in this paper, we study the convergence rate to the above stationary solutions for the Boltzmann equation which is a fundamental equation in statistical physics for non-equilibrium rarefied gas. By combining the dissipation from the viscosity and heat conductivity on the fluid components and the dissipation on the non-fluid component through the celebrated H-theorem, a convergence rate of the same order as the one for the compressible Navier-Stokes is obtained by constructing some energy functionals.