We study the relationship between the properties of the isovector giant dipole resonance of finite nuclei and the symmetry energy in the framework of the relativistic mean field theory with six different parameter sets of nonlinear effective Lagrangian.A strong linear correlation of excited energies of the dipole resonance in finite nuclei and symmetry energy at and below the saturation density is found.This linear correlation leads to the symmetry energy at the saturation density at the interval 33.0 MeV ≤ S(ρo)≤37.0 MeV.The comparison to the present experimental data in the soft dipole mode of 132 Sn constrains approximately the symmetry energy at ρ = 0.1 fm-3 at the interval 21.2MeV～ 22.5 MeV.It is proposed that a precise measurement of the soft dipole mode in neutron rich nuclei could set up an important constraint on the equation of state for asymmetric nuclear matter.