A polarization diversity receiver scheme is presented for improving efficiency of balance homodyne detection.The proposed scheme may mitigate polarization fluctuation between signal and local oscillator field. With simple linear optical component and electronic processing circuit, the noise caused by differential phase and polarization mode between signal and local oscillators may be significantly decreased. To track the polarization fluctuation, novel algorithm based on polarization diversity receiver which can achieve better performance in terms of linear quantum optics principle is proposed.