In this paper, we first present a statistics analysis of the picture distortion caused by quantization, channel errors and subsequent error propagation, and propose rate-distortion (R-D) estimation models which are functions oftwo parameters, i.e. quantization step and intra refreshing ratio. Second, based on our proposed R-D models, a joint optimal quantization step and intra refreshing ratio selection algorithm is developed by using Lagrange multiplier theory. Next, a novel rate control algorithm, combined with macroblock (MB) coding mode selection, is proposed. The algorithm focuses on bit allocation at group of pictures (GOP) level and picture level to improve the encoding quality, and the MB mode selection addresses improving the local subjective quality within a picture.Finally, extensive simulation results show that our algorithm can make the encoder adapting itself in time to the variation of the channel bandwidth and significantly improve the end-to-end video quality over packet loss network.