Through the analysis and comparison of shortcomings and advantages of existing technologies on object modeling in 3D applications, we propose a new modeling method for virtual scene based on multi-view image sequence to model irregular objects efficiently in 3D application. In 3D scene, this method can get better visual effect by tracking the viewers real-time perspective position and projecting the photos from different perspectives dynamically. The philosophy of design, the steps of development and some other relevant topics are discussed in details, and the validity of the algorithm is analyzed. The results demonstrate that this method represents more superiority on simulating irregular objects by applying it to the modeling of virtual museum.