Physically-based modeling is used in virtual assembly system to improve the feeling of immersion and make the evaluation results convinced. The handling of unilateral constraints is the capstone of modeling physical behaviors. To take advantage of both impulse-based method and analytical method, hybrid scheme for adaptive simulation is presented, where the constraint states can be transited automatically. Collision is solved using impulse-based method while contact with friction is handled by formulating to linear complementarity problem. To make these methods efferent, physical models are reorganized according to the assembly hierarchy. The scheme and methods were implemented in the prototype system Virtual Assembly Process Planning (VAPP). It proved the algorithms can meet the requirement of rigid body simulation in virtual assembly system.