The nucleation process of stick-slip instability was analyzed based on the experimental measurements of strain and fault slip on homogeneous and non-homogeneous faults. The results show that the nucleation process of stick-slip on the homogeneous fault is of weak slip-weakening behavior under constant loading point velocity. The existence of a short "weak segment" on the fault makes slip-weakening phenomenon in nucleation process more obvious, while the existence of a long "weak segment" on the fault makes the nucleation process changed. The nucleation is characterized by accelerating slip in a local region and rapid increase of shear stress along the fault in this case, which is more coincident with the rate and state friction law. During the period when fault is locked, increasing of shear stress causes lateral elastic dilation near the fault, and the rebound of the dilation at the time of instability causes an instantaneous increase of normal stress in the fault plane, which is an important factor making fault be rapidly locked and its strength recovered.