A 3D finite element vibration model of water turbine generator set is constructed considering the coupling with hydropower house foundation.The method of determining guide bearing dynamic characteristic coefficients according to the swing of the shaft is proposed, which can be used for studying the self-vibration characteristic and stability of the water turbine generator set.The method fully considers the complex supporting boundary and loading conditions;especially the nonlinear variation of guide bearing dynamic characteristic coefficients and the coupling effect of the whole power-house foundation.The swing and critical rotating speed of an actual generator set shaft system are calculated.The simulated results of the generator set indicate that the coupling vibration model and calculation method presented in this paper are suitable for stability analysis of the water turbine generator set. A 3D finite element vibration model of water turbine generator set is constructed considering the coupling of hydropower house foundation. Method of determining guide bearing dynamic characteristic coefficients according to the swing of the shaft is proposed, which can be used for studying the self-vibration characteristic and stability of the water turbine generator set. The method fully considered the complex supporting boundary and loading conditions; especially the nonlinear variation of guide bearing dynamic characteristic coefficients and the coupling effect of the whole power-house foundation. The swing and critical rotating speed of an actual generator set shaft system are calculated. The simulated results of the generator set of that coupling vibration model and calculation method presented in this paper are suitable for stability analysis of the water turbine generator set.