Fetal electrocardiogram (FECG) extraction is very important issue for fetal health assessment. In this paper, we propose a fast one-unit independent component analysis with reference (ICA-R) that suits for to extract the FECG. Most the previous ICA-R algorithms only focused on how to optimize the cost function of ICA-R, and had little attention on the improvement of cost function. They didnt fully take advantage of the prior information about desired signal to improve the ICA-R. This paper firstly uses the kurtosis information of desired FECG signal to simply the non-Gaussian measurement function and then constructs a new cost function by directly using nonquadratic function of extracted signal to measure its non-Gaussian. The new cost function does not involve the computation of difference between the function of Gaussian random vector and that of extracted signal, which is time consuming. Centering and whitening are used as observed signal preprocessing to further more reduce the computation complexity. The proposed method has lower computation complexity than other improved one-unit ICA-R method, while they have the same error performance. Simulations are performed separately on artificial and real-world electrocardiogram signals.