The use of vehicle- or air-borne Ground Penetrating Synthetic Aperture Radar (GPSAR) to quickly detect landmines over large areas is becoming a trend. However, producing too many false alarms in GPSAR landmine detection is a major challenge in practical applications of GPSAR. Support Vector Machine (SVM), employing structural risk minimization theory, does not need large amounts of training data, which makes it suitable for solving the landmine detection problem. In this paper, a novel SVM with a hypersphere instead of a hyperplane classification boundary is proposed for landmine detection in GPSAR. The HyperSphere-SVM (HS-SVM) can be trained with both landmine and clutter data, or with landmine data only, which are called the two-class HS-SVM and the one-class HS-SVM, respectively. The HS-SVM has better generalization capability than the traditional HyperPlane-SVM (HP-SVM) with respect to varying operating conditions. Quantitative comparisons have been made using real data collected with the rail-GPSAR landmine detection system, which show that both the two-class and the one-class HS-SVMs have better detection performance than the HP-SVM.