This paper proposed a novel feature selection method LUIFS (latent utility of irrelevant feature selection) that not only selects the relevant features, but also targets at discovering the latent useful irrelevant attributes by measuring their supportive importance to other attributes. The method minimizes the information lost and simultaneously maximizes the final classification accuracy. The classification error rates of the LUIFS method on 16 real-life datasets from UCI machine learning repository were evaluated using the ID3, Nave-Bayes, and IB (instance-based classifier) learning algorithms, respectively; and compared with those of the same algorithms with no feature selection (NoFS), feature subset selection (FSS), and correlation-based feature selection (CFS). The empirical results demonstrate that the LUIFS can improve the performance of learning algorithms by taking the latent relevance for irrelevant attributes into consideration, and hence including those potentially important attributes into the optimal feature subset for classification.