Software vulnerability mining is an important way to detect whether there are some loopholes existing in the software, and also is an important way to ensure the secu-rity of information systems. With the rapid development of information technology and software industry, most of the software has not been rigorously tested before being put in use, so that the hidden vulnerabilities in software will be exploited by the attackers. Therefore, it is of great significance for us to actively de-tect the software vulnerabilities in the security maintenance of information systems. In this paper, we firstly studied some of the common-ly used vulnerability detection methods and detection tools, and analyzed the advantages and disadvantages of each method in different scenarios. Secondly, we designed a set of eval-uation criteria for different mining methods in the loopholes evaluation. Thirdly, we also proposed and designed an integration testing framework, on which we can test the typical static analysis methods and dynamic mining methods as well as make the comparison, so that we can obtain an intuitive comparative analysis for the experimental results. Final-ly, we reported the experimental analysis to verify the feasibility and effectiveness of the proposed evaluation method and the testing framework, with the results showing that the final test results will serve as a form of guid-ance to aid the selection of the most appropri-ate and effective method or tools in vulnera-bility detection activity.