Background:Structured reports are not widely used and thus most reports exist in the form of free text.The process of data extraction by experts is time-consuming and error-prone,whereas data extraction by natural language processing (NLP) is a potential solution that could improve diagnosis efficiency and accuracy.The purpose of this study was to evaluate an NLP program that determines American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) descriptors and final assessment categories from breast magnetic resonance imaging (MRI) reports.Methods:This cross-sectional study involved 2330 breast MRI reports in the electronic medical record from 2009 to 2017.We used 1635 reports for the creation of a revised BI-RADS MRI lexicon and synonyms lists as well as the iterative development of an NLP system.The remaining 695 reports that were not used for developing the system were used as an independent test set for the final evaluation of the NLP system.The recall and precision of an NLP algorithm to detect the revised BI-RADS MRI descriptors and BI-RADS categories from the free-text reports were evaluated against a standard reference of manual human review.Results:There was a high level of agreement between two manual reviewers,with a κ value of 0.95.For all breast imaging reports,the NLP algorithm demonstrated a recall of 78.5％ and a precision of 86.1％ for correct identification of the revised BI-RADS MRI descriptors and the BI-RADS categories.NLP generated the total results in ＜1 s,whereas the manual reviewers averaged 3.38 and 3.23 min per report,respectively.Conclusions:The NLP algorithm demonstrates high recall and precision for information extraction from free-text reports.This approach will help to narrow the gap between unstructured report text and structured data,which is needed in decision support and other applications.