Abstract
Background:
Over the last decade, natural language processing (NLP) has provided various solutions for information extraction (IE) from textual clinical data. In recent years, the use of NLP in cancer research has gained considerable attention, with numerous studies exploring the effectiveness of various NLP techniques for identifying and extracting cancer-related entities from clinical text data.
Objective:
We aimed to summarize the performance differences between various NLP models for IE within the context of cancer to provide an overview of the relative performance of existing models.
Methods:
This systematic literature review was conducted using three databases (PubMed, Scopus, and Web of Science) to search for articles extracting cancer-related entities from clinical texts. 33 articles were eligible for inclusion. We extracted NLP models and their performance by F1 scores. Each model was categorized into the following categories: Rule-based, Traditional Machine Learning, CRF-based, Neural Network, and Bidirectional transformer. The average of the performance difference for each combination of categorizations was calculated across all articles.
Results:
The articles covered various scenarios, with the best performance for each article, ranging from 0.355 to 0.985 in F1 score. Examining the overall relative performances, the bidirectional transformer category outperformed every other category (by between 0.2335 and 0.0439 on average F1 score). The percentage of articles on implementing bidirectional transformers has increased over the years.
Conclusions:
NLP has demonstrated the ability to identify and extract cancer-related entities from unstructured textual data. Generally, more advanced models outperform less advanced ones. The bidirectional transformer category performed the best.
Over the last decade, natural language processing (NLP) has provided various solutions for information extraction (IE) from textual clinical data. In recent years, the use of NLP in cancer research has gained considerable attention, with numerous studies exploring the effectiveness of various NLP techniques for identifying and extracting cancer-related entities from clinical text data.
Objective:
We aimed to summarize the performance differences between various NLP models for IE within the context of cancer to provide an overview of the relative performance of existing models.
Methods:
This systematic literature review was conducted using three databases (PubMed, Scopus, and Web of Science) to search for articles extracting cancer-related entities from clinical texts. 33 articles were eligible for inclusion. We extracted NLP models and their performance by F1 scores. Each model was categorized into the following categories: Rule-based, Traditional Machine Learning, CRF-based, Neural Network, and Bidirectional transformer. The average of the performance difference for each combination of categorizations was calculated across all articles.
Results:
The articles covered various scenarios, with the best performance for each article, ranging from 0.355 to 0.985 in F1 score. Examining the overall relative performances, the bidirectional transformer category outperformed every other category (by between 0.2335 and 0.0439 on average F1 score). The percentage of articles on implementing bidirectional transformers has increased over the years.
Conclusions:
NLP has demonstrated the ability to identify and extract cancer-related entities from unstructured textual data. Generally, more advanced models outperform less advanced ones. The bidirectional transformer category performed the best.
| Original language | English |
|---|---|
| Journal | JMIR Medical Informatics |
| Publication status | Accepted/In press - 17 Jun 2025 |
Keywords
- Natural Language Processing
- Information Extraction
- Clinical Textual Data
- Performance
- Review
- Rule-based
- Machine Learning
- Neural Network
- Bidirectional transformer
