Detecting negations from clinical reports using AI
Negations detection is a critical step in extracting clinical information from unstructured text, such as medical reports and clinical notes.
Distinguishing between the presence and absence of an infection can completely change the clinical interpretation of a document and significantly affect the quality of healthcare data integration.
Traditional regular-expression- based approaches (REGEX), while effective, require significant manual effort: defining specific rules, annotating large amounts of data, and iterative feedback from clinicians to update and maintain the rules in accordance with real-world medical language.
A model based on pre-trained transformers offers a low-effort alternative, significantly reducing the need for high-accuracy clinical input.
The developed model builds on BioBIT (an Italian biomedical language model), trained on approximately 38,000 examples from public datasets translated into Italian (Bioscope and i2b2 2010). This process refined BioBIT into BioNEG, a model specialized in negation detection.
BioNEG was then validated on approximately 500 sentences extracted from clinical reports from three Italian hospitals (Policlinico di Milano, Papa Giovanni XXIII in Bergamo, and ASST Ovest Milanese in Legnano). Each sentence was independently reviewed by two annotators to determine whether the clinical concept was present or absent.
Forty percent of this annotated dataset was then used for additional fine‑tuning, producing BioNEGft.
The Regex and BioNEG models achieved comparable performance (accuracy 92% and F1 score 0.93), but with a significant advantage in terms of reduced manual effort.
BioNEGft showed only a minimal F1 improvement (0.94), which may not justify the additional manual work needed to produce local annotations.
Combining the two approaches (Regex and BioNEG) in an AND configuration achieved the best overall performance (F1 0.96 and accuracy 0.95), indicating that they complement each other effectively as a double‑check system.
The study demonstrates that a model like BioNEG, designed to detect negation assertions in Italian clinical texts, can match the performance of traditional regular-expression-based approaches while requiring far less human intervention and offering greater adaptability.
References:
- Buonocore TM, Mognaschi S, Beretta L, Venturelli S, Fattizzo B, Bruschi E, Cassani C, Benatti S, Marinoni S, Vettoretti S, Gabetta M. From Regex to Encoders: Low-Effort Negation Detection for Healthcare Data Integration in Italian Hospitals. pp 62–66 – Artificial Intelligence in Medicine (AIME 2025). (link)
