A liver cancer diagnosis frequently leads to surgery, with the goal of completely removing all malignant tissue. To ensure that no tumor remains in the organ, the resected tissue is subjected to frozen-section analysis during the procedure. This analysis takes time: Patients remain under general anesthesia throughout, operating room staff are occupied, and with every passing minute the risk of complications increases.
A research project has now demonstrated that combining optical coherence tomography (OCT) with AI-based anomaly detection delivers fast and reliable results in the analysis of liver specimens, with the potential to accelerate this process in the future. The findings have been published in Scientific Reports.
OCT technology: A 3D view into the interior
OCT is an imaging modality widely used in ophthalmology, for example, to examine the optic nerve. Using light waves, it generates precise 3D cross-sectional scans of tissue, enabling a noninvasive look into its interior. The method is extremely fast: 3D images are acquired within seconds.
At University Hospital RWTH Aachen, the idea arose to use this established technique to image tissue intraoperatively during tumor resection. The method had never previously been applied in this field. Together with the Fraunhofer Institute for Production Technology IPT, 173 OCT scans from 69 patients were acquired under laboratory conditions for testing purposes, comprising 88 scans of normal liver parenchyma and 85 scans of various tumor types.
An unprecedented combination
The novel data set was then analyzed using a machine-learning approach that, compared with previously employed methods, enables considerably faster model training: anomaly detection. This method is particularly well-suited when more nonmalignant than malignant specimens are available.
Ulrich Krispel, an anomaly detection expert at Fraunhofer Austria, explains, “What is special about this method is that the model is trained exclusively on good examples—that is, scans of normal liver parenchyma. The method then reliably detects deviations from this distribution. Using the available data, we achieved a mean accuracy of 81% and have thereby demonstrated that anomaly detection is well-suited as a decision-support tool in this context. Our work has provided the proof of concept.”
The combination of these two methods represents a world first: This is the first application of AI-based anomaly detection to OCT images of human liver tissue.
Results in seconds
Classification results—indicating whether the scans show normal liver parenchyma or tumor tissue—are available after just a few seconds of computation time, enabling rapid intraoperative decision-making.
The accuracy of the classification can then be verified as usual by histopathological examination. According to the study results, the classification is expected to be confirmed in the vast majority of cases, depending on tumor type: One tumor type is recognized with an accuracy of 94.3%, another with 84.5%, and a third with 65.9%.
Iakovos Amygdalos, the clinician at University Hospital RWTH Aachen who originally proposed exploring OCT in this field, is satisfied with the initial results. “I believe this approach holds great potential for the development of a fast and precise intraoperative diagnostic tool for the characterization of suspicious liver lesions. In the future, this could significantly shorten operations, reduce the burden on staff and make the procedure more patient-friendly.”
The translation of this approach from laboratory conditions into the operating room will now be the subject of further research. Caroline Girmen from Fraunhofer IPT adds, “With this project, we have laid the groundwork for establishing OCT as an intraoperative imaging modality in liver surgery. The next steps will be to test the technology under real operating conditions and to miniaturize the sensor system so that it can seamlessly integrate into the surgical workflow in the long term, functioning as a complement to histopathological examination.”
Publication details
Ulrich Krispel et al, Differentiating malignancy from liver parenchyma in Ex-Vivo OCT images using anomaly detection, Scientific Reports (2026). DOI: 10.1038/s41598-026-54850-0
Journal information:
Scientific Reports
Provided by
Fraunhofer Austria Research GmbH
The content is provided for information purposes only.