Feasibility of Depth-in-Color Enface Optical Coherence Tomography for Colorectal Polyp Classification Using Ensemble Learning and Score-Level Fusion

Feasibility of Depth-in-Color Enface Optical Coherence Tomography for Colorectal Polyp Classification Using Ensemble Learning and Score-Level Fusion | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL Journal of Biophotonics This is a preprint and has not been peer reviewed. Data may be preliminary. 8 June 2025 V1 Latest version Share on Feasibility of Depth-in-Color Enface Optical Coherence Tomography for Colorectal Polyp Classification Using Ensemble Learning and Score-Level Fusion Authors : Andrew D. Thrapp 0000-0003-0777-6218 , Sean D’Mello , Constantinos Pitris , Christos Photiou , Genevieve Lamphier , Erica Villareyna-Lopez , Anita Chung , … Show All … , Catriona Grant , Hinnerk Schulz-Hildenbrandt , Oscar Caravaca-Mora , Tiffany Miller , Du-Ri Song , Hamed Khalili , Norman S. Nishioka , and Guillermo Tearney [email protected] Show Fewer Authors Info & Affiliations https://doi.org/10.22541/au.174940051.10805673/v1 486 views 256 downloads Contents Abstract Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract In colorectal cancer (CRC) precursor lesions can be missed during screening due to ambiguity, limited depth sensitivity, or obscured by colonic folds. Optical coherence tomography (OCT) with automated detection may enhance accuracy. OCT imaging was performed on polyps and polyp fragments (300 patients). En face projections were then annotated. In processing, depth was then encoded in color to generate en face OCT projections. The projections were used to train an ensemble network based on malignant potential. The area under the curve (AUC) for the detection of malignant potential of all polyps was 0.90, for diminutive (<5 mm) the AUC was 0.88. Indicating a high degree of accuracy for classification of malignant potential ex vivo . Should results hold in vivo , this algorithm would meet the ASGE’s NPV PIVI criteria, which could allow clinical utilization of OCT for lower colon ‘diagnose and leave’ and/or ‘resect and discard’ strategies for diminutive colon polyps. Feasibility of Depth-in-Color Enface Optical Coherence Tomography for Colorectal Polyp Classification Using Ensemble Learning and Score-Level Fusion Andrew D. Thrapp 12 , Sean D’Mello 1 , Constantinos Pitris 4 , Christos Photiou 4 , Genevieve Lamphier 1 , Erica Villareyna-Lopez 1 , Anita Chung 1 , Catriona Grant 1 , Hinnerk Schulz-Hildenbrandt 12 , Oscar Caravaca-Mora 12 , Tiffany Miller 12 , Du-Ri Song 12 , Hamed Khalili 5 , Norman S. Nishioka 5 , Guillermo Tearney* 1,3,6 [1] Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, USA, [2] Department of Dermatology, Harvard Medical School, Boston, USA [3] Department of Pathology, Mass General Brigham and Harvard Medical School, Boston, USA. [4] KIOS Center of Excellence, Dept. of Electrical Engineering and Computer Engineering, University of Cyprus, Cyprus [5] Division of Gastroenterology, Massachusetts General Hospital, 02114 Boston, MA, USA. [6] Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA 02139, USA —————— *Correspondence email: [email protected] —————— ACKNOWLEDGMENTS We thank the John and Dottie Remondi Family Foundation, National Institute of Health (1R01CA280972) National Cancer Institute, and (5R01EB034107) National Institute of Biomedical Imaging and Bioengineering for their generous support of this research. We would like to thank those who participated in imaging the specimens, including Miquela Murray, Estelle Chiavassa, Nitasha Bhat, Lauren Kole, Elizabeth Biddle, Zachary Zansa, Abigail Gregg, Ronald Yang, and Sophia Zongbi. ABSTRACT In colorectal cancer (CRC) precursor lesions can be missed during screening due to ambiguity, limited depth sensitivity, or obscured by colonic folds. Optical coherence tomography (OCT) with automated detection may enhance accuracy. OCT imaging was performed on polyps and polyp fragments (300 patients). En face projections were then annotated. In processing, depth was then encoded in color to generate en face OCT projections. The projections were used to train an ensemble network based on malignant potential. The area under the curve (AUC) for the detection of malignant potential of all polyps was 0.90, for diminutive (<5 mm) the AUC was 0.88. Indicating a high degree of accuracy for classification of malignant potential ex vivo . Should results hold in vivo , this algorithm would meet the ASGE’s NPV PIVI criteria, which could allow clinical utilization of OCT for lower colon ‘diagnose and leave’ and/or ‘resect and discard’ strategies for diminutive colon polyps. KEYWORDS Colon polyp detection, Optical coherence tomography (OCT), Automated classification, Colorectal cancer (CRC), Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) criteria, Convolutional neural network (CNN) ABBREVIATIONS Adn, Adenoma; AUC, area under the curve; Hyp, Hyperplastic; Nor, Normal; NPV, negative predictive value; OCT, optical coherence tomography; PIVI, Preservation and Incorporation of Valuable Endoscopic Innovations; PPV, positive predictive value; ROC, receiver operating characteristic; SSAP, Sessile Serrated Adenoma / Polyp; INTRODUCTION Colorectal cancer (CRC) is the third most common cancer with prevalence rising among people <50 years old [4, 5]. Colonoscopy has been demonstrated to improve patient outcomes [4, 5]. Even though it is highly effective, missed polyps can lead to interval cancers (5-10% screenings), that develop between surveillance intervals [6, 7]. Subjective screening leads to disparate care, for instance, the adenoma-detection rate (ADR) can vary around 6% between physicians [8], and every 1% increase in ADR correlates with a 3% reduced incidence rate of CRC and 5% reduction in mortality [9, 10]. Objective screening may improve outcomes. New optical modalities are being developed which can screen more objectively, but effective translation requires benchmarking against field specific standards. The Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) standards apply to new in-vivo diagnostic modalities. Two thresholds were established for colon polyps: (1) if suspected rectosigmoid diminutive (=90% a see-and-leave protocol can be adopted, and (2) there is >=90% agreement with post-polypectomy surveillance intervals for all polyps (<=5 mm) then a resect-and-discard protocol can be adopted. In CRC, tissue undergoes dysplastic changes then follows one of the two main architectural sequences [11, 12]. The most common, responsible for ~70% of sporadic cancers, is the adenoma-carcinoma sequence, characterized by chromosomal instability (CIN) [13]. The remaining (~30% of sporadic cancers) are associated with the sessile serrated lesions that arise from hyperplastic lesions; genetically this is characterized by the CpG island methylator pathway (CIMP) [13]. Some tissue architecture is superficial and can be described using Kudo’s pit pattern classification, designed for White Light Endoscopy (WLE) and Narrow Band Imaging (NBI) [3]. Differences can be more subtle; hyperplastic polyps, are benign, but have surface features that are architecturally similar to SSAP [14, 15]. Features that vary with depth fall within the imaging depth of an optical coherence tomography (OCT) system (1-2 mm). OCT is a promising in vivo microscopic imaging technology for the detection and diagnosis of adenomas. 3D visualization of tissue structure is possible using forward- [1, 2] and side-viewing probes [3-11] as well as those with a capsule geometry [12-14]. Early studies showed qualitative differences between OCT images of normal colonic mucosa and adenomas [9, 15]. These pit patterns closely resemble the pit patterns used to diagnose NBI / chromoendoscopy images [12, 16-18]. OCT studies have been performed ex-vivo [2, 9-11, 15, 19-23], and in-vivo OCT has been demonstrated in animals [4, 5, 20, 21, 24], and in patients [1, 2, 10, 12, 22, 25, 26]. Multimodal system variants include spectroscopic OCT [23, 27-30], Doppler [4] and multimodality (OCT and fluorescence or Raman) [5, 6, 24, 31, 32]. They have been applied in animals [5, 6, 21, 24, 32-38], in excised human tissue [23, 27, 28, 30, 31, 39-41], and in humans in vivo [1]. Modern OCT high-speed systems have also been configured to be fast enough to image the entire 3D colon in a realistic procedure time which has been shown with a capsule platform [13]. Algorithms that use OCT intensity, the OCT-derived scattering coefficient (\(µ_{s}\)), or depth-dependent spectra, have shown promise for distinguishing colonic mucosa ex vivo [8, 23, 28, 30, 41-43] and in-vivo [31, 44]. Different texture derived features were input into a support vector machine (SVM) [28]. They showed 95% sensitivity and 94% specificity for detecting cancer and adenomatous polyps [28]. The same group later used an Angular Scattering Index (ASI), derived from the µ_s maps and showed that the ASI can discriminate polyps [41]. Deep learning has also been used, Saratxaga et. al. showed that lesions in a rat model (lat. res. 10 µm) can be classified using the Xception model using B-scan patches, they reported 98% specificity, and 78% specificity [20]. Kendall et. al. reported an ex vivo OCT polyp study (lat. res. 5 µm) using multi-modal fusion deep learning [23]. Scalar features input into a feed-forward model were fused with OCT-derived spectral data input into a bi-directional Recurrent Neural Network (RNN). This was used to classify a-lines with a sensitivity of 98% and specificity of 95%, notably randomization was on a per A-line basis, not per-patient. It is not clear that animal models will generalize to humans with different imaging conditions. Sample numbers in human tissue studies are generally low (<15 polyps) and it is not clear that the models can adequately account for biological variability. The lateral resolution for all studies was <=10 µm, and it’s unclear that this will generalize to probes that have been shown to screen large portions of the organ. The majority of studies have had minimal or no SSAP in the dataset. Furthermore, though normal uninvolved mucosa is often included in data analysis [20, 28, 41], this is pathologically different than polypoid aggregates, and its inclusion may over represent the model’s performance. Here, we present an ex-vivo OCT study comprising polyps and polyp fragments excised from 300 patients (lat. res. 27 µm). An analysis was performed on polyps (all sizes), and a sub-analysis of diminutive polyps (<5 mm). An ensemble network was used; two convolutional neural networks (CNNs) were fused at the score-level into an SVM. Performance metrics were benchmarked against the PIVI criteria. To the best of our knowledge, this is the largest human polyp/polyp fragment study using OCT to date including a large number of SSAP. System description. The imaging system is based on a 1310 nm swept source Axsun engine (Figure 2). Light is guided from the source through a beam splitter (90/10, Thorlabs Inc. TW1300R2A1), on one path to the sample via circulators (Thorlabs CIR1310-APC). The sample reflectance is recombined with the reference arm in BS2 (50/50 Thorlabs Inc. TW1300R5A2). A variable reference arm attenuator (Thorlabs VOA50-APC) is used to adjust reference arm power. The VDL has a 7.5 mm travel range and can cover an optical path of 15 mm. Fiber-based polarization controllers (Thorlabs FPC020) are used to adjust power in each balanced detector. Three fiber types are used, including single mode (SMF28), polarization maintaining (Panda 1310), and short-segment multimode. The scanning head has a collimator (Thorlabs F220APC-1310), 2 galvanometer scanners (Thorlabs GVS002), and a 2.5 cm diameter scan lens (Thorlabs LSM03). Study Using Excised Human Polyps/Polyp Fragments Using Excised Polyps of All Sizes. An ex-vivo study was conducted using polyps and polyp fragments excised from 300 patients (Massachusetts General Brigham Protocol #2007P000656). Following standard colonoscopic resection, the polyps were intercepted immediately post-excision, placed in a petri dish, and imaged using an Axsun-based OCT system. Samples were subsequently sent for standard-of-care histological processing. Independent pathological diagnoses were rendered by the pathology core. The study protocol was optimized part way through imaging to reduce sampling errors; polyp features are not macroscopically visible making it difficult to orient the polyps on the sample tray, consequently, a number of polyps were imaged upside down. To reduce this, polyps from patient numbers 91-300 were imaged on both sides by flipping them over after the initial imaging pass. Performance metrics, including sensitivity, specificity, positive predictive value (PPV), and NPV, were calculated separately for the diminutive polyps. ROC analyses were conducted to benchmark the system’s performance, with the diminutive subset analyzed to ensure the system could distinguish between polyps with no malignant potential (NMP) including suspicious normal samples, hyperplastic polyps, and polyps with malignant potential (MP) including conventional adenomas, and sessile serrated adenomas and polyps. Diminutive Polyp Sub-Analysis. The first PIVI criteria was designed to determine if diminutive (< 5 mm) polyps have malignant potential. To ensure that architectural changes associated with polyp size did not affect our results a specific sub-analysis was performed. Classification was a two-step process, involving training a classifier and subsequent classification of the test set using two convolutional neural networks (CNNs) which output class probabilities, which were then used as inputs into an SVM. In the sub analysis, the existing CNN was used (trained on all polyp sizes). The SVM, however, was re-trained on only diminutive polyps. The test set only included diminutive polyps. Annotation protocol. Enface projections were annotated by an expert pathologist. Four classes were annotated: Normal, Hyperplastic, Adenoma, Sessile Serrate Adenomas / Polyps (SSAP). The pathologist received four maps for annotation, comprising three intensity-based en face projections: surface, mid, and deep and a single attenuation coefficient (\(\mu_{t}\)) map. High-confidence regions were annotated (QuPath) using the pathology core diagnosis, raw histology data, and Kudo’s pit pattern criteria. Polyps that exhibited suboptimal orientation, tissue folding, or poor imaging quality (e.g., artifacts, out-of-focus) were excluded from the analysis if the pathologist could not annotate the images with high confidence (Figure 4). Pre-processing. Annotations. A scan with tissue classified as normal by the pathologist was included in the dataset only if it represented the sole class present. This strategy was implemented to mitigate the influence of uninvolved normal mucosa on NPV and to more accurately assess the model’s performance concerning polypoid aggregates. When both the original and flipped polyp scans were labeled, the results were analyzed separately, followed by calculating an area-weighted average for each metric. This methodology effectively treats the combined scans as a single, large area sample. In instances where samples contained discontinuous regions of a single class, they were similarly aggregated into a single sample. Scans exhibiting multiple classes, with the exception of normal-labeled tissue, were analyzed as separate samples. OCT Pre-Processing. OCT volumes were generated using standard swept-source processing methods, a Hanning window was used before the Fourier transform [16]. To ensure that the OCT images were properly weighted for intensity, and depth dependent differences were attributable to attenuation in a sample we performed a confocal gate correction using a mirror translated away from focus using a z stage, and an attenuator [17]. Volumes used in processing were displayed in the square magnitude representation (\(20\log_{10}\)) to better capture depth dependent features [16]. The polyp surface was automatically segmented using adaptive thresholding (Supplementary methods). Enface slice maps were generated by counting pixels from the top layer of the tissue to a target depth in tissue (assumed constant n=1.4). Three slices thickness were used: surface (0-150 µm), mid (151-300 µm), and deep (301-450 µm). Enface projections were individually autoscaled to fill the dynamic range (1% underflow, 1% saturation) and combined into each channel of an RGB reconstruction. The implementation of autoscaling enables the full utilization of the system’s dynamic range; however, it may introduce image distortions. This issue is most pronounced in tissues with high attenuation, particularly in superficial (top) slices, where deeper layers become increasingly distorted. Practically, this distortion disrupts the visual appearance and shading of crypt structures, impairing their accurate representation. Such artifacts would not be present if attenuation was not a factor. The attenuating structures, as well as the autoscaling process do also remove the context clues necessary to resolve depth-aggregated features. It was critical to recover this, to do that, sum images were also generated which contain the depth-aggregated features and then perform identical autoscaling. The sum and slice images, once generated, were then saved into memory for the classification step. Classifier Design. Colon polyps, viewed from the top, exhibit differences in crypt size and shape depending on their class, viewed from the top down this appears as frequencies differences. These differences have been shown to exist in multi-scale included having a fractal dimension [18, 19]. One option to extract the spatial frequency differences was using a series of fixed Gabor filters [20]. This is similar to the inception module which instead learns convolutional kernels [1]. The module is sensitive to scale as it combines 1x1, 3x3, and 5x5 convolutions within the same layer (Figure 5). The GooLeNet architecture is a network built on these modules[1]. The full GooLeNet architecture has been well reported and can be found in Ref. [1], in brief, it uses global average pooling for parameter reduction instead of fully connected layers to preserve spatial information and mitigate over fitting, and has a depth of 22 layers making it ideal to learn rich, hierarchical feature representations [1]. The network architecture was not modified, except that the last fully connected layer was replaced to match the dimension of the classifier (Output Size = 2) and a pretrained implementation was used. Classification Pipeline. The polyps were randomized on a per-patient basis 50% into a training and validation set (N = 123: Nor = 47, Hyp = 20, Adn = 44, SSAP = 12) and 50% into a test set (N = 94: Nor = 16, Hyp = 21, Adn = 38, SSAP = 19) the diminutive polyps were a subset of this same test set (N = 78, Nor = 16, Hyp = 19, Adn = 28, SSAP = 15). Randomization was done on a per-patient basis to ensure that the model was adequately robust to biological variability. Data was augmented with random x- and y-axis reflection, rotation of 0-90 degrees, and translation. Scale was not used for augmentation as pit size is indicative of class. Machine learning hyperparameter optimization was initially done using a validation set extracted from the training data (25% random patches). Once optimal hyperparameters were fixed and the entire training and validation dataset was used to train the model. The slice and sum images were converted into patches using a scanning square (1 x 1 mm, 100 x 100 pix, overlap: 80%). A patch was only counted as a certain class if 90% of pixels in the patch area had a single-class annotation. The training set was then used to train a convolutional neural network (GoogLeNet [1]). The model was trained for 30 epochs (Learning rate = 0.01). The resulting CNN predicted class probability was input as a score into a linear support vector machine (SVM). The SVM was trained using the training set (5-fold cross validation) with a linear kernel, automatic kernel scaling, box constraint level 1, and standardized data. Data preprocessing, feature extraction, and machine learning model development were conducted using MATLAB R2023a (MathWorks, Natick, MA, USA). The implementation utilized the Neural Network Toolbox and the Statistics and Machine Learning Toolbox to facilitate algorithm development and evaluation. RESULTS The increased biology-informed contrast providing depth-in-color is shown for different polyp types. The performance of the classifier is then shown on all polyps, and a sub-analysis of diminutive polyps. Increased Pit and Crypt Contrast Using Depth-In-Color OCT The enface slice and sum reconstructions were generated using depth-in-color encoding and the resulting full-polyp images were generated ( Error! Reference source not found. ). The projections show clear differences between the crypt patterns of different the classes Normal, Adenoma, Hyperplastic, and SSAP ( ). Normal tissue, as well as uninvolved normal mucosa, maintains the typically honeycomb crypt distribution and ~150 µm size, while adenoma maintains long, spaghetti like predominantly blue (deep) crypts, hyperplastic shows differences in crypt patterns serration project as a blurring of the crypts when contrast to normal tissue, finally sessile serrated lesions can be visualized as enlarged crypts with blue rings around the crypt entrance. Network Performance for Polyps of All Sizes The receiver operating characteristic (ROC) analysis and confusion matrix (Figure 9 (A, B)) highlight the effectiveness of the proposed classification method for colon polyps of all sizes. The performance metrics for the CNN-based approach were an area under the curve (AUC) of 0.90, an accuracy of 86%, and Sensitivity was 95% [85-100%], indicating a strong ability to correctly identify neoplastic polyps, while specificity was 73% [59-87%]. The positive predictive value (PPV) was 0.84 [0.75-0.94], and the negative predictive value (NPV) was 0.90 [0.82-0.98], suggesting the system reliably differentiates between malignant and benign polyps. The ROC operating point selected was the one that met PIVI. Sub-Analysis of Diminutive Polyps and Benchmarking Against the PIVI Criteria A sub-analysis of diminutive polyps (≤5 mm) was conducted with the following performance metrics: an AUC was 0.88, with an accuracy of 86%, sensitivity of 93% [82-100%], and specificity of 77% [63-91%]. The PPV was 0.83 [0.72-0.95], and the NPV was 0.90 [0.81-0.99]. If these results are replicated in vivo , our system would meet the first PIVI threshold. Sub-Analysis Including Normal Uninvolved Mucosa in Test Set Previous studies report accuracy exceeding 90% [21-24]. However, these studies are not directly comparable to ours. One key difference was the presence of normal uninvolved mucosa in the test set [22, 23]. This factor significantly influences classification accuracy. Furthermore, when endoscopy is not available it is important to differentiate polyps from normal tissue. When normal tissue is included in the test set for our model, performance metrics align more closely with those that have been previously reported: AUC: 0.95, Accuracy: 90%, Sensitivity: 94% [83-100%], Specificity: 86% [76-97%], PPV: 0.85 [0.74-0.96], and NPV: 0.94 [0.87-1.00]. In this sub-analysis there were more total samples, so the entire model (CNN and SVM) was retrained with a larger training set (66% training, 33% test). Although these metrics suggest improved performance, this introduces bias into the NPV calculation. Normal tissue exhibits distinct structural characteristics that CNNs can effectively classify, primarily by detecting spatial frequency differences. Patches of uninvolved mucosa often get accurately classified as “negative.” Suspicious tissue, which is later identified as normal pathologically, may have undergone structural changes. Thus, it is crucial to remove uninvolved mucosa from the test set, to avoid overstating performance. There are limits to this approach, as determining what the clinician considered suspicious during endoscopy is inherently complex. The exclusion of normal mucosa unless it is the sole class, likely improves performance, but it may not have eliminated bias when used to distinguish normal tissue vs suspicious tissue with normal pathology. In summary, the NPV and hence PIVI is met with an ex-vivo study of polyps of all sizes as well as diminutive polyps. When normal uninvolved tissue is included in the data set the ability to detect polyps further improves. DISCUSSION Limitations The study had several limitations. The samples were small and thin, there was often tissue damage during excision caused by the forceps, tissue folding and orientation issues, artifacts such as fecal matter, out-of-focus or out-of-range regions, and a low signal-to-noise ratio due to dirty optics. The need to return the tissue for standard-of-care processing within five minutes made real-time lesion orientation challenging. The protocol changes from patient #90–300 which involved flipping the tissue sampling errors related to tissue folding and orientation issues (82% of patients had annotatable data vs. 64% prior to the change). Fecal contents were present in <5% of polyps. Constant focusing issues were difficult to correct since the system had a standard range of 7 mm in air and depending on the sample’s orientation or the size of the polyp, large sections could be missed. This was partially mitigated through focus adjustments, and fewer than 5% of polyps had regions out of range. Bright specular reflections also resulted in unusable regions, which could be misinterpreted as crypts when viewed in en-face. Images were acquired in a beam-normal geometry (without flattening), leading to distortion in the spatial frequencies that the CNN used for classification. Additionally, the study was conducted as part of standard-of-care. While this increased sample yield, it made co-registration of the imaged areas with histology challenging. However, because colon polyps are often pronounced, identifying the specific locations of lesions within the tissue was generally straightforward. Although the exact location of the histology section is not known the features are spatially typical of Kudo’s pit patterns and that reported in WLE / NBI. Translation of Classification Approach to In-Vivo The first PIVI threshold, which requires a negative predictive value (NPV) of 90% or greater, would be met if this approach holds in-vivo. The second PIVI (surveillance interval agreement of 90%) requires a larger study. The performance metrics of our system are comparable to WLE / NBI [25], each have a surveillance interval agreement of 95%. It is likely the approach will perform better in-vivo. In-vivo and ex-vivo imaging each have distinct advantages and challenges. In the case of in-vivo imaging, the organ is intact but there are also fecal contents, although these can be reduced with bowel preparation. Other artifacts may occur depending on probe type. In the case of capsule-like probes these are motion artifacts, limited focus and OCT range control [26, 27]. Motion artifacts have been reported in a capsule with high resolution driveshaft scanning to be visible in ~20% of frames when using extremely high magnification [26]. Tissue folding is a problem observed in capsule probes [26] and will likely render parts of the image un-analyzable, [27]. Defocus will also need to be compensated and can be improved with either extended depth of focus or numerical refocusing [28, 29]. The imaging range exceeding the bowel wall can be adjusted in real time using a tracker with a dynamically adjustable variable delay line [30]. Forward viewing probes which can capture enface OCT projections can be used with localization performed via a conventional endoscope [31]. These probes will not be subject to tissue folding, motion artifacts, or OCT range control. They would, however, suffer from sampling error. These findings are not dissimilar from those in endoscopy. Similar performance metrics were seen with white light imaging (Sensitivity = 84%, specificity = 86%, NPV = 0.96) and narrow band imaging (Sensitivity = 84%, specificity = 84%, NPV = 0.96). Importantly, both studies then showed screening interval agreement of 95%. Meeting the second PIVI criteria for diminutive polyps. CONCLUSION This study demonstrates that an OCT system, combined with machine learning algorithms can effectively classify ex-vivo colorectal polyps, achieving performance metrics that meet the AGSE’s PIVI’s NPV criteria. Specifically, the system achieved high accuracy, sensitivity, and specificity in distinguishing polyps with malignant potential, importantly a large number of SSAP which can be easily confused with hyperplastic were in the dataset. The ability to meet these clinical benchmarks suggests that OCT, when integrated with automated analysis, holds significant promise as a diagnostic tool for real-time classification of colorectal lesions. ARTIFICIAL INTELLIGENCE Artificial intelligence was used as part of the classifier as described in the paper. Large language models (LLMs, ChatGPT 4o) were used to check sentence structure and provide recommendations on improving the flow of text. It was also used to extract the method (Supplementary Method) from a large MATLAB code file and provide a draft which was then checked and edited. FINANCIAL DISCLOSURE Guillermo Tearney has a financial/fiduciary interest – Equity (>1%) – in SpectraWAVE. He receives royalties from MIT, Terumo, Nidek, Heidelberg Engineering, SpectraWAVE, iLumen. He performs consulting at SpectraWAVE, Horizon Therapeutics, Novo Nordisk. He has committee/board membership at Dana Farber, Harvard Cancer Center - Co-chair of CARPED (Cancer Risk, Prevention, and Early Detection Program), Markus Institute for Aging Research – Vice-chair, Research Committee, Chair, Ventures Committee, SpectraWAVE – Clinical Advisory Board, Commercialization Council, Partners Healthcare Innovation – Member, MGB Research Strategic Plan, Sustaining Financial Strength Committee – co-chair. He engages in industry-sponsored research from Amgen, Astra Zeneca, Horizon Therapeutics, Canon USA, iLumen, Terumo (catheter materials), Bloch Quantum. Andrew Thrapp, Du-Ri Song, Constaninos Pitris, Guillermo Tearney. are listed as inventors on patent application “WO2024031010A1” titled “Retrograde tethered capsule endomicroscopy systems and methods,” for which they have a right to receive royalties, and Andrew Thrapp and Guillermo Tearney are listed as inventors on “US2025/030032” titled “Systems and Methods for Automated Detection of Colon Polyps Using Depth-in-Color Encoding and Machine Learning” held or filed by the General Hospital Corporation, for which they have a right to receive royalties. CONFLICT OF INTEREST The authors declare no financial or commercial conflict of interest. DATA AVAILABILITY STATEMENT Summary data that support the findings of this study are available from the corresponding author upon reasonable request. SUPPLEMENTARY DATA The polyps were primarily excised from the rectosigmoid. Table 1 shows the distribution of polyps by location in the training and test sets. Table 1 - Locations where polyps were extracted from and how they were distributed in the training and test sets. Rectosigmoid 37 25 Descending 4 4 Transverse 23 19 Ascending 15 32 Cecum 17 12 Unknown 3 1 Surface Map Generation Using A Custom Adaptive Thresholding Routine To generate the surface map of the tissue, we utilized a custom adaptive thresholding algorithm designed to segment the enface projections from 3D OCT volumes. The process involved several key steps to ensure accurate surface delineation: 1. Data Preprocessing : The raw OCT volume data was loaded from the respective directory and preprocessed. The enface projection for each B-scan was generated by summing the pixel intensities across the depth dimension. This provided a 2D representation of the tissue surface in each frame. A pre-existing tissue mask was applied to isolate tissue regions and exclude noise. 2. Low Pass Filtering and Identifying a High-Confidence Region : For each B-scan, the pixel intensities were processed to remove low-signal noise through rectification and intensity normalization. A Gaussian filter (σ = 5) was applied to smooth the image and eliminate high-frequency noise. A pixel-wise threshold was then calculated based on the mean intensity of non-zero values, retaining only the pixels with intensities above 25% of the mean to highlight tissue structures. The binary mask was then used as a high confidence region. 3. Surface Detection : For each frame, the location of the tissue surface was then identified by detecting maximum intensity in each column of the image (inside the high confidence region). A moving average (window = 5) was applied to smooth the detected surface. To account for irregularities, an adaptive reset check was introduced. A for-loop compared the detected surface with the smoothed mean. If the deviation exceeded 25%, the detected surface was reset at the values where that was exceeded, this led to interpolation through multiple iterations of smoothing. 4. Fine-tuning Surface Segmentation : The surface was further refined by iteratively correcting outliers using the moving mean approach. This adaptive adjustment ensured that the surface map accurately followed the contour of the tissue, even in the presence of noise or artifacts. 5. Interpolation and Fine-tuning : To handle dropped frames and missing data, interpolation was applied between frames where the surface could not be detected. A low-pass Gaussian filter (σ = 1 and 3) was used to smooth the final surface map. Regions that deviated significantly from the smoothed surface were adaptively corrected based on the difference between the raw surface and the smoothed data. REFERENCES 1. Szegedy, C., et al. Going deeper with convolutions . in Proceedings of the IEEE conference on computer vision and pattern recognition . 2015.2. Crockett, S.D., et al., Sessile Serrated Adenomas: An Evidence-Based Guide to Management. Clinical Gastroenterology and Hepatology, 2015. 13 (1): p. 11-26.e1.3. Kudo, S.-e., et al., Diagnosis of colorectal tumorous lesions by magnifying endoscopy. Gastrointestinal Endoscopy, 1996. 44 (1): p. 8-14.4. Weinberg, B.A. and J.L. Marshall, Colon Cancer in Young Adults: Trends and Their Implications. Curr Oncol Rep, 2019. 21 (1): p. 3.5. Winawer, S.J., Colorectal cancer screening. Best practice & research Clinical gastroenterology, 2007. 21 (6): p. 1031-1048.6. Sanduleanu, S., A.M. Masclee, and G.A. Meijer, Interval cancers after colonoscopy—insights and recommendations. Nature reviews Gastroenterology & hepatology, 2012. 9 (9): p. 550-554.7. le Clercq, C., et al., Interval Colorectal Cancers Frequently Have Subtle Macroscopic Appearance: A 10 Year-Experience in an Academic Center. Gastroenterology, 2011. 140 (5): p. S-112-S-113.8. Hassan, C., et al., Variability in adenoma detection rate in control groups of randomized colonoscopy trials: a systematic review and meta-analysis. Gastrointestinal endoscopy, 2023. 97 (2): p. 212-225.e7.9. Pohl, H. and D.J. Robertson, Colorectal Cancers Detected After Colonoscopy Frequently Result From Missed Lesions. Clinical Gastroenterology and Hepatology, 2010. 8 (10): p. 858-864.10. Rex, D.K., et al., Quality indicators for colonoscopy. Gastrointestinal Endoscopy, 2015. 81 (1): p. 31-53.11. Fearon, E.R., Molecular Genetics of Colorectal Cancer. Annual Review of Pathology: Mechanisms of Disease, 2011. 6 (1): p. 479-507.12. Pino, M.S. and D.C. Chung, The Chromosomal Instability Pathway in Colon Cancer. Gastroenterology, 2010. 138 (6): p. 2059-2072.13. Bettington, M., et al., The serrated pathway to colorectal carcinoma: current concepts and challenges. Histopathology, 2013. 62 (3): p. 367-386.14. Amaro, A., S. Chiara, and U. Pfeffer, Molecular evolution of colorectal cancer: from multistep carcinogenesis to the big bang. Cancer and Metastasis Reviews, 2016. 35 (1): p. 63-74.15. Kashida, H., et al., Endoscopic characteristics of colorectal serrated lesions. Hepato Gastroenterology-Current Medical and Surgical Trends, 2011. 58 (109): p. 1163.16. Drexler, W., Optical Coherence Tomography. 2015.17. Faber, D.J., et al., Quantitative measurement of attenuation coefficients of weakly scattering media using optical coherence tomography. Optics Express, 2004. 12 (19): p. 4353-4365.18. Zeng, Y., et al., The angular spectrum of the scattering coefficient map reveals subsurface colorectal Cancer. Scientific reports, 2019. 9 (1): p. 2998.19. Costas, P., T. Andrew, and J.T.M.D. Guillermo. Morphological segmentation and fractal analysis for the classification of colon polyps from en face optical coherence tomography (OCT) images . in Proc.SPIE . 2023.20. Serre, T., Robust Object Recognition with Cortex-Like Mechanisms. IEEE Trans. Pattern Anal. Mach. Intell, 2007. 29 (3): p. 411-426.21. Saratxaga, C.L., et al., Characterization of Optical Coherence Tomography Images for Colon Lesion Differentiation under Deep Learning. Applied Sciences, 2021. 11 (7): p. 3119-.22. Kendall, W., et al., Deep learning classification of ex vivo human colon tissues using spectroscopic OCT. bioRxiv, 2023: p. 2023.09. 04.555974.23. Luo, H., et al., Human colorectal cancer tissue assessment using optical coherence tomography catheter and deep learning. J Biophotonics, 2022. 15 (6): p. e202100349.24. Haolin, N., et al. In vivo colorectal polyp evaluation using an optical coherence tomography catheter and deep learning: results of a feasibility study . in Proc.SPIE . 2024.25. Wallace, M.B.M.D., et al., Accuracy of in vivo colorectal polyp discrimination by using dual-focus high-definition narrow-band imaging colonoscopy. Gastrointestinal Endoscopy, 2014. 80 (6): p. 1072-1087.26. Adler, D.C., et al., Three-dimensional endomicroscopy of the human colon using optical coherence tomography. Opt Express, 2009. 17 (2): p. 784-96.27. Song, D.-R., et al. Safety study of tethered capsule endomicroscopy (TCE) pull back through long segments of the small intestine . in Endoscopic Microscopy XVII . 2022. SPIE.28. Yin, B., et al., Extended depth of focus for coherence-based cellular imaging. Optica, 2017. 4 (8): p. 959-965.29. Ralston, T.S., et al., Inverse scattering for optical coherence tomography. Journal of the Optical Society of America A, 2006. 23 (5): p. 1027-1037.30. Tearney, G.J., B.E. Bouma, and J.G. Fujimoto, High-speed phase- and group-delay scanning with a grating-based phase control delay line. Optics Letters, 1997. 22 (23): p. 1811-1813.31. Liang, K., et al., Endoscopic forward-viewing optical coherence tomography and angiography with MHz swept source. Optics Letters, 2017. 42 (16): p. 3193-3196. Information & Authors Information Version history V1 Version 1 08 June 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Collection Journal of Biophotonics Keywords automated classification colon polyp detection colorectal cancer (crc) convolutional neural network (cnn) optical coherence tomography (oct) preservation and incorporation of valuable endoscopic innovations (pivi) criteria Authors Affiliations Andrew D. Thrapp 0000-0003-0777-6218 Harvard Medical School Department of Dermatology View all articles by this author Sean D’Mello Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Constantinos Pitris University of Cyprus View all articles by this author Christos Photiou University of Cyprus View all articles by this author Genevieve Lamphier Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Erica Villareyna-Lopez Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Anita Chung Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Catriona Grant Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Hinnerk Schulz-Hildenbrandt Harvard Medical School Department of Dermatology View all articles by this author Oscar Caravaca-Mora Harvard Medical School Department of Dermatology View all articles by this author Tiffany Miller Harvard Medical School Department of Dermatology View all articles by this author Du-Ri Song Harvard Medical School Department of Dermatology View all articles by this author Hamed Khalili Massachusetts General Hospital Division of Gastroenterology View all articles by this author Norman S. Nishioka Massachusetts General Hospital Division of Gastroenterology View all articles by this author Guillermo Tearney [email protected] Massachusetts General Hospital Wellman Center for Photomedicine View all articles by this author Metrics & Citations Metrics Article Usage 486 views 256 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Andrew D. Thrapp, Sean D’Mello, Constantinos Pitris, et al. Feasibility of Depth-in-Color Enface Optical Coherence Tomography for Colorectal Polyp Classification Using Ensemble Learning and Score-Level Fusion. Authorea . 08 June 2025. DOI: https://doi.org/10.22541/au.174940051.10805673/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.174940051.10805673/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9ff0002d8b370db4',t:'MTc3OTMyODk2NA=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();