Is Computer-Assisted Tissue Image Analysis the Future in Minimally Invasive Surgery? A Review on the Current Status of Its Applications.

We reviewed the scientific literature for articles in which CATIA was used to distinguish normal from abnormal human tissues, screen for high-risk cancer cases, and assist in diagnosis and treatment. Using the keywords ‘computer-assisted tissue-image analysis’, ‘computer-aided-diagnosis-systems’, ‘tissue-texture-analysis’, ‘tissue texture image analysis’, ‘endoscopy’ and by filtering with words such as ‘computer-assisted-diagnosis’ and ‘tissue-texture-analysis’, we searched the scientific databases PubMed, IEEE, Xplore, Elsevier, and Google Scholar for relevant articles from 2010 to 2020. Last day of access was the 16th of July 2021. Our search further specifically isolated the articles in the disciplines of dermatology, gastroenterology, otorhinolaryngology (ENT), and gynaecology. No automation tools were used in the process. Articles which did not analyse tissue, reported no tissue texture parameters, as well as any article dealing with numerical measurements or algorithms were excluded from our study. Articles dealing with computing and virtualization were disregarded from our review as well as more complicated, machine learning techniques using colour spectra and filters. However, papers that used tissue texture analysis and augmented their diagnostic methodology using colour spectra or other technologies were included in this review. The isolated articles included digital analysis form, computation analysis, texture image analysis, or any other form of digital CAD specifically on tissue. Each of these forms of analysis use different tools and methods of scrutiny and, therefore, an individualized approach was performed to isolate the relevant articles as described in Figure 1 . A total of 28 relevant articles were selected on the basis of CAD methods used and tissue examined, the data are summarized in Table 2 , Table 3 and Table 4 in order to be assessed and compared. This includes references, CAD method, sample-size, results, conclusions, and critical comments. In most studies, the numerical and quantifiable values produced by tissue texture analysis and interpreted by algorithms came from endoscopic observation of a suspected abnormal tissue region and were correlated with the histopathologic results of a tissue biopsy under light microscopy. Most studies used the algorithms derived from correlating CATIA results with histopathologic criteria for malignancy in individual tissues and relied on computer-assisted diagnostic methods and tissue analysis to distinguish pre-malignant from malignant tissues based on repetition, number of patients, and abnormal tissue variability. All images used in the studies were of human tissue. The images were captured by photography in dermatologic and histopathologic cases, by endoscopy with an external and capsule camera in the gastrointestinal cases, and with a 3-chip camera during hysteroscopy and laparoscopy. All studies focused on identifying neoplastic disease by using CATIA to compare tissue image optical criteria with histopathologic results. A few studies compared normal with abnormal regions of interest captured from the same images and compared to histopathologic findings. Pre-processing procedures were used for reducing image noise, such as γ-correction algorithm (Digital image processing algorithm that compensates for the nonlinear effect of signal transfer between electrical and optical devices) and liner image stretching (Point operation in digital image processing to improve an image by stretching the range of intensity values to a desired range of values). Furthermore, image partitioning (Image segmentation to smaller image parts), was used to select the Regions of Interest (ROIs). Image discriminating texture features (Method to extract relevant information from an image), used in all studies, tried to isolate the best single algorithm or combination of algorithms to distinguish benign from malignant tissue. Manual image segmentation was performed in most studies. Automated segmentation was reported in gastrointestinal videos. Texture and colour discrimination characteristics, multistage illumination (different image scale sizes), correction algorithms (Digital image processing luminance correction to a range of values), histogram equalization (Method that normalises the histogram values to a specific range of values), and support vector machine (SVM) algorithms (Supervised machine learning algorithm that solves two group classification problems) were the main tools used for image analysis.

Clinicians and computer scientists have been exploring the possibilities of using computer-assisted tissue image analysis (CATIA) in distinguishing normal from abnormal tissues. The fast development and application of endoscopy for diagnosis and treatment in daily medical practice, and the use of high-definition video recording facilitate research on CATIA [ 1 ]. Processing evaluation is based on a manual or automated image interpretation that filters artefacts from a database of images. We reviewed existing studies to evaluate the potential of performing optical biopsies of human tissues and the reliability of the results, evaluated the gained experience from CATIA on various tissues, and explored the possibilities of implementing tissue image texture analysis in daily medical practice within the context of computer-assisted diagnostics (CAD) during minimally invasive surgery. In the studies reviewed, tissue image processing techniques focus on three aspects. Tissue image features, colour-spectrum characterization and filtering as well as algorithm and statistical evaluation during or after a patient’s endoscopic procedure ( Table 1 ). Examining suspicious tissue without an invasive procedure by in vivo optical biopsy with tissue image analysis has several advantages: preventing vascular and tissue injury, haemorrhage, haematoma, spread of malignant cells, and risk for infection and scarring. In addition, CATIA allows for comparing suspicious tissue to its neighbouring healthy regions. Thus far, final diagnosis and treatment follows histopathologic examination. Nevertheless, tissue image analysis may guide the physician during biopsy sampling by providing a high-risk or low-risk tissue malignancy score. CATIA could serve as a ‘second opinion’, augmenting the physician’s decision on the biopsy sampling location and allowing a preliminary tissue image result. In the suspicion of malignancy, the histopathological examination can be prioritised. Moreover, tissue image analysis could decrease the risk for error, especially in difficult and suspicious cases with extensive visual tissue variability [ 2 ]. Tissue images during minimally invasive surgery and endoscopic images during colonoscopy and gastroscopy for CATIA could be manually isolated and evaluated when an abnormality or lesion is suspected. An automated system could also be used to define frames of normal and abnormal characteristics in endoscopic segments with different visual appearance [ 3 ]. An expert might easily choose the frames that need further processing, though the inexperienced practitioner might find this more difficult. Specific tissue segments could be isolated from a video, and groups of frames with suspicious pathologic features could be visualized and selected for CATIA. Clustering and classification techniques facilitate the selection of automated images and allows surveillance of defined targets [ 1 ]. Skin, gastrointestinal tract [GIT], larynx, and endometrial [ 4 , 5 ] tissues were analysed for malignancy by optical biopsies. The purpose of the study was to provide a review of computer-assisted tissue image analysis studies during minimally invasive surgery and endoscopy. In addition, we review and evaluate the impact of in vivo optical biopsies performed by tissue image analysis on the surgeon’s diagnostic ability and sampling precision.

Skin diseases evaluated by using CATIA focus on distinguishing malignant from benign conditions. One study on basal cell carcinoma (BCC), two on squamous cell carcinoma (SCC), three on melanoma, one on vitiligo, and one on skin lesions were reported ( Table 2 ). CATIA distinguished normal from benign and malignant tissues with an accuracy of 90% and diagnostic accuracy of 94%, among 80 skin samples of BCC and SCC. In another study of tissue image analysis, the accuracy of distinguishing nevi melanoma from benign disease was from 73% to 81% [ 4 , 5 , 6 ]. Raman spectroscopy and principal component algorithm were used on skin images and distinguished normal from BCC and melanoma with an accuracy of 92.4%, sensitivity of 99.1%, and specificity of 93.3%. In another study focused on texture image analysis of melanoma versus normal and benign precancerous lesions, discrimination accuracy was 91.9% and 82.8%, respectively [ 7 ]. The principal component analysis (PCA) algorithm, used in microscopy images from histopathologic sections to discriminate 145 different spectra of skin biopsies between BCC and melanoma, exhibited accuracy of 92.4%, sensitivity of 99.1%, and specificity of 93.3% [ 6 , 8 ]. A combination of PCA-hLDA and Raman spectroscopy, followed by tissue linear discriminant analysis classification models, demonstrated accuracy of 86%, sensitivity of 100%, and specificity of 66% in differentiating oral cavity SCC of the tongue [ 17 ]. Computer-assisted tissue texture analysis (CATTA) of 80 microscopy images of 200 × 200 pixels recognition against BCC provided 89.92% for allocation accuracy and 93.75% for diagnosis accuracy in locating the region of interest (ROI) for the lesion and discriminating texture features [ 5 ]. PCA’s use was further recognized in BCC and melanoma discrimination from normal skin biopsies in vitro. PCA and elucidation distance classified samples according to histopathologic features with an accuracy of 92.4%, sensitivity of 99.1%, and specificity of 93.3%, by using Raman spectroscopy coupled with a fiber optic Raman probe [ 6 , 8 ]. Similarly, Silveria FL et al., 2015, successfully discriminated non melanoma skin lesions from non tumour human skin tissues in vivo before surgery by using Raman spectroscopy and multivariate statistics [ 7 ]. PCA/DA and PLS/DA, based on mathematical metric space distances, accurately distinguished non melanoma from normal and benign precancerous lesions with 91.9% and 82.8% accuracy, respectively. High-frequency ultrasound and ultrasound elastography (USE) were used to distinguish benign from malignant skin lesions, and histopathologic evaluation for malignancy was correlated with a ratio of compressibility. In characterizing these lesions as malignant, a diagnostic value of 3.0 to 3.9 resulted in 100% sensitivity and specificity [ 18 ]. Computerized algorithms were also used in the primary diagnosis and recurrence management of vitiligo. Vitiligo patients’ primary treatment efficacy was assessed by the digital method (MPR-CDIAS) and proved to perform objective analysis of repigmentation or depigmentation in vitiligo skin lesions in response to treatment. Most CATIA studies we reviewed concern research on GIT endoscopy ( Table 3 ). Tissue image analysis from a wireless endoscopy capsule of 14 patients’ small bowels, reported high specificity and sensitivity in distinguishing malignant from benign tumours [ 4 ]. Oesophageal metaplasia was seen in the images of 34 patients undergoing esophagoscopy using 3D optical coherence tomography (OCT) and the detection rate of metaplasia increased by 72% [ 4 ]. Chromo endoscopy images using CAD tools compared to standard colonoscopy images and histopathologic biopsies improved the detection rate of celiac disease diagnosis by 10%, the detection of dysplastic cells among inflammatory bowel disease by 7%, and of dysplasia in polyp and ulcerative colitis by 9.3% and 1.3%, respectively [ 9 ]. Celiac disease diagnosis was achieved by using gastroscopy and CATIA in identifying mucosa alterations in 2835 duodenum images from 290 children [ 9 ]. Using 350 oesophageal and 129 gastric mucosa images captured by fiber optic telescope, an algorithm system application diminished the honeycomb effect on final images, thus improving the clarity and diagnosis [ 19 ]. On a previous study, we identified the optimal conditions of image capturing during hysteroscopy and laparoscopy [ 10 ] and the ability of a CAD system to distinguish normal from abnormal epithelia by using animal models and video recordings from minimally invasive gynaecological surgeries. Endoscopic images were captured by using animal models at a clinically optimum illumination and focus with 720 × 576 pixels and 24 bits colour for (a) various testing targets from a colour palette with a known colour distribution, (b) different viewing angles, and (c) two different distances from calf endometrium by hysteroscopy and from poultry abdomen by laparoscopy. Human hysteroscopic and laparoscopic pelvic images from the endometrium and ovaries, respectively, were also captured and analysed. For texture feature analysis, three different sets were considered: (i) statistical features (SF), (ii) Spatial Gray Level Dependence Matrices (SGLDM), and (iii) Gray Level Difference Statistics (GLDS). All images were γ-corrected, and the extracted texture feature values were compared against the texture feature values extracted from the uncorrected images [ 1 , 10 ]. In another study, endometrium images in a hysteroscopy office from 40 women with 209 normal and 209 abnormal ROI were compared. Neural network models were also trained to classify 100 normal and 100 abnormal endometrial images with increased CAD sensitivity and specificity, however, without significant difference [ 1 , 10 ]. On the basis of the above results, a standardized protocol was proposed for image capture conditions to optimize tissue texture analysis. In a similar study, data of 52 patients were examined, and 516 ROIs were captured. The ROIs were equally distributed among cases of normal and endometrial hyperplasia. RGB images first γ-corrected and then converted into HSV and YCrCb reached 81% correct classification of ROI by using SF and GLDS features with an SVM classifier. Neophytou M et al., 2015, evaluated 52 hysteroscopic images of 258 normal and 258 abnormal ROIs extracted manually by the gynaecologist, and tissue diagnosis was verified by histopathology after biopsy [ 1 ]. The YCrCb colour system with SF+GLDS colour texture features based on SVM modelling could correctly classify 81% of the cases with a sensitivity and specificity of 78% and 81%, respectively, for normal and hyperplastic endometrium [ 1 ]. The imaging processes of four studies evaluating normal and abnormal tissue for the endometrium and cervix are summarized in Table 4 . The third group of studies deals with hysteroscopic images and lacks the volume of data to extract concrete results. The review study of cervical cancer screening suggests an interesting algorithm, diagnosing the lesion margins by using a colour features discrimination process [ 13 ]. An analysis of hysteroscopic images of 28 patients with abnormal uterine bleeding and images of 39 patients without any pathologic features extracted 167 texture and vessel features for each image. Using artificial neural networks, four tissue features were selected to classify the images further. The specific software system verified the histopathologic diagnosis of 39 patients with normal endometrium and 10 patients with carcinoma with classification accuracy of 91.2%, and specificity and sensitivity of 83.8% and 93.6%, respectively [ 20 ].

The camera systems, monitors, operative techniques, and skills developed with minimally invasive surgery provide tissue images and magnification with exceptional clarity. The abdomen and individual organs can be examined in situ with ease, without disturbing the anatomic features or the pathologic condition before treatment. In addition, video images can be used intra- and postoperatively to re-evaluate the pathologic condition and operative technique and for teaching purposes. They provide the surgeon with excellent quality real-time video, assessing cavities and areas of the human body impossible to observe with the naked eye. The easy access to tissue images facilitates, encourage, and accelerate the application of bioinformatics using different algorithms, which are correlated with the histopathological findings [ 1 , 10 ]. Dual-working channel endoscopes can enable an image-guided punch biopsy by using OCT. Matched OCT images obtained in vivo corresponding to histological biopsies can improve the accuracy and reliability of the technique [ 21 ]. An improvement in image resolution and the development of more specific imaging technology, such as polarization sensitive OCT, may also improve the accuracy of detecting buried pathologic features [ 21 ]. However, dual-working channel endoscopes increase the tip diameter of the scope, which is a big disadvantage when small cavities are observed, as in hysteroscopy. OCT is frequently used in ophthalmology and can provide information about cell architecture and morphology up to 15 nano microns below epithelial cells [ 14 ]. Tissue visual signs, image texture analysis, and selected features by electronic neural network systems can serve as biomarkers distinguishing abnormal from normal tissue. Precancerous as well as cancerous conditions are characterized as images with a complex set of attributes. Colour, texture, and relative geometry are predominately useful, while region shape is significantly less so. Regions are frequently amorphous, or, for a few region classes, exhibit a shape which may be only approximately modelled, and even in these cases, the model may be image dependent. The overall region of interest in the images may in general correlate with the histopathologic cancerous characteristics, such as abnormal tissue architecture, neo-angiogenesis, oedema, and cellular dysfunction. Images from a histopathologic section produced by microscopy may be interpreted by visual signs and tissue image features by computer-assisted diagnosis [ 22 ]. Such translation from microscopy tissue section characteristics to tissue image textures demand an allocation of data and computer system training [ 23 ]. CAD may have the potential to diagnose early disease, including cancer [ 1 ]. The loading of data with digital features of normal and abnormal tissue, with both visual and histopathologic characteristics, is essential in building the primary level of bioinformatics. The functionality and efficiency of CAD depends on network capacity, speed of data processing, and technological support [ 1 ]. The texture discrimination of capsule endoscopy (CE) video frames can be improved by modelling classical texture descriptors in the colour scale plane instead of the colour plane, as usually assumed by classical approaches [ 4 ]. Higher order statistics applied to the joint distribution of classical texture descriptors appear effective for texture characterization. Future work will include introducing different classification schemes [ 4 ]; augmenting the database, which is important in generalizing the results, especially when higher order statistics’ modelling is involved; exploring the temporal dynamics of texture information, since taking information from neighbour frames may improve classification performance [ 4 ]. Optical coherence tomography (OCT) is widely considered a real-time intraoperative tumour margin assessment because of its high-resolution (HR) images, rapid scanning, and optical properties [ 24 ]. However, although OCT provides HR images, the combination of OCTSS and spectral domain (SD) is still insufficient to effectively classify different types of internal organs [ 24 ]. The main reason is that OCT images are simply composed of the reflectivity of light (elastic scattering property), which can only reflect the texture information instead of molecular information [ 25 ]. OCT is a minimally invasive method to evaluate buried glands or other subsurface features and may be used to evaluate the efficacy of other endoscopic therapies, such as cryoablation and photodynamic therapies, not only in the GIT but also in the skin and abdominal cavity [ 26 ]. Raman spectrum is aimed at improving the accuracy of tissue margins’ delineation by detecting the margin of tumour surrounded by normal tissues, e.g., muscle. Based on the integrated system, OCT and RS can acquire the measurement with similar experimental conditions [ 27 ]. This allows for real-time review and assessment of the margins. Tumour margin detection can be evaluated with different algorithms and tissue types. 3D optical coaxial tomography and Raman spectroscopy were the two additional modalities used in combination with the tissue texture analysis to augment CATIA diagnostic ability. Coaxial tomography seems to provide extra information regarding the tissue cell layers below the superficial layer and can be used as an added tool to the optic system. Raman spectroscopy provides highly specific 3D spectra with intensity and time axes mainly used during microscopy for histopathologic sections. Although many ENT articles have been published on CAD, research on tissue texture analysis was missing. No studies using CAD for endometriosis were found. The intensity, density, and variety of tissue hue found in cases of pelvic and abdominal endometriosis would facilitate CATIA research in clinical practice. CATIA could probably contribute to the identification and quantification of endometriosis, especially the depth and extent of the disease on one tissue, the epithelium, and could probably assist in surgical treatment and the depth of destruction by laser and other modalities. Prospective and randomized studies are needed before CATIA is implemented in clinical practice. We found no prospective randomized studies published on CATIA. Most studies we reviewed failed to provide convincing evidence regarding the efficiency and efficacy of their image processing to distinguish normal from abnormal tissue, detect with high accuracy malignant tissues, and verify histopathologic results. The number of cases and image samples analysed in these studies were small, and the methods used were not well described. A major limitation of some studies was the absence of co-registered histologic features of lesions in system datasets, which was due to procedural difficulties; for example, using OCT to guide a biopsy during OCT imaging and using biopsy forceps with a small size and area coverage [ 4 , 11 ]. To address this limitation, one study compared the in vivo and ex vivo images of one biopsy from an endoscopic mucosal resection specimen obtained at the gastroesophageal junction [ 28 ]. Another limitation in cross sectional studies was comparing patients who were at different time points in treatment and stages of the disease. Variations in disease severity and responses across the patient population contribute to variations in data. In all studies, illumination was adjusted for optimal viewing but not for calibrating results to include the viewing angle, distance, and magnification of images. Experiments during hysteroscopy demonstrated that when three different texture feature algorithms, SGDLM, GLDS, and SF, were used, CATIA results were reliable when the distance of the telescope tip to the tissue target was within 3 cm and the viewing angle was kept within 15 degrees deviation [ 10 ]. Selecting the best algorithm or combination of algorithms for the diagnosis of malignant tissue and new cases was a major challenge in almost all studies. CATIA technology needs to be adapted to clinical use, with real-time image analysis supported by a physician-friendly interface. Use of this technology for the diagnosis of malignancy is to diminish false negative results, a fact that is usually accompanied by an increase in false positives and a reduction in specificity. In the studies we reviewed, the major advantage of CATIA was comparing an abnormal tissue region to adjacent normal healthy tissue. However, the comparison between the healthy and the adjacent unhealthy area was neglected in the examination. Comparing images can be used during the intra- and postoperative period to re-evaluate the pathologic features and operative technique. Easy access to tissue images facilitates, encourages, and accelerates the application of bioinformatics by using different algorithms correlated with histopathologic findings [ 1 , 10 ].

CATIA results are encouraging, as many studies demonstrate the CAD systems’ potential to confirm, with high accuracy, abnormal tissue findings diagnosed by histopathology. Our review of CATIA research shows that much information can be extracted and used to diagnose and distinguish normal from abnormal tissue. The naked eye can analyse colour frequencies and detect shape and size differences of 100 microns in diameter. Minimally invasive surgery can facilitate and increase human sight limits up to 34 times that of the naked eye [ 28 ]. This is a great advantage to non-experienced eyes as well as in shortening the diagnostic time and starting treatment earlier. CATIA and OCT will enable the evaluation of several cell layers now beneath human visual capacity. New technical and computational advances will improve optical biopsy and the precision of lesion excision during minimally invasive surgery. The exchange of knowledge, collaboration, identification of tasks, and CATIA method selection strategy will further improve CAD implementation in daily practice at a low cost. More extensive validation on a larger dataset, along with well-designed studies using CATIA, will be required once it is used in a clinical setting as a software system. When CATIA proves that it may increase the surgeon’s diagnostic ability and sampling precision, it could augment the intraoperative management decision and the surgeon’s performance. Additionally, it could minimize complications such as haemorrhage, haematoma, the spread of malignant cells, infection and scarring from multiple biopsies, as well as extensive tissue injuries. A proven efficacy of a CAD method discrimination ability, after validation by prospective and randomized studies, will allow the clinical implementation of CATIA systems and optical biopsies.