Applications of artificial intelligence in urologic oncology.

Artificial intelligence (AI) is a field of technology that uses computer systems to mimic intelligent behaviors such as human learning, reasoning, problem-solving, and cognition [ 1 ]. AI has been studied for its potential and is already being utilized in various industries. AI is also expected to bring revolutionary changes to health care, especially in facilitating clinical decision-making [ 2 3 ]. AI can be used for various purposes, including aiding in diagnosis in outpatient settings, establishing treatment plans, performing predictive analytics, remote monitoring of patients through wearable devices and mobile apps, and optimizing electronic medical records [ 3 4 5 ]. To understand AI in the medical field, it is necessary to explore the following concepts. (1) Machine learning (ML): this refers to learning based on data, recognizing patterns, and making predictions. ML is categorized into unsupervised, supervised, and reinforcement learning. ML algorithms learn patterns from data; therefore, the more data that are used, the better the algorithm performs [ 6 ]. ML has applications in various medical areas, including predicting patient outcomes, categorizing diseases, and monitoring patients. (2) Deep learning (DL): this is a subset of ML that helps solve more complex problems using artificial neural networks (ANNs). DL is the basis for medical image analysis, speech recognition, and natural language processing [ 7 8 ]. (3) Computer vision: this is the field of AI that analyzes and understands image and video data. Computer vision mainly analyzes medical image data to detect and quantify tumors, abnormalities, and diseases [ 9 ]. (4) Natural language processing: this focuses on understanding and processing text and audio data, and in the medical field, it can be used to interpret medical records to derive disease patterns and key information to help in decision-making [ 10 ]. (5) Robotics: in the medical field, robotics technology, which is applied to surgical robots and automated medical systems, is not directly related to AI. However, robotics technology is expected to advance significantly with the application of AI, which will substantially change the medical field [ 11 ]. Much research has been conducted on ML and DL over the past 10 years. Technologies utilizing these applications are expected to be helpful in personalized drug treatment, image recognition diagnosis, and clinical decision-making when applied to actual medical practice. In urology, AI has been studied for its potential use in various diseases. In this review, we discuss the applications of AI in the management of urological cancers.

For prostate cancer, AI is used in various ways to help in diagnosing, treatment decision-making, and predicting prognosis. The studies on application of AI in prostate cancer are summarized in Table 1 . As the importance of clinically significant prostate cancer has risen, multiple attempts have been made using multiparametric magnetic resonance imaging (MRI) with AI to distinguish clinically significant lesions. Ishioka et al. [ 22 ] developed a computer-aided diagnosis system using convolution neural network (CNN) algorithms with prostate biopsy slides and MRI images of 335 patients. Data from 301 patients were used to create the training algorithm, and the remaining 34 patients were divided into two groups and tested. The areas under the curve (AUCs) were 64.5% and 63.6%, respectively, and 16 and 7 patients, respectively, were incorrectly diagnosed as having prostate cancer. Takeuchi et al. [ 32 ] also studied whether prostate cancer could be predicted using multiparametric MRI in patients who underwent biopsy, using a multilayer ANN. Of the 334 patients, 232 were used for training, the remaining 102 for testing, and 48% of patients without prostate cancer were analyzed. Of patients with prostate cancer and a Gleason score (GS) of 7 or higher, 16% and 6% were missed [ 32 ]. Aldoj et al. [ 15 ] developed a CNN model that semi-automatically diagnosed prostate cancer using MRI. When only MRI images were used, the AUC was 89%. When a combination of diffusion-weighted imaging and dynamic contrast-enhanced MRI was used, the effect was improved, with an AUC of over 90% [ 15 ]. Seetharaman et al. [ 30 ] created a CNN-based model to detect clinically significant cancer from MRI studies. When they tested data from patients who underwent MR-ultrasound fusion biopsy and prostatectomy, the AUCs were 75% and 80%, respectively. They also detected 18% of lesions that radiologists missed. In addition, various AI models have been studied to measure tumor grade. Ström et al. [ 31 ] developed an AI model to diagnose Gleason grade. The model was trained with 6,682 slides from 976 patients and was tested with 1,631 slides from 246 patients and showed high accuracy with an AUC reaching 99% [ 31 ]. The correlation between cancer length by the AI and the pathologist was 0.87 for the external dataset. This demonstrated diagnostic performance that was the same or above that of the pathologists. These results were similar and externally validated in a study by Bulten et al. [ 13 ]. The use of AI in predicting prostate cancer prognosis is also being actively studied. Bibault et al. [ 17 ] used an AI algorithm to predict survival outcomes in prostate cancer patients. Using the prostate cancer dataset of 8,776 individuals from the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial, the training set included data from 7,021 individuals; the remaining 1,755 individuals made up the testing set. The accuracy of the 10-year survival rate and 10-year cancer-specific survival rate were 87% and 98%, respectively [ 17 ]. One study demonstrated the use of an AI model to predict the recurrence rate and progression of prostate cancer based on biomarkers [ 12 ]. Anti-Ki-67 and ERG antibody immunofluorescence staining was performed using 648 samples (424 tumor tissues, 224 normal tissues), and the results showed only a 5% difference compared with prediction based on manual analysis [ 12 ]. There have also been studies predicting patient complications during surgery. Hung et al. [ 21 ] developed a model to predict patient incontinence by analyzing automated performance metrics (APMs) during robotic prostatectomy using a DL model. When eight surgeons were divided into two groups of four by APMs, the 3- and 6-month continence rates of the top group of four significantly differed at 47.5% versus 36.7%, and 68.3% versus 59.2%, respectively [ 21 ]. Overall, AI algorithms are expected to play a role in redefining diagnosis and management in prostate cancer. The continued evolution of AI and its application to actual medical settings has the potential to improve outcomes in patients with prostate cancer. The application of AI has also been studied for bladder cancer diagnosis and outcome prediction ( Table 2 ). Diagnosis and staging of bladder cancer are mainly performed by using cystoscopy and transurethral bladder surgery. Because cystoscopy has limitations in distinguishing unclear lesions, such as inflammatory mucosal change and carcinoma in situ , various attempts have been made to use AI to improve the accuracy of cystoscopy. Ikeda et al. [ 36 ] developed a CNN model to help diagnose bladder cancer by training of 2,102 cystoscopic images, with a sensitivity of 89.7% and a specificity of 94.0%. Lorencin et al. [ 38 ] used a dynamic conditional convolutional network and multilayer perceptron for bladder cancer diagnosis using 1,997 bladder cancer images and 986 normal images. In that study, the AUC value was as high as 99% [ 38 ]. Yoo et al. [ 49 ] created a diagnostic tool through AI that used the red/green/blue method for diagnosing bladder cancer through cystoscopy. The accuracy of this study, which examined 10,991 cystoscopy images, was 94.1%. When using the red/green/blue method, the accuracy was 98% in distinguishing between benign tumors and bladder cancer (low-grade or high-grade). Various diagnostic tools have also been investigated to enhance pathology diagnosis. Sokolov et al. [ 42 ] analyzed urine cytology results using polymer microscopy and ML and developed a method for diagnosing bladder cancer with an accuracy of 94%. Pan et al. [ 40 ] developed a pathological AI diagnostic model to quickly diagnose invasion depth and histological grade using bladder cancer samples. After training with 854 slides, the AUCs of the model at the patch level and for whole-slide images were 87% and 84.7%, respectively [ 40 ]. These results demonstrate that the performance is not inferior to that of an existing senior pathologist. Loeffler et al. [ 37 ] conducted research to develop AI to predict FGFR3 gene mutation using existing stained slides without genetic analysis. By training the system using slides from 327 individuals and conducting a performance test on the slides from 182 other individuals, the AUC was reported to be about 62.5%. This showed better performance than that of conventional pathologists. AI tools for diagnosing bladder cancer based on imaging modalities such as computed tomography (CT) and MRI are being actively studied. Yang et al. [ 48 ] attempted to distinguish non-muscle-invasive bladder cancer (NMIBC) from muscle-invasive bladder cancer (MIBC) using 1,200 bladder cancer CT images. By conducting training and validation tests with eight DL algorithms, one model resulted in a high AUC of 99.8%. Xu et al. [ 47 ] conducted a study on distinguishing between NMIBC and MIBC using MRI. Their study, conducted with 1,104 MRI images, had an accuracy of 96.3% and an AUC of 98.6%, which was more accurate than conventional readings. AI models have also been used to predict recurrence and survival outcomes after treatment for bladder cancer. Wang et al. [ 44 ] used an ML model based on a support vector machine to predict 5-year survival and cancer-specific mortality after radical cystectomy with an accuracy of over 74%–77%. Using an ML ensemble model, Hasnain et al. [ 35 ] created a predictive model with more than 70% sensitivity and specificity for recurrence and survival rates at 1, 3, and 5 years after bladder cancer resection. Because the model was evaluated using multiple predictors before and after surgery rather than a single predictor, it was more effective than existing methods. The model can help to determine patients’ outpatient visit intervals and adjuvant treatment. Predicting bladder cancer treatment response using ML has also been studied. Cha et al. [ 33 ] developed a computerized decision support system based on CT to distinguish patients who could respond well to neoadjuvant chemotherapy. Wu et al. [ 45 ] also used the CNN model to predict which patients would respond well to chemotherapy. These studies may help to avoid the adverse effects of chemotherapy in bladder cancer patients. AI research on kidney cancer primarily aims to aid diagnosis and predict treatment response and survival ( Table 3 ). With the use of conventional CT and MRI, it is difficult to distinguish between a benign tumor and renal cell carcinoma (RCC) in an accidentally discovered small renal mass. However, one study showed that by using ML and DL algorithms to analyze the shape in CT studies, angiomyolipoma, clear cell RCC (ccRCC), papillary RCC (pRCC), and oncocytoma could be distinguished [ 59 ]. Yan et al. [ 64 ] used an ANN in 3-phase CT to determine angiomyolipoma, ccRCC, and pRCC, and the accuracy ranged from 90.7% to 100%. In a study by Feng et al. [ 55 ], when 42 photos of 58 patients with small renal masses (17 angiomyolipoma and 41 patients with RCC) were selected and tested, the analysis could distinguish those with angiomyolipoma from those with RCC with 93.9% accuracy. Coy et al. [ 54 ] also used DL to classify ccRCC and oncocytoma in 179 patients with kidney lesions at an accuracy of 74.4%. Several studies have used AI to diagnose RCC on pathology slides. Fenstermaker et al. [ 56 ] used slides from 42 patients with RCC to not only diagnose RCC but also classify RCC subtype and Fuhrman pathology grade, with accuracies of 99.1%, 97.5%, and 98.4%, respectively. Zhu et al. [ 65 ] also analyzed RCC subtype using samples resected at the time of surgery and samples from renal biopsy, with an accuracy from 9% to 7%. Abdeltawab et al. [ 66 ] developed a computational model to distinguish between ccRCC and pRCC, with 91% accuracy for pRCC and 90% for ccRCC. When tested using external data, ccRCC was distinguished with 82%–90% accuracy. Other studies have gone beyond differentiating tumors by using CT scans and ML to distinguish the pathology grade of RCC. Kocak et al. [ 57 ] also analyzed noncontrast CT using an ANN model only for 81 patients with ccRCC (56 high-grade and 25 low-grade) and classified them with an accuracy of 81.5%. Gene expression has recently been found to be essential in predicting patient prognosis in ccRCC, and a study by Kocak et al. [ 58 ] reported that CT can predict PBRM1 mutation. In their study, PBRM1 mutations were classified with an accuracy of 88.2% by use of an ANN model [ 58 ]. In recent years, various attempts have been made to predict the prognosis of RCC through AI. Ning et al. [ 60 ] created a computational model to evaluate the prognosis in patients with RCC by combining CT and histopathology test results, genes, and clinical data. Schulz et al. [ 61 ] trained a multimodal DL model with multiple histopathology, CT, and MRI photos, as well as genetic data to show a better 5-year survival rate in predicting the current TNM stage. In addition, low-risk and high-risk groups were categorized according to the level of risk. However, there was a limitation of a small sample size; therefore, further research is needed to complement these results [ 61 ]. One of the main treatment methods for cancer of the urinary system is robotic surgery. Urinary organs are often difficult to access in an open surgery; therefore, the use of robots has been explored for a long time as a suitable surgical method. Not only is robotic surgery suitable for patients because it has fewer complication rates than conventional open surgery, but it also allows surgeons to perform surgery more precisely. Accordingly, there have been recent attempts to further complement and enhance robotic surgery using AI. Surgeons’ performance using robotics is associated with good prognosis in the patients. Hung et al. [ 20 ] proposed a method using ML to objectively evaluate surgeons’ performance by APM, operation of cameras and instruments during surgery, and energy device usage during robotic-assisted radical prostatectomy. This method has the advantage of providing feedback in the right direction, not only to individual surgeons, but also to academic institutions in evaluating performance, unlike the existing method of evaluation by experts. Accordingly, Hung et al.’s research [ 69 ] focuses on developing programs for educating and training surgeons. This includes efficient training methods using ML and simulation-based training. In one study by Hung et al. [ 20 ], datasets of robotic prostatectomy cases were learned, and each dataset was categorized based on hospitalization periods to less than or equal to 2 days and more than 2 days. By training with a new set of 50 surgical patients, the accuracy of classification according to the hospitalization periods was 87%. Their research also focuses on developing ML models to predict and improve surgical outcomes. This can help to optimize surgical strategies and create patient-tailored treatment plans [ 20 70 71 72 ]. Although this is a promising research area, it is still in the early stages, and more training data are needed for clinical application. Therefore, automated datasets from multiple institutions and many surgeons are required. AI’s role in surgical procedures is not yet clear, and this research is in its infancy. AI in surgery is expected to evolve to AI-enhanced and AI-assisted surgery. Application of AI in surgery first requires an AI algorithm that categorizes and identifies various anatomical parameters and surgical tools visible during the procedure [ 73 74 ]. This is comparable to the ability to distinguish road structures and other cars in real time, for normal autonomous driving. Several studies on algorithms have identified abnormal areas in real time in gastrointestinal endoscopy images, but these findings have limitations when applied to surgery considering the difference in complexity between endoscopic examination and surgery [ 75 76 ]. Some studies have reported that they could use DL and computer vision technology to identify surgical instruments in robotic surgery images or segment and recognize organs in images of surgeries, such as cholecystectomy and uterine myomectomy [ 39 77 ]. Takeshita et al. [ 78 ] reported that a CNN model accurately recognized seminal vesicles in a small dataset of 26 prostate cancer patients who underwent robotic prostatectomy with a Dice coefficient of 0.73.

While prognosis and outcome prediction using ML algorithms, such as those based on a support vector machine [ 26 44 46 55 57 ], have been widely investigated in the past decade, computer vision technologies like CNN are now rapidly advancing and the application of DL algorithms in various medical image analyses is being extensively studied [ 79 ]. ML algorithms are highly flexible and make minimal assumptions, while traditional statistical methods rely on strong assumptions, such as the type of error distribution, linear relationships, and proportional hazards [ 80 81 ]. Furthermore, ML algorithms have advantages over traditional statistical techniques when handling huge dimensional data and integrating various data types into predictions. These advantages of ML are valuable in omics science, especially in radiomics analysis in the field of medical imaging. Radiomics is a method of AI in which the computer is trained to interpret medical images using characteristics undetectable by the human eye [ 82 ]. It involves extracting large quantities of quantitative radiomics features from medical images using data-characterization algorithms. Most studies on computer-aided diagnosis in urological cancer have used radiomics from different modalities and have demonstrated promising results [ 14 28 34 43 47 54 58 82 ]. In this review, we presented studies on urological cancers which demonstrated that AI algorithms can significantly enhance the process of diagnosis, treatment, and prognosis prediction. Using ML and DL algorithms, it will be possible to identify patterns in medical data that were previously difficult to detect, which can help to enhance the management of urological cancers. Despite AI’s great promise in urological cancers, there are still some important limitations. Current AI models cannot fully understand or predict complex and unpredictable human physiologic phenomena. Despite the advantages of AI over traditional statistical models, AI cannot answer all the questions. Traditional statistical approaches have the advantage of being simple to interpret and they produce clinician-friendly measures of association, such as odds ratios or hazard ratios [ 80 ]. In addition, ML algorithms require large datasets, require iterative refinement with regard to the real medical problem, and may have overfitting issues. There are many examples where ML has done poorly in the medical field [ 80 ]. Furthermore, there are limitations of most studies on the application of AI in urological cancers. First, many of them validated the developed models within their own dataset without external validation and did not compare AI models with traditional statistical models. Second, there are many differences in AI algorithms, data quality, and training parameters between studies. These limitations lead to the lack of generalizability of the study results. ML algorithms rely on large amounts of patient data, and application of AI in clinical practice can lead to privacy and data security concerns. The ethical and legal responsibility issues of AI in medical decisions and results are complex, and clarifying the responsibility for AI decisions remains a difficult task. Many limitations will gradually be overcome with advances in AI technology and data quality, but further research and development are needed before these limitations can be fully addressed. The future of urological cancer management using AI is likely to include several transformational changes. DL algorithms will be applied beyond the analysis of radiologic imaging modalities or pathology slides to real-time surgery under vision and complex omics data. In addition, AI models can analyze large amounts of patient data, including disease characteristics, genetic profiles, and comorbidities to predict cancer prognosis and suggest an appropriate management strategy. Generative AI, which has been rapidly advancing recently, is expected to bring about innovative changes that will greatly help in decision-making by interpreting various patient data [ 83 84 ]. Robotic surgeries are already playing an important role in urological cancer treatment, and the integration of AI can further improve the accuracy and efficiency of these surgeries [ 85 ]. AI algorithms may be helpful in virtual surgery training and for providing feedback to urologists who are less experienced on robotic surgery. The International Organization for Standardization defines robotic autonomous surgery as performing tasks based on current state and senses without human assistance [ 86 87 ]. Shademan et al. [ 88 ] reported that small-bowel anastomosis was performed in pigs using a fully autonomous robot known as Smart Tissue Autonomous Robot (STAR) [ 73 88 89 ]. Ultimately, AI technology will continue to develop and robotic surgery will evolve beyond AI-assisted surgery to autonomous surgery.

AI technology is being actively researched in various fields as a tool for diagnosis, prognosis, predicting treatment response, and decision-making in urological cancers, and it is expected to be applied to clinical practice in the near future. Despite technological, legal, and ethical concerns, AI will change the landscape of urological cancer management.

We searched the PubMed-MEDLINE database from January 2018 to November 2023. We included articles containing both the keywords “urology” and “artificial intelligence” along with one of the following: “machine learning,” “deep learning,” “neural network,” “renal cell carcinoma,” “kidney cancer,” “urothelial carcinoma,” “bladder cancer,” “prostate cancer,” and “robotic surgery.” Considering the recent emerging AI trends in ML and DL, the search period was limited to studies published in the past 5 years. In addition, articles not written in English, review articles, editorial comments, and book chapters were excluded. We retrieved the studies and evaluated their eligibility by screening them according to the inclusion and exclusion criteria. We also screened their references for additional studies. Each study’s inclusion and exclusion criteria were highly heterogeneous, but they all focused on urological cancer. Finally, 58 studies on urological cancers (prostate cancer: 21 [ 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 ], bladder cancer: 20 [ 5 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 ], kidney cancer: 17 [ 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 ]) were identified as shown in Fig. 1 . Furthermore, studies with high impact on the application of AI in general surgery were included. Data including research design, study population, AI algorithm, model, and evaluation metrics of the selected studies were collected.