Attention-based deep learning for accurate cell image analysis

Attention-based deep learning for accurate cell image analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Attention-based deep learning for accurate cell image analysis Lipeng Lai, Xiangrui Gao, Fan Zhang, Xueyu Gao, Mengcheng Yao, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3933549/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract High-content analysis (HCA) holds enormous potential for drug discovery and research, but widely used methods can be cumbersome and yield inaccurate results. Noisy and redundant signals in cell images impede accurate deep learning-based image analysis. To address these issues, we introduce X-Profiler, a novel HCA method that combines cellular experiments, image processing, and deep learning modeling. X-Profiler combines the convolutional neural network and Transformer to encode high-content images, effectively filtering out noisy signals and precisely characterizing cell phenotypes. In comparative tests on drug-induced cardiotoxicity, mitochondrial toxicity classification, and compound classification, X-Profiler outperformed both DeepProfiler and CellProfiler, as two highly recognized and representative methods in this field. Our results demonstrate the utility and versatility of X-Profiler, and we anticipate its wide application in HCA for advancing drug development and disease research. Biological sciences/Computational biology and bioinformatics/High-throughput screening Biological sciences/Drug discovery/Drug screening/High-throughput screening Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Introduction The Developing novel pharmaceutical drugs represents a substantial investment that involves significant amount of time and resources, but with low success rate. The major obstacles hindering drug development include ineffective drug activity, intractable drug toxicity, as well as marketing difficulties 1,2 . Medical research focuses on identifying the causes and deciphering the mechanisms related to a disease, aiming to eventually develop accurate diagnostic tools and effective treatments. In the last decades, high-throughput methods that are able to generate enormous data sets in a short amount of time and in a cost-effective manner 3 . The exploitation of so-called ‘big data’ will enable us to undertake research projects never previously possible but should also stimulate a re-evaluation of all our data practices. The high-throughput and affordability of current technologies such as omics and imaging, allowing quantitative measurements of many putative targets (e.g. DNA, RNA, protein, metabolite), has exponentially increased the volume of scientific data available for this arduous task 4 . Data-driven medicinal chemistry approaches have the potential to improve decision making in drug discovery projects, providing that all researchers embrace the role of ‘data scientist’ and uncover the meaningful relationships and patterns in available data 5 . The success of developing deep learning models heavily relies on the availability of vast amounts of data and advanced algorithms 6 . Massive data helps to train the model effectively without premature overfitting, while the reasonableness and complexity of the network structure endow the model with good memory and information extraction capability 7 . In addition, the diversity of enormous data makes model training and prediction easier by eliminating complex preprocessing steps, such as segmentation, tracking, tracing, and spatial conversion. As a result, the task can be performed through an end-to-end prediction, simplifying the entire process 8 . Cellular morphology is a potentially rich data source for interrogating biological perturbations, especially at a large scale 9–11 . For example, Cellular morphological profiling of compounds has been used to determine their mechanism of action 8,12,13 , identify their targets 14,15 , discover relationships with genes 16,17 , and characterize cellular heterogeneity 18 . Genes have been analyzed by creating profiles of cell populations in which the gene is perturbed by CRISRP and RNA interference (RNAi); these profiles have been used to represents the functional landscape of essential human genes 19–21 and identify genetic interactions 22,23 , or characterize cellular heterogeneity 24 . Nowadays the Joint Undertaking for Morphological Profiling (JUMP)-Cell Painting Consortium released a dataset which contain images and profiles from the Cell Painting assay for over 116,750 unique compounds, over-expression of 12,602 genes, and knockout of 7,975 genes using CRISPR-Cas9 in human osteosarcoma cells (U2OS) 25 . One of the main tools currently used in image processing and interpretable feature extraction is CellProfiler 26,27 , developed by the imaging platform of Broad Institute, which designed to enable biologists without training in computer vision or programming to quantitatively measure phenotypes from thousands of images automatically, a wide variety of measurements can be generated for each identified cell or subcellular compartment, including morphology 28 , intensity, and texture 29 among others. Another feature extraction tool based on deep learning deepprofiler presented by Cimini and Carpenter et al. used weakly supervised learning (WSL) for modeling associations between images and treatments, and showed that it encodes both confounding factors and phenotypic features in the learned representation 30 . Despite the obvious advantages of ‘big data’, the field has to overcome important challenges most of the analyses performed have inherent biologic and experimental errors and rely on capturing a snapshot of complex and dynamic biological systems 31 . Consequently, untimely sample collection due to an incorrect experimental design or simply due to randomness can lead to too much ‘noise’ that would not allow us to clearly identify inciting factors or errors 3 . During cellular phenotypic experiments, the compound effects on cells are random, and the properties of compounds cannot be accurately distinguished from images of one or a few cells 32 . Therefore, a ‘noise’ filter tool was needed to adaptively filter out irrelevant cell-slice images and accurately extract features for cell phenotype characterization. Transformers use a self-attention mechanism to weigh the importance of each token in the context of the entire input sequence. This allows the model to capture long-range dependencies and relationships between tokens 33 . To address the aforementioned issues, we combined the convolutional neural network with Transformer to developed X-Profiler, a deep learning-based method for processing ‘noise’ filter and downstream task related feature extraction, to extract valuable information from high-content images for drug analysis. In this study, We conducted a comprehensive analysis of cell phenotype images, covering aspects from data processing to model training and validation. To evaluate the potential applicability of our method in high-content image processing, we have chosen three widely recognized downstream tasks in drug discovery: Task 1 - determining drug inhibition on hERG (the human Ether-à-go-go-Related Gene) ion channels, Task 2 - predicting drug-induced mitochondrial toxicity, and Task 3 - classifying compounds. In all tasks, X-Profiler has consistently shown performance surpassing the state-of-the-art (SOTA) levels, demonstrating more accurate and effective feature extraction capabilities. Results and Disscussion Model architecture of X-Profiler In this study, we address the issue of high impurity in the dataset by introducing a novel deep learning framework. Instead of expecting each individual slice of the dataset to conform directly to a desired label mapping, we explicitly assign them distinct rankings and weights. Formally, we utilized cell painting data to train and verify our model, segmenting large cellular images into individual slices based on the positional information of cells and constructing intercellular associations by arbitrarily selecting portions of single slices. We introduced the concept of "patch"—bundles of different single cell slice images subjected to the same perturbation. By employing a ResNet to extract image features and utilizing transformer to capture the relational patterns within the same patch, we can identify and exclude poor-quality images, thereby enhancing the model's accuracy. The training and validation processes of the model were shown in Fig. 1 a. In the course of model training and validation, 6 to 12 of single-cell images slices were bundled into a patch and fed into the model in batches. During the training phase, the deep learning model iteratively learns from the training data. In each epoch, which is a full pass through the dataset, the model performs a forward pass to make predictions and then calculates the loss by comparing these predictions to the actual labels. The backward pass follows, where the model uses backpropagation to compute the gradient of the loss with respect to each weight. The optimizer then updates the model's weights in an effort to minimize the loss. After each epoch, the model's performance is evaluated on a separate validation set, which helps monitor its ability to generalize to unseen data. During the prediction phase (Fig. 1 b), we performed random sampling on the images and packaged single-cell images into patches, comprising 10 to 750 units. Subsequent to this, the model engages in forward propagation and employs the output of the linear layer for the score prediction. Synchronously with the acquisition of the model's predictions, we are also presented with a ranking for each individual cell image within the respective patch, culminating the comprehensive data processing endeavor. X-Profiler’s performance on hERG inhibition, Mitochondrial toxicity and compound s classification We designed three tasks to evaluate the capability of X-Profiler. For the hERG inhibition task, we selected 100 compounds (47 toxic and 53 non-toxic) for HCS experiments and generated corresponding high-content images using a high-content imager. We chose 38 compounds (19 toxic and 19 non-toxic) as test dataset for the determination of drug inhibition on hERG ion channels. The high content images were employed to assess the hERG inhibition classification capabilities of the three models: X-Profiler, DeepProfiler, and CellProfiler. Our results showed that X-Profiler outperformed feature extractor DeepProfiler and CellProfiler, achieving an accuracy of 90.6% in cell images classification and the precision, recall and f1 scores for X-Profiler were 80.0%, 95.5% and 86.2%, respectively (Figure. 2a). Moreover, we showed the landscape for compound features using UMAP and two separated cluster for hERG channel inhibitors and non-hERG channel inhibitors were presented. (Figure. 2b). The silhouette score (ss) and adjusted rand score (ars) of clusters between toxicity and non-toxicity compounds was 0.446 and 0.272, respectively, which indicated X-Profiler have the ability to capture the cell phenotype changes affect by hERG channel inhibitors. For the mitochondrial toxicity task, we mapped all 100 compounds with Tox21 dataset annotation 34 . Out of these, 77 compounds bore additional mitochondrial toxicity labels (Supplementary Table 2). We selected a test set of 15 compounds, of which 1 were labeled 'Active', 3 were labeled 'Inactive', and 11 were labeled 'Inconclusive'. Mitochondrial toxicity caused by certain compounds is a widespread form of organ toxicity, which can result in multiorgan damage in the heart, liver, bone, and brain 35–37 . The alterations in cell morphology, texture, and intensity caused by compounds are strongly correlated with mitochondrial toxicity 38 . We evaluated the classification performance of the three models. The X-Profiler model outperforms the other two models in terms of accuracy, recall, precision and f1 score, and the values for X-Profiler are as follows: 54.5%, 52.4%, 47.3% and 43.3%, as showed in Figure. 2c. The landscape for mitochondrial toxicity was showed in Fig. 2 d and the silhouette score and adjusted rand score of clusters between activate, inactivate and inconclusive compounds was 0.023 and 0.083, respectively. For the suboptimal performance of Mitochondrial toxicity, there may be two reasons for this. 1) the among of labeled drugs are decreased and the classification increased, less data implies lower model robustness; 2) the effects of mitochondrial dysfunction are often confusingly cell type-specific, as is the case for the majority of known mitochondrial diseases 39 , the human HepG2 cells was applied for labeling, while iPSC-CMs might exhibit different toxicity reactions to the same compounds. For the compounds classification task, it involved classifying images based on the cell response to different compounds. For the same compound, 20% of the images were used for testing, while 80% of the images were used for training. Our results showed that the X-Profiler model outperformed DeepProfiler and CellProfiler, achieving an accuracy, precision, recall and f1 score of 0.867, 0.892, 0.871 and 0.881, respectively. Apparently, X-Profiler demonstrates superior ability in discerning relationships among images (Fig. 2 e, Fig. 2 f). Comparison of distance and angle for high-dimensional features In our investigation, we quantified the distance and cosine value between the image high-dimensional features of various compounds at differing concentrations (X-Profiler: 512-d; DeepProfiler: 672-d; CellProfiler: 613-d). We intended to enhance our understanding of the model's capacity to discriminate between cellular features post-pharmacological intervention. We determined the cosine similarity of compounds with DMSO, as illustrated in Fig. 3 a. The cosine values spanned from − 1 to 1, indicative of an angular spectrum extending from − 180 degrees to 180 degrees. It was observed that the cosine values of toxic versus non-toxic compounds exhibited distinct angles within a high-dimensional space. Although the Mahalanobis distance of the compounds in relation to DMSO did not reveal a marked variance between toxic and non-toxic compounds, we noted that the Mahalanobis distance for varying concentrations within a single compound performed that the distance value augmented concomitant with concentration and these values were utilized to deduce the IC50 values for each compound (Fig. 3 b). A comparative analysis of X-Profiler with CellProfiler and DeepProfiler is presented in Fig. 3 c. The distribution of the absolute discrepancies between true and predicted IC50 values underscores X-Profiler's superior precision, with a mean and standard deviation for all compounds being 2.66 and 2.94, respectively. In contrast, DeepProfiler and CellProfiler yielded mean values of 4.45 ± 4.84 and 3.48 ± 3.56, respectively. The comparative analysis of IC50 prediction curves for the compound carvedilol by X-Profiler, DeepProfiler, and CellProfiler serves as a compelling exemplar, highlighting X-Profiler's preeminent predictive accuracy in determining IC50 values (Fig. 3 d). Utilizing the calculated feature distances, we inferred the IC50 values of the compounds, thereby demonstrating that X-Profiler has great potential to replace traditional IC50 detection methods. Multiple cell slices enable accurate grasp of the compound property During the feature extraction phase employed by X-Profiler, we obtained a class embedding for a patch of images, referred to as ‘multiple cells features’ (cls embedding), along with the embedding for each individual image in the patch, termed ‘single cells features’. Our research conducted separate analyses on multiple cells and single cells image features in compound classification tasks. Additionally, we compared the landscape of X-Profiler feature with DeepProfiler and CellProfiler. Herein, the multiple cells features aggregate the mean of 100 single cell slice attributes, whereas the single cells features are a stochastic selection from the entirety of single cell slices (Fig. 4 a to Fig. 4 f). We selected the first ten compounds (alphabetically sorted) from a set of 100 compounds for the t-SNE plot illustration and presented compound distance in two-dimensional space. Within our results, we observed that CellProfiler lacks the capability to differentiate changes in cellular phenotype between compounds, regardless of whether they form single cells features clusters or multiple cells features clusters (Fig. 4 e, Fig. 4 f). Additionally, the single cells feature clusters generated by DeepProfiler also failed to distinguish between compounds. (Fig. 4 d). The distance between compounds features from X-Profiler and DeepProfiler is more explicit in the landscape of multiple cell features than in that of single ones. For instance, For instance, the presence of benzethonium chloride and betrixaban compounds is evident within the single cells feature landscape as a cluster (Fig. 4 a), while in landscape of multiple cells, they evolve to form discrete clusters (Fig. 4 b). This suggests that integrating a greater number of image slices can yield more comprehensive features to accurately predict compound properties. Furthermore, in comparison to X-Profiler, the multiple cells features of DeepProfiler merely attain the level of single-cell clustering as exhibited by X-Profiler (Fig. 4 c). We believe that the deployment of convolutional networks within X-Profiler and DeepProfiler might engender more efficacious features for the analysis of cell painting data than traditional statistical algorithms such as CellProfiler, notwithstanding the limited interpretability inherent to these features. Nevertheless, X-Profiler exhibits superior predictive capacity in contrast to DeepProfiler. The incorporation of an attention mechanism within X-Profiler augments its cellular feature aggregation, outperforming DeepProfiler which only calculates the mean of individual cell features. This underscores the importance of leveraging transformer architecture to enhance feature extraction capabilities in intricate and unstructured data. Our study conducted a comparative examination of single cell-slice image from four compounds (specifically bedaquiline, belzutifan, carvedilol, and daclatasvir) as selected from Fig. 4 a. The landscape representation of t-SNE revealed that the cellular embeddings of the drugs belzutifan, carvedilol, and daclatasvir formed a tight cluster, markedly distinct from bedaquiline. Concurrently, the cellular images of the compounds belzutifan, carvedilol, and daclatasvir appeared indistinguishable upon visual inspection (Fig. 4 g), and displayed comparable intensity in the computation of Reactive Oxygen Species (ROS) and Mitochondrial Membrane Potential (MMP) under varying concentrations, a characteristic not shared with the compound bedaquiline (Fig. 4 h). In an endeavor to comprehend the results more profoundly, we researched the mechanism of action (MOA) of the three compounds. Despite diverging therapeutic indications - belzutifan for Von Hippel-Lindau (VHL) syndrome-associated clear-cell renal cell carcinoma (ccRCC), carvedilol for hypertension, and daclatasvir for Chronic hepatitis C genotype 3 (GT-3 HCV) - their MOAs suggest that they may exert similar effects on cardiomyocytes. Belzutifan inhibits hypoxia-inducible factor 2α (HIF-2α), and research suggests that HIF-2α inhibitors can reverse pulmonary hypertension and that there are shared pathophysiologic mechanisms between cancer and heart failure 40,41 . Carvedilol, a nonselective beta-adrenergic antagonist, has anti-free radical and antioxidant effects, and can resist oxidation and reduce ROS production 42 . It further impedes the cardiac Mitochondrial Permeability Transition (MPT), which can depolarize mitochondrial membranes and uncouple oxidative phosphorylation (OXPHOS) 42 . Daclatasvir, a pangenotypic NS5A replication complex inhibitor with a dual antiviral effect, inhibits RNA replication and viral assembly. Cellular ROS levels rise during HCV infection, and evidence suggests that anti-RNA viral drugs are associated with intracellular ROS levels 43,44 . Given that cell phenotype reflects the proper classification of drug mechanisms, the application of deep learning holds substantial promise for future endeavors in drug discovery and therapeutic indication investigations. X-Profiler can act as an effective cell-slice filter To showcase the capability of X-Profiler in distilling the most salient single-cell images, we generated an attention heatmap using X-Profiler, which highlighted the cells pertinent to the task throughout the entirety of the image concerning the cardiotoxicity task (Supplementary Fig. 1a and Supplementary Fig. 1b). As indicated by the self-attention mechanism, the similarity between the embeddings of single cell-slice image and the embeddings of the CLS token is positively associated with the classification task 45 . In the heatmap, cells that are more relevant to the downstream task are represented by brighter and warmer pink dots, and these dots clearly illustrate the differences between different cells in terms of task correlation. In assessing the Transformer's capacity to segregate specific cell slices, we analyzed the attributes of task-relevant single-cell slice images alongside those deemed irrelevant. We presented the landscape of relevant and irrelevant single cells image features in Fig. 5 a, in which the dots symbolize task-related images, while inverted triangles represent those unrelated to the task. The single cells image features unrelated to the downstream tasks were indistinguishable among multiple compounds, while the single cells image features related to tasks were separately clustered and widely spaced apart in accordance with their respective compounds. This demonstrates that the multiple cell slices encoder can operate as a cell filter. Two ancillary observations emerged from this clustering exercise: 1) In cellular experiments involving various compounds with distinct properties, there are often both dead cells and unaffected normal cells that form a common group and do not contribute to classification. After passing through the single-cell slice encoder, the features of these cells are similar; 2) Upon exposure to different compounds, the cell phenotypes undergo more significant changes, making them markedly distinct from each other. To further demonstrate the screening capabilities of X-Profiler for cell phenotype, we compared cell-slice images that were considered relevant and irrelevant based on the X-Profiler attention map (Fig. 5 b). Each cell-slice image was comprised of three channels: nucleus (DNA), mitochondrial membrane potential (MMP) and mitochondrial reactive oxygen species (ROS). Task relevant images exhibited more accurate staining, higher fluorescence intensity, greater contrast between channels compared to the task irrelevant images. Moreover, relevant images exhibited more distinct cell structures in the ROS and MMP channels, leading to more accurate cell identification. In terms of image quality, task relevant images were lucid and free of impurities, in stark contrast to the irrelevant images, which exhibited a halo on the image surface caused by optical structural artifacts, resulting in unclear images and high noise levels. Cell phenotype images are distinct from general images in two key ways. Primarily, they contain a high degree of redundancy, with dark backgrounds and bright cells dominating most of the image. As a result, it can be challenging to distinguish differences between cells using generic training. Secondly, cell phenotype images are often besieged with noise, resulting from experimental manipulations and batch-to-batch variability, necessitating varied concentration gradients and experimental duplicates to mitigate stochastic influences. Prevalent models and processing techniques have struggled to address these challenges, often yielding poor results. Consequently, the attention mechanism employed in X-Profiler enables the extraction of highly informative, accurate, and structurally intact single-cell slice images, while filtering out low-informative, erroneous, and poor-quality images. This selective extraction has precipitated an increase in the model's prowess following image preprocessing. Thanks to the advancement of current high-throughput and high-content techniques, enormous biological data are generated every moment to help us gain better understanding of lifes. However, significant noise accompanies the true signal in these data that are generated. For example, cell painting technique allows us to stain various organelles simultaneously, but the capability to use these images to establish biologically relevant models is largely affected by the algorithm we use to distinguish the information from the noise. Current methods, in general, treat every slice of the data, such as each single cell image, equally, and have their limitations in fully utilizing the high content data. Here we propose a deep learning model that has the transformer architecture at its core to effectively assign different weights on different pieces of data. By doing this, we are able to build models that pay more attention to high quality data and less attention to low quality data, which equivalently suppress the affection of noise. The results in this stuty showed that by putting different attentions on different pieces of data, our models achieve better performances in the earlier parts of this study. It is also noted the same approach can be applied to more broader areas to improve the performance of current approaches. Model parameter selection Numerous parameters can influence the performance of a model, such as epochs, training batches, learning rate, optimizers, and transformer attention layers, to name a few. In this segment of our research, we evaluated a selection of critical parameters, excluding subjective human expertise. Initially, we explored the variable of patch numbers, a key element of our study, by conducting trials during the training and validation phases with patch numbers ranging from 8 to 12, and during the prediction phase with a broader range from 10 to 750. Our findings indicated that the optimal patch number for training and validation was 8, while during the prediction phase, 100 proved to be most effective. This was evidenced by the superior performance metrics, including accuracy, area under the curve (AUC), and recall, which peaked when the training and validation patch number were set to 8 and demonstrated a plateau in improvement as the prediction patch number reached 100 (Fig. 6 a). Regarding the consumption of computing resources, the quantity of patches used during the training and validation phases exhibited no tangible impact on the duration of processing or GPU utilization. Nevertheless, a discernible improvement in resource usage was correlated with an increase in the number of patches designated for the prediction phase (Fig. 6 b, Fig. 6 c). Additionally, our study encompassed trials concerning the batch size and the attention layers. The outcomes revealed that the batch size bore no significant consequence on the model's performance and the optimal configuration for the attention layers was determined to be three (Supplementary Table 3). Conclusion and Future directions Compared to DeepProfiler and CellProfiler, X-Profiler is a novel HCA process that leverages both our understanding of cellular images and advanced deep learning algorithms. It excels in screening and utilizing informative and high-quality cell-slice images to extract task-relevant cellular features, resulting in superior performance in three drug property-related tasks: 1) determination of drug inhibition on hERG ion channels, 2) prediction of drug-induced mitochondrial toxicity, and 3) classification of compounds. In addition, X-Profiler can concatenate different compounds based on similar cell phenotypes, highlighting its potential in the field of drug repurposing and indication expansion. In conclusion, HCS is an emerging field that is still rapidly evolving in terms of experimental implementation and analytical methods and has the potential to address diverse biological problems. The key to achieving credible results in downstream tasks lies in the ability of analytical tools or computational models to select useful parts from a large number of cell images of varying quality. Among the tested solutions for downstream tasks, CellProfiler and DeepProfiler performed mediocrely, while X-Profiler showed promising results. This is because X-Profiler, which leverages deep learning to use multiple cell slices for model training and verification simultaneously, can accurately capture key information and eliminate noise interference. Additionally, X-Profiler can distinguish between cells with similar phenotypes but perturbed by different compounds, suggesting many possibilities for downstream applications. Overall, the conclusion emphasizes the potential of X-Profiler as a valuable tool for high-content cell image analysis, which can contribute to various fields such as drug discovery and development, disease diagnosis, and personalized medicine. In the next phase of our research, we plan to expand the application of X-Profiler to more cell phenotyping tasks and high-content images induced by other perturbations, such as RNA interference (RNAi) or CRISPR Cas9. Our goal is to demonstrate the versatility and effectiveness of our model in analyzing various types of cellular image data. However, we also acknowledge that algorithms based on cell phenotype images have limitations, particularly when the changes in cell phenotype induced by a compound are minimal, which may make it difficult for the model to accurately assess its toxicity and other characteristics. To address this issue, we intend to explore the integration of other high-level information sources, such as transcriptome data or videos of cells captured under brightfield conditions, using multimodal learning techniques. We believe that incorporating multiple sources of information will enhance the accuracy and robustness of our model, enabling it to tackle a wider array of biological problems. Methods Constructing the cardiotoxicity high-content image data To obtain an appropriate image dataset for the hERG inhibition task, we selected 100 compounds for HCS experiments and generated corresponding high-content images using a high-content imager. Supplementary Table 2 provides detailed information for each compound, including plate name, drug name, hERG IC50 value range, and cardiotoxicity label. We first downloaded all compounds with hERG IC50 values using the python API interface of the chEMBL Database and subsequently selected 100 compounds by deduplicating and filtering drug molecules with ambiguous hERG IC50 values. The compounds were then labeled as cardiotoxic and non-cardiotoxic based on their hERG IC50 values using a threshold of 10 µM (less than or equal to 10 µM for toxic labeling and greater than 10 µM for nontoxic labeling). This resulted in 47 cardiotoxic and 53 non-cardiotoxic compounds, on which we performed cellular experiments using hiPSC-CMs. The hiPSC-CMs at day 30 after cardiac induction were cryopreserved as Cauliscell hiPSC-CMs (Cauliscell Company, Nanjing, China) and thawed in a 37°C water bath with gentle shaking. After centrifugation and counting, the cells were added to 384-well plates precoated with 10 µg/ml recombinant human vitronectin at 12,000 cells/well (Cauliscell) in cardiomyocyte plating medium (Cauliscell). After 24 hours, the volume was replaced with cardiomyocyte maintenance medium, which was changed every other day. Once the cells started to beat rhythmically, we added the compounds, which were selected from the FDA-approved compound library (FDA-approved drug library, MedChemExpress) for cardiotoxicity testing. The working concentrations of each compound were 10 µM, 3.33 µM, 1.11 µM, 0.37 µM, 0.12 µM, and 0.04 µM, with 3 replicate instances set for each compound at each concentration. The control group was set with 0.1% DMSO (Sigma), and the blank control was set with water. After incubating the cells with compounds for 72 hours, working concentration dyes of CM-H2DCFDA (Thermo) at 5 µM, TMRM (Thermo) at 20 nM, and Hoechst33342 (Thermo) at 5 µg/ml were prepared with maintenance medium (Cauliscell) and added to the cells for 30 min. The cells were washed with HBSS (Beyotime Biotechnology), and YOYO-3 (Thermo) was added at a final concentration of 2 nM, followed by incubation at room temperature for 10 min and washing with HBSS (Beyotime Biotechnology). Finally, the 384-well plate (PerkinElmer) was placed on a high-content imager (Operetta CLS, PerkinElmer), and images were taken with a 20x water objective in the confocal model with 9 fields of view for one well, corresponding to the filter parameters shown in Supplementary Table 1. Constructing the mitochondrial toxicity high-content image data For the mitochondrial toxicity task, we utilized open-source high-content images from Bray et al. [ 20 ]. A total of fifty-five plate high-content images with numerous compounds were downloaded, and the mitochondrial toxicity labels, obtained from the PubChem Database ( https://pubchem.ncbi.nlm.nih.gov/assay/pcget.cgi?query=download&record_type=datatable&actvty=all&response_type=save&aid=720637 ), were assigned to each compound, and the assay is qHTS assay for small molecule disruptors of the mitochondrial membrane potential from tox21 program. CellProfiler and DeepProfiler data processing For the cardiotoxicity and mitochondrial toxicity high-content images, we extracted cell phenotype features using standard procedures of CellProfiler (version 4.2.4). Subsequently, we utilized the machine learning model called LGBMClassifier from scikit-learn to perform classification tasks on toxicity data, after dealing with cell phenotype features. Once the locations of single cells were identified in the images by CellProfiler, we inputted the images and locations to DeepProfiler (version 0.3.1). During classification tasks, we employed a pretrained model named EfficientNet, which was deployed within DeepProfiler, and calculated embeddings using DeepProfiler. The CellProfiler-LightGBM and DeepProfiler results were evaluated and compared with X-Profiler to assess their accuracy and generalizability. In all three models (X-Profiler, DeepProfiler and CellProfiler-LightGBM), we partitioned the 285 compounds labeled with mitochondrial toxicity into a training set comprising 190 compounds and a test set containing 95 compounds. For the 100 cardiotoxic compounds, the training and testing sets contained 68 and 32 compounds, respectively. ROS and MMP Measurement To measure ROS and M MP, we utilized cell phenotype features calculated from our high-content images of 100 compounds, which were used for cardiotoxicity assessment. We designed two CellProfiler analysis protocols that can automatically detect and quantify fluorescence intensity, which proved to be useful for analyzing large image datasets. The ROS and MMP measurements were obtained from the suppressed fluorescent channels of the H2DCFDA and TMRM cell-based assay kit. We computed a reduced dataset with the well-mean feature vector per well, followed by normalizing all features by subtracting the mean of each plate layout from each feature. X-Profiler model Design X-Profiler is a plug-and-play flexible model framework that is not restricted to a particular backbone. The model processes cell-slice images by normalizing and grouping them by compound and concentration. First, single cell locations are extracted from CellProfiler by calculating the center coordinate of the nucleus. Second, images are rescaled with the global fluorescence intensity, and third, DeepProfiler crops cell slices from 3 channels based on the x and y coordinates for the center of a single nucleus. In all experiments, cell slices were cropped from a region of 96 x 96 pixels centered on the nucleus without resizing. The resulting 96 x 288-pixel images were preserved for model construction. Cell-slice images stained with different dyes were integrated as different channels into a single cell-slice multichannel image. This approach reduces noise interference and allows the model to perform the classification task using a few representative cells. Furthermore, the training phase inputs a limited number of cell images at a time to prevent the model from overfitting prematurely, which brought by complex information of too many cell images. The X-Profiler model consists of three main parts: a single cell-slice encoder, a multiple cell slices encoder, and a classifier. The single cell-slice encoder uses a uniform CNN network (ResNet18) to obtain an unbiased latent representation of single cell-slice image and improve the generalization of the latent representation. The multiple cell slices encoder is built by Transformer, which has strong contextualization capabilities to integrate information from each element in the sequence. A CLS token is added to the top of the sequence to filter out irrelevant cell representations and improve model robustness. The self-attentive mechanism of Transformer is used to fuse the latent representations corresponding to the CLS token, which are then fed into the classifier constructed by a linear layer for classification. The overall flow of prediction is illustrated in Fig. 1 . X-Profiler Training and Inference During the training process, we used the standard supervised training method with cross-entropy as the loss function, along with learning rate warm-up and cosine decay techniques. In order to account for various experimental and computational factors, as well as potential inaccuracies in the labels themselves, we also employed label smoothing to reduce the model's confidence. In the testing phase, we made a slight modification to the training approach. While in training we randomly selected 8 cell-slice images to form multiple cell-slices sequences, in testing we increased the number of cell slices included in each sequence, the number of each sequence were 12, 50, 100, 150, 200, 250, 300, 350, 400 450, 500, 550, 600, 650, 700, 750, respectively. In order to take into account the performance of the model and the consumption of the calculation at the same time, we randomly sampled 100 sets of multiple cell slices and computed the average classification result from those sequences as the final outcome. This approach offers the advantage of improving the chances of selecting valuable cell-slice images while also reducing computational costs by not evaluating all individual cells. The optimized X-Profiler models were trained up to 50 epochs. To prevent an overfitting problem, dropout and adamw regularization methods were used. Model training was implemented using pytorch (1.12.0 + cu116). A workstation with an Intel(R) Xeon(R) Gold 5320 (104 core processors, 2.2 GHz) CPU and NVIDIA A100 graphics-processing unit was used for model training. X-Profiler attention map As we utilized the CLS token embeddings generated by Transformer for classification purposes, we obtained the attention map by calculating the dot product between the query representation and key representation among the tokens in Transformer. By applying the Softmax function, each element of the attention map was assigned a value ranging from 0 to 1. Cluster and statistical analysis To evaluate the performance of feature extraction methods for cell-slice image analysis, we employed the t-SNE algorithm in the python sklearn package to reduce the dimensionality of features or embeddings computed by X-Profiler, CellProfiler, and DeepProfiler. We then examined the resulting component distributions to investigate the relationship between X-Profiler and the cell phenotype observed in images, as well as the effectiveness of X-Profiler compared to CellProfiler or DeepProfiler in capturing useful information. To assess the clustering performance, we used the silhouette score and adjusted random score method in sklearn to calculate the silhouette coefficient. We applied the Kruskal‒Wallis test to test the null hypothesis that the MMP or ROS intensity of images treated with compound at six different concentrations (0.04 µM, 0.12 µM, 0.37 µM, 0.11 µM, 3.33 µM, and 10 µM) were equal, setting significance at p < 0.05. Declarations Competing interests The authors declare that they have no conflict of interest. Author contributions Xiangrui Guo, Xiaodong Wang and Lipeng Lai conceived this research. Fan Zhang, Mengcheng Yao and Xueyu Guo curated the dataset. Xiangrui Gao, Fan Zhang and Xueyu Guo performed data analysis. Xiangrui.Gao devised deep learning algorithms. Xiaoxiao Wang ang Dong Chen conducted the HCS experiments. Xiangrui Gao, Xueyu Guo, Fan Zhang, Mengcheng Yao and Xiaoxiao Wang wrote and modified the paper. Xiaodong Wang and Lipeng Lai supervised this work. References Ashburn, T.T., Thor, K.B.: Drug repositioning: identifying and developing new uses for existing drugs. Nat. Rev. Drug Discovery. 3 , 673–683 (2004) Ni, J., et al.: Obstacles and opportunities in Chinese pharmaceutical innovation. Globalization health. 13 , 1–9 (2017) Perakakis, N., Yazdani, A., Karniadakis, G.E., Mantzoros, C.: Vol. 87 A1–A9 (Elsevier, (2018) Paananen, J., Fortino, V.: An omics perspective on drug target discovery platforms. Brief. Bioinform. 21 , 1937–1953 (2019). 10.1093/bib/bbz122 Lusher, S.J., McGuire, R., van Schaik, R.C., Nicholson, C.D., de Vlieg, J.: Data-driven medicinal chemistry in the era of big data. Drug discovery today. 19 , 859–868 (2014) Mehonic, A., et al.: Memristors—From in-memory computing, deep learning acceleration, and spiking neural networks to the future of neuromorphic and bio‐inspired computing. Adv. Intell. Syst. 2 , 2000085 (2020) Chen, H.: Machine learning for information retrieval: Neural networks, symbolic learning, and genetic algorithms. J. Am. Soc. Inform. Sci. 46 , 194–216 (1995) Tian, G., Harrison, P.J., Sreenivasan, A.P., Carreras-Puigvert, J., Spjuth, O.: Combining molecular and Cell Painting image data for mechanism of action prediction. Artif. Intell. Life Sci. 3 , 100060 (2023) Bickle, M.: The beautiful cell: high-content screening in drug discovery. Anal. Bioanal. Chem. 398 , 219–226 (2010) Perlman, Z.E., et al.: Multidimensional drug profiling by automated microscopy. Science. 306 , 1194–1198 (2004) Snijder, B., Pelkmans, L.: Origins of regulated cell-to-cell variability. Nat. Rev. Mol. Cell Biol. 12 , 119–125 (2011) Ljosa, V., et al.: Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J. BioMol. Screen. 18 , 1321–1329 (2013) Young, D.W., et al.: Integrating high-content screening and ligand-target prediction to identify mechanism of action. Nat. Chem. Biol. 4 , 59–68 (2008) Sundaramurthy, V., et al.: Integration of chemical and RNAi multiparametric profiles identifies triggers of intracellular mycobacterial killing. Cell. host microbe. 13 , 129–142 (2013) Akbarzadeh, M., et al.: Morphological profiling by means of the Cell Painting assay enables identification of tubulin-targeting compounds. Cell Chemical Biology 29, 1053–1064. e1053 (2022) Castoreno, A.B., et al.: Small molecules discovered in a pathway screen target the Rho pathway in cytokinesis. Nat. Chem. Biol. 6 , 457–463 (2010) Lejal, V., Cerisier, N., Rouquié, D., Taboureau, O.: Assessment of Drug-Induced Liver Injury through Cell Morphology and Gene Expression Analysis. Chem. Res. Toxicol. 36 , 1456–1470 (2023) Loo, L.-H., et al.: An approach for extensibly profiling the molecular states of cellular subpopulations. Nat. Methods. 6 , 759–765 (2009) Feldman, D., et al.: Pooled genetic perturbation screens with image-based phenotypes. Nat. Protoc. 17 , 476–512 (2022) Feldman, D., et al.: Optical pooled screens in human cells. Cell. 179 , 787–799 (2019) Funk, L., et al.: The phenotypic landscape of essential human genes. Cell. 185 , 4634–4653 (2022) Liberali, P., Snijder, B., Pelkmans, L.: A hierarchical map of regulatory genetic interactions in membrane trafficking. Cell. 157 , 1473–1487 (2014) Fischer, B., et al.: A map of directional genetic interactions in a metazoan cell. Elife. 4 , e05464 (2015) Yin, Z., et al.: A screen for morphological complexity identifies regulators of switch-like transitions between discrete cell shapes. Nat. Cell Biol. 15 , 860–871 (2013) Chandrasekaran, S.N., et al.: JUMP Cell Painting dataset: morphological impact of 136,000 chemical and genetic perturbations. bioRxiv , 2003. 2023.534023 (2023). (2023) Stirling, D.R., et al.: CellProfiler 4: improvements in speed, utility and usability. BMC Bioinform. 22 , 1–11 (2021) Dao, D., et al.: CellProfiler Analyst: interactive data exploration, analysis and classification of large biological image sets. Bioinformatics. 32 , 3210–3212 (2016) Rocha, L., Velho, L., Carvalho, P.C.P.: in Proceedings. XV Brazilian Symposium on Computer Graphics and Image Processing. 99–105 (IEEE) Haralick, R.M., Shanmugam, K., Dinstein, I.H.: Textural features for image classification. IEEE Trans. Syst. man. cybernetics, 610–621 (1973) Moshkov, N., et al.: Learning representations for image-based profiling of perturbations. Biorxiv , 2022.2008. 503783 (2022). (2012) Alber, M., et al.: Integrating machine learning and multiscale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences. NPJ Digit. Med. 2 , 115 (2019) Loo, L.-H., Wu, L.F., Altschuler, S.J.: Image-based multivariate profiling of drug responses from single cells. Nat. Methods. 4 , 445–453 (2007) Vaswani, A., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30 (2017) Richard, A.M., et al.: The Tox21 10K compound library: collaborative chemistry advancing toxicology. Chem. Res. Toxicol. 34 , 189–216 (2020) Begriche, K., Massart, J., Robin, M.-A., Borgne-Sanchez, A., Fromenty, B.: Drug-induced toxicity on mitochondria and lipid metabolism: mechanistic diversity and deleterious consequences for the liver. J. Hepatol. 54 , 773–794 (2011) Hargreaves, I.P., Al Shahrani, M., Wainwright, L., Heales, S.J.: Drug-induced mitochondrial toxicity. Drug Saf. 39 , 661–674 (2016) Tang, X., Wang, Z., Hu, S., Zhou, B.: Assessing drug-induced mitochondrial toxicity in cardiomyocytes: implications for preclinical cardiac safety evaluation. Pharmaceutics. 14 , 1313 (2022) Haghighi, M., Caicedo, J.C., Cimini, B.A., Carpenter, A.E., Singh, S.: High-dimensional gene expression and morphology profiles of cells across 28,000 genetic and chemical perturbations. Nat. Methods. 19 , 1550–1557 (2022) Meyer, J.N., Hartman, J.H., Mello, D.F.: Mitochondrial Toxicity. Toxicol. Sci. 162 , 15–23 (2018). 10.1093/toxsci/kfy008 de Wit, S., Glen, C., de Boer, R.A.: Lang, N. N. Mechanisms shared between cancer, heart failure, and targeted anti-cancer therapies. Cardiovascular. Res. 118 , 3451–3466 (2022) Niemann, B., et al.: Caloric restriction reduces sympathetic activity similar to beta-blockers but conveys additional mitochondrio-protective effects in aged myocardium. Sci. Rep. 11 , 1931 (2021) Pereira, C.: Drug-induced cardiac mitochondrial toxicity and protection: from doxorubicin to carvedilol. Curr. Pharm. Design. 17 , 2113–2129 (2011) Lozano-Sepúlveda, S.A., Rincón-Sanchez, A.R., Rivas-Estilla, A.M.: Antioxidants benefits in hepatitis C infection in the new DAAs era. Ann. Hepatol. 18 , 410–415 (2019) Reshi, M.L., Su, Y.-C., Hong, J.-R.: RNA viruses: ROS-mediated cell death. Int. J. cell. biology 2014 (2014) Ghosh, M.C., et al.: Therapeutic inhibition of HIF-2α reverses polycythemia and pulmonary hypertension in murine models of human diseases. Blood J. Am. Soc. Hematol. 137 , 2509–2519 (2021) Additional Declarations There is NO Competing Interest. Supplementary Files SupplFig1.png Supplementary Figure 1: Attention heatmap of the cell painting image treated with alfuzosin. The colored solid dots marked cells are increasingly relevant to the attention score in the images with the changes gradually from dark blue to bright pink. SupplementaryTable.xlsx rs.pdf Reporting Summary Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3933549","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":275189954,"identity":"82f241db-91e7-4622-95a1-eb8addea315c","order_by":0,"name":"Lipeng Lai","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIie3RvQrCMBDA8ZPAuVzaNVDRV6gU1MGHqbiKCC4ODoLQLuLuY4gvkBJwE1fHquAsCCIqYv3CrekomP9yBPKDHAEwmX4wi71mARiTyawB6gi+CQFDHySIDAQ+BMh9Em2Y59G2cwayQzo6+75oW8DWm1Xqw6ymNxkDCcVnrpyLLgJ6XiuVUMXho+Rhik9jiaIRAKGjIdXLg5QUxVLespEKo1Oyu6JcHAWZiNV0+EBQWaHnLsbJLkyzi20vogNd68XiUu1E71hvl8LheptGnj+SC77f4QNLvf4mcP0efS0wmUymv+sOxeA7isjGM3AAAAAASUVORK5CYII=","orcid":"","institution":"XtalPi","correspondingAuthor":true,"prefix":"","firstName":"Lipeng","middleName":"","lastName":"Lai","suffix":""},{"id":275189955,"identity":"5dbb6ddb-ac0f-4bee-8b18-e0d9291e5ed0","order_by":1,"name":"Xiangrui Gao","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xiangrui","middleName":"","lastName":"Gao","suffix":""},{"id":275189956,"identity":"ea5caf09-4882-4f94-97f6-1fd9b7c0f57e","order_by":2,"name":"Fan Zhang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Fan","middleName":"","lastName":"Zhang","suffix":""},{"id":275189957,"identity":"38cf7413-f7dd-420e-9b91-c8493207e2f9","order_by":3,"name":"Xueyu Gao","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xueyu","middleName":"","lastName":"Gao","suffix":""},{"id":275189958,"identity":"848dc246-4b0f-4ba3-9d85-5eb7ebfb9af0","order_by":4,"name":"Mengcheng Yao","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Mengcheng","middleName":"","lastName":"Yao","suffix":""},{"id":275189959,"identity":"6153e45a-a7d3-4fce-bd55-6718a5b9ab86","order_by":5,"name":"Xiaoxiao Wang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xiaoxiao","middleName":"","lastName":"Wang","suffix":""},{"id":275189960,"identity":"4eec2655-ef78-4d3e-a074-1c23006ced85","order_by":6,"name":"Chen Dong","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Chen","middleName":"","lastName":"Dong","suffix":""},{"id":275189961,"identity":"f62f88d5-0dd1-4abb-b678-ac25a496d5cc","order_by":7,"name":"Genwei Zhang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Genwei","middleName":"","lastName":"Zhang","suffix":""},{"id":275189962,"identity":"f1681470-7df9-46ea-a876-842fa519733f","order_by":8,"name":"Xiaodong Wang","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Xiaodong","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2024-02-06 10:23:26","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3933549/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3933549/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":51823674,"identity":"ad39a56a-c54e-4a8b-a035-4f46bb8c9253","added_by":"auto","created_at":"2024-02-29 16:30:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":57566,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThe architecture of the X-Profiler model\u003c/strong\u003e. \u003cstrong\u003e(a)\u003c/strong\u003e The pipline of model training and validation. \u003cstrong\u003e(b)\u003c/strong\u003e The pipline of model prediction.\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/49b1ce140c0faf1cf9586348.png"},{"id":51823673,"identity":"bda2c779-974f-4bb2-81fc-c57feb65559b","added_by":"auto","created_at":"2024-02-29 16:30:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":81937,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThe performance of the X-Profiler on three tasks\u003c/strong\u003e. \u003cstrong\u003e(a)\u003c/strong\u003e The Model’s performance on hERG inhibition classification; \u003cstrong\u003e(b)\u003c/strong\u003e The landscape of X-Profiler feature on hERG inhibition classification task using t-SNE. \u003cstrong\u003e(c) \u003c/strong\u003eThe Model’s performance on mitochondrial toxicity classification. \u003cstrong\u003e(d)\u003c/strong\u003e The landscape of X-Profiler feature on mitochondrial toxicity classification task using t-SNE. \u003cstrong\u003e(e)\u003c/strong\u003e The Model’s performance on compounds classification.\u003cstrong\u003e (f)\u003c/strong\u003e The landscape of X-Profiler feature on five random selected compounds.\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/cab3aa94f9c9a8f4f9cfdcf5.png"},{"id":51823672,"identity":"89296398-a8e7-49d1-bc7e-63fde07b0ea1","added_by":"auto","created_at":"2024-02-29 16:30:13","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":31179,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThe feature distance and angle characterize a compound's toxicity and its dose-dependency. (a, b) \u003c/strong\u003eThe feature cosine value and mahalanobis distance for compounds when compared with DMSO. The x-axis corresponds to the compounds, while the y-axis denotes the cosine value. Circular dots represent toxic compounds, and upward triangles represent non-toxic compounds. The colors green, blue, yellow, purple, brown, and red respectively correspond to compound concentrations of 0.04µM, 0.12µM, 0.36µM, 1.11µM, 3.33µM, and 10.00µM. \u003cstrong\u003e(c)\u003c/strong\u003eThe frequency distribution histogram of the absolute differences between true IC50 values and their predicted counterparts. From left to right, the sequence is as follows: X-Profiler, DeepProfiler, and CellProfiler, with the predicted IC50 values derived from the mahalanobis distance calculation. \u003cstrong\u003e(d)\u003c/strong\u003e The fitted curves for the IC50 values of the compound carvedilol. From left to right, the sequence is as follows: X-Profiler, DeepProfiler, and CellProfiler. The x-axis represents the logarithmic values of concentration, with the green and red vertical dashed line indicating the actual IC50 value and predicted IC50 value, respectively.\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/0a5e9fef9f631f2e271f05a5.png"},{"id":51823676,"identity":"1a2808a2-8441-44db-9a0f-936efd71dc14","added_by":"auto","created_at":"2024-02-29 16:30:13","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":160772,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFeature landscape of single cell features and multiple cell features\u003c/strong\u003e. \u003cstrong\u003e(a)\u003c/strong\u003eThe feature landscape of multiple cell features images using the X-Profiler model, each point represents a batch of 100 cell-slice images.\u003cstrong\u003e (b)\u003c/strong\u003e The feature landscape of single cell features\u003cstrong\u003e \u003c/strong\u003eusing the X-Profiler model, each point represents a single cell-slice image. \u003cstrong\u003e(c)\u003c/strong\u003e The feature landscape of multiple cell features using DeepProfiler, each point represents average value of 100 single cell-slice images’ feature which were random seleted.\u003cstrong\u003e (d)\u003c/strong\u003e The feature landscape of single cell features\u003cstrong\u003e \u003c/strong\u003eusing the DeepProfiler, each point represents a single cell-slice image and images were random seleted. \u003cstrong\u003e(e)\u003c/strong\u003e The feature landscape of multiple cell features using the CellProfiler model, each point represents average value of 100 single cell-slice images’ feature which were random seleted. (\u003cstrong\u003ef) \u003c/strong\u003eThe feature landscape of single cell features\u003cstrong\u003e \u003c/strong\u003eimages using the CellProfiler model, each point represents a single cell-slice image and images were random seleted.\u003cstrong\u003e(g)\u003c/strong\u003e The single cell-slice images of four compounds. For the four compounds (Bedaquiline, belzutifan, carvedilol and daclatasvir), each compound has one row and three columns of images. \u003cstrong\u003e(h)\u003c/strong\u003e The box plot of MMP and ROS intensity of belzutifan, carvedilol, daclatasvir and betrixaban. ROS: reactive oxygen species. MMP: mitochondrial membrane potential.\u003c/p\u003e","description":"","filename":"image4.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/31da784e9f1bce1de9656dce.png"},{"id":51823677,"identity":"014898d0-184e-4280-b0e5-0166a2ae054b","added_by":"auto","created_at":"2024-02-29 16:30:14","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":136662,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eX-Profiler can act as a filter to identify cell slices that are relevant to a downstream classification task.\u003c/strong\u003e \u003cstrong\u003e(a) \u003c/strong\u003eThe landscpe map of X-Profiler embeddings of relevant (shape in dot) and irrelevant (shape in lower triangle) cell-slice images of 5 random selected compounds. \u003cstrong\u003e(b) \u003c/strong\u003eTask\u003cstrong\u003e \u003c/strong\u003erelevant (left) and irrelevant (right) images of compounds belzutifan.\u003c/p\u003e","description":"","filename":"image5.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/98d500e06725034fd8469830.png"},{"id":51823678,"identity":"1d663228-383b-4979-a9f5-fe9abefdd178","added_by":"auto","created_at":"2024-02-29 16:30:14","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":18295,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eX-Profiler performance and resource utilization. (a) \u003c/strong\u003eThe model's performance across various patch numbers, encompassing both its training and prediction processes. \u003cstrong\u003e(b)\u003c/strong\u003eThe computational time usage of the model across different patch numbers. \u003cstrong\u003e(c)\u003c/strong\u003eThe GPU utilization of the model across different patch numbers.\u003c/p\u003e","description":"","filename":"image6.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/51d52d9d6132f4d11b29fc96.png"},{"id":55041724,"identity":"b5ab93e1-eeec-4e1e-9130-7e2e63b6a766","added_by":"auto","created_at":"2024-04-21 08:18:20","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":940312,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/904aa448-2350-4347-8d7d-68569c42ebf1.pdf"},{"id":51823680,"identity":"e0bc67c7-7086-49ae-9a0e-c865dc03df17","added_by":"auto","created_at":"2024-02-29 16:30:14","extension":"png","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":8766876,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSupplementary Figure 1: \u003c/strong\u003eAttention heatmap of the cell painting image treated with alfuzosin. The colored solid dots marked cells are increasingly relevant to the attention score in the images with the changes gradually from dark blue to bright pink.\u003c/p\u003e","description":"","filename":"SupplFig1.png","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/0c21b62623c0f87e1364a4db.png"},{"id":51823675,"identity":"5ee9b1d2-71ed-48a0-b93e-d689b2bb8ddf","added_by":"auto","created_at":"2024-02-29 16:30:13","extension":"xlsx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":15833,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryTable.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/0760f4fbde27848576dea7a6.xlsx"},{"id":51823679,"identity":"556cd0d6-5e64-4484-a987-5ac5ac535a85","added_by":"auto","created_at":"2024-02-29 16:30:14","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":1573466,"visible":true,"origin":"","legend":"Reporting Summary","description":"","filename":"rs.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3933549/v1/24e7a228f12fb64e5d1f5202.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Attention-based deep learning for accurate cell image analysis","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe Developing novel pharmaceutical drugs represents a substantial investment that involves significant amount of time and resources, but with low success rate. The major obstacles hindering drug development include ineffective drug activity, intractable drug toxicity, as well as marketing difficulties\u003csup\u003e1,2\u003c/sup\u003e. Medical research focuses on identifying the causes and deciphering the mechanisms related to a disease, aiming to eventually develop accurate diagnostic tools and effective treatments. In the last decades, high-throughput methods that are able to generate enormous data sets in a short amount of time and in a cost-effective manner\u003csup\u003e3\u003c/sup\u003e. The exploitation of so-called \u0026lsquo;big data\u0026rsquo; will enable us to undertake research projects never previously possible but should also stimulate a re-evaluation of all our data practices. The high-throughput and affordability of current technologies such as omics and imaging, allowing quantitative measurements of many putative targets (e.g. DNA, RNA, protein, metabolite), has exponentially increased the volume of scientific data available for this arduous task\u003csup\u003e4\u003c/sup\u003e. Data-driven medicinal chemistry approaches have the potential to improve decision making in drug discovery projects, providing that all researchers embrace the role of \u0026lsquo;data scientist\u0026rsquo; and uncover the meaningful relationships and patterns in available data\u003csup\u003e5\u003c/sup\u003e. The success of developing deep learning models heavily relies on the availability of vast amounts of data and advanced algorithms\u003csup\u003e6\u003c/sup\u003e. Massive data helps to train the model effectively without premature overfitting, while the reasonableness and complexity of the network structure endow the model with good memory and information extraction capability\u003csup\u003e7\u003c/sup\u003e. In addition, the diversity of enormous data makes model training and prediction easier by eliminating complex preprocessing steps, such as segmentation, tracking, tracing, and spatial conversion. As a result, the task can be performed through an end-to-end prediction, simplifying the entire process\u003csup\u003e8\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eCellular morphology is a potentially rich data source for interrogating biological perturbations, especially at a large scale\u003csup\u003e9\u0026ndash;11\u003c/sup\u003e. For example, Cellular morphological profiling of compounds has been used to determine their mechanism of action\u003csup\u003e8,12,13\u003c/sup\u003e, identify their targets\u003csup\u003e14,15\u003c/sup\u003e, discover relationships with genes\u003csup\u003e16,17\u003c/sup\u003e, and characterize cellular heterogeneity\u003csup\u003e18\u003c/sup\u003e. Genes have been analyzed by creating profiles of cell populations in which the gene is perturbed by CRISRP and RNA interference (RNAi); these profiles have been used to represents the functional landscape of essential human genes\u003csup\u003e19\u0026ndash;21\u003c/sup\u003e and identify genetic interactions\u003csup\u003e22,23\u003c/sup\u003e, or characterize cellular heterogeneity\u003csup\u003e24\u003c/sup\u003e. Nowadays the Joint Undertaking for Morphological Profiling (JUMP)-Cell Painting Consortium released a dataset which contain images and profiles from the Cell Painting assay for over 116,750 unique compounds, over-expression of 12,602 genes, and knockout of 7,975 genes using CRISPR-Cas9 in human osteosarcoma cells (U2OS)\u003csup\u003e25\u003c/sup\u003e. One of the main tools currently used in image processing and interpretable feature extraction is CellProfiler\u003csup\u003e26,27\u003c/sup\u003e, developed by the imaging platform of Broad Institute, which designed to enable biologists without training in computer vision or programming to quantitatively measure phenotypes from thousands of images automatically, a wide variety of measurements can be generated for each identified cell or subcellular compartment, including morphology\u003csup\u003e28\u003c/sup\u003e, intensity, and texture\u003csup\u003e29\u003c/sup\u003e among others. Another feature extraction tool based on deep learning deepprofiler presented by Cimini and Carpenter et al. used weakly supervised learning (WSL) for modeling associations between images and treatments, and showed that it encodes both confounding factors and phenotypic features in the learned representation\u003csup\u003e30\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eDespite the obvious advantages of \u0026lsquo;big data\u0026rsquo;, the field has to overcome important challenges most of the analyses performed have inherent biologic and experimental errors and rely on capturing a snapshot of complex and dynamic biological systems\u003csup\u003e31\u003c/sup\u003e. Consequently, untimely sample collection due to an incorrect experimental design or simply due to randomness can lead to too much \u0026lsquo;noise\u0026rsquo; that would not allow us to clearly identify inciting factors or errors\u003csup\u003e3\u003c/sup\u003e. During cellular phenotypic experiments, the compound effects on cells are random, and the properties of compounds cannot be accurately distinguished from images of one or a few cells\u003csup\u003e32\u003c/sup\u003e. Therefore, a \u0026lsquo;noise\u0026rsquo; filter tool was needed to adaptively filter out irrelevant cell-slice images and accurately extract features for cell phenotype characterization. Transformers use a self-attention mechanism to weigh the importance of each token in the context of the entire input sequence. This allows the model to capture long-range dependencies and relationships between tokens\u003csup\u003e33\u003c/sup\u003e. To address the aforementioned issues, we combined the convolutional neural network with Transformer to developed X-Profiler, a deep learning-based method for processing \u0026lsquo;noise\u0026rsquo; filter and downstream task related feature extraction, to extract valuable information from high-content images for drug analysis.\u003c/p\u003e \u003cp\u003eIn this study, We conducted a comprehensive analysis of cell phenotype images, covering aspects from data processing to model training and validation. To evaluate the potential applicability of our method in high-content image processing, we have chosen three widely recognized downstream tasks in drug discovery: Task 1 - determining drug inhibition on hERG (the human Ether-\u0026agrave;-go-go-Related Gene) ion channels, Task 2 - predicting drug-induced mitochondrial toxicity, and Task 3 - classifying compounds. In all tasks, X-Profiler has consistently shown performance surpassing the state-of-the-art (SOTA) levels, demonstrating more accurate and effective feature extraction capabilities.\u003c/p\u003e"},{"header":"Results and Disscussion","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eModel architecture of X-Profiler\u003c/h2\u003e \u003cp\u003eIn this study, we address the issue of high impurity in the dataset by introducing a novel deep learning framework. Instead of expecting each individual slice of the dataset to conform directly to a desired label mapping, we explicitly assign them distinct rankings and weights. Formally, we utilized cell painting data to train and verify our model, segmenting large cellular images into individual slices based on the positional information of cells and constructing intercellular associations by arbitrarily selecting portions of single slices. We introduced the concept of \"patch\"\u0026mdash;bundles of different single cell slice images subjected to the same perturbation. By employing a ResNet to extract image features and utilizing transformer to capture the relational patterns within the same patch, we can identify and exclude poor-quality images, thereby enhancing the model's accuracy. The training and validation processes of the model were shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea. In the course of model training and validation, 6 to 12 of single-cell images slices were bundled into a patch and fed into the model in batches. During the training phase, the deep learning model iteratively learns from the training data. In each epoch, which is a full pass through the dataset, the model performs a forward pass to make predictions and then calculates the loss by comparing these predictions to the actual labels. The backward pass follows, where the model uses backpropagation to compute the gradient of the loss with respect to each weight. The optimizer then updates the model's weights in an effort to minimize the loss. After each epoch, the model's performance is evaluated on a separate validation set, which helps monitor its ability to generalize to unseen data. During the prediction phase (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb), we performed random sampling on the images and packaged single-cell images into patches, comprising 10 to 750 units. Subsequent to this, the model engages in forward propagation and employs the output of the linear layer for the score prediction. Synchronously with the acquisition of the model's predictions, we are also presented with a ranking for each individual cell image within the respective patch, culminating the comprehensive data processing endeavor.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eX-Profiler\u0026rsquo;s performance on hERG inhibition, Mitochondrial toxicity and compound\u003c/b\u003es \u003cb\u003eclassification\u003c/b\u003e\u003c/p\u003e \u003cp\u003eWe designed three tasks to evaluate the capability of X-Profiler. For the hERG inhibition task, we selected 100 compounds (47 toxic and 53 non-toxic) for HCS experiments and generated corresponding high-content images using a high-content imager. We chose 38 compounds (19 toxic and 19 non-toxic) as test dataset for the determination of drug inhibition on hERG ion channels. The high content images were employed to assess the hERG inhibition classification capabilities of the three models: X-Profiler, DeepProfiler, and CellProfiler. Our results showed that X-Profiler outperformed feature extractor DeepProfiler and CellProfiler, achieving an accuracy of 90.6% in cell images classification and the precision, recall and f1 scores for X-Profiler were 80.0%, 95.5% and 86.2%, respectively (Figure. 2a). Moreover, we showed the landscape for compound features using UMAP and two separated cluster for hERG channel inhibitors and non-hERG channel inhibitors were presented. (Figure. 2b). The silhouette score (ss) and adjusted rand score (ars) of clusters between toxicity and non-toxicity compounds was 0.446 and 0.272, respectively, which indicated X-Profiler have the ability to capture the cell phenotype changes affect by hERG channel inhibitors.\u003c/p\u003e \u003cp\u003eFor the mitochondrial toxicity task, we mapped all 100 compounds with Tox21 dataset annotation\u003csup\u003e34\u003c/sup\u003e. Out of these, 77 compounds bore additional mitochondrial toxicity labels (Supplementary Table\u0026nbsp;2). We selected a test set of 15 compounds, of which 1 were labeled 'Active', 3 were labeled 'Inactive', and 11 were labeled 'Inconclusive'. Mitochondrial toxicity caused by certain compounds is a widespread form of organ toxicity, which can result in multiorgan damage in the heart, liver, bone, and brain\u003csup\u003e35\u0026ndash;37\u003c/sup\u003e. The alterations in cell morphology, texture, and intensity caused by compounds are strongly correlated with mitochondrial toxicity\u003csup\u003e38\u003c/sup\u003e. We evaluated the classification performance of the three models. The X-Profiler model outperforms the other two models in terms of accuracy, recall, precision and f1 score, and the values for X-Profiler are as follows: 54.5%, 52.4%, 47.3% and 43.3%, as showed in Figure. 2c. The landscape for mitochondrial toxicity was showed in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ed and the silhouette score and adjusted rand score of clusters between activate, inactivate and inconclusive compounds was 0.023 and 0.083, respectively. For the suboptimal performance of Mitochondrial toxicity, there may be two reasons for this. 1) the among of labeled drugs are decreased and the classification increased, less data implies lower model robustness; 2) the effects of mitochondrial dysfunction are often confusingly cell type-specific, as is the case for the majority of known mitochondrial diseases\u003csup\u003e39\u003c/sup\u003e, the human HepG2 cells was applied for labeling, while iPSC-CMs might exhibit different toxicity reactions to the same compounds.\u003c/p\u003e \u003cp\u003eFor the compounds classification task, it involved classifying images based on the cell response to different compounds. For the same compound, 20% of the images were used for testing, while 80% of the images were used for training. Our results showed that the X-Profiler model outperformed DeepProfiler and CellProfiler, achieving an accuracy, precision, recall and f1 score of 0.867, 0.892, 0.871 and 0.881, respectively. Apparently, X-Profiler demonstrates superior ability in discerning relationships among images (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ee, Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eComparison of distance and angle for high-dimensional features\u003c/h2\u003e \u003cp\u003eIn our investigation, we quantified the distance and cosine value between the image high-dimensional features of various compounds at differing concentrations (X-Profiler: 512-d; DeepProfiler: 672-d; CellProfiler: 613-d). We intended to enhance our understanding of the model's capacity to discriminate between cellular features post-pharmacological intervention. We determined the cosine similarity of compounds with DMSO, as illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea. The cosine values spanned from \u0026minus;\u0026thinsp;1 to 1, indicative of an angular spectrum extending from \u0026minus;\u0026thinsp;180 degrees to 180 degrees. It was observed that the cosine values of toxic versus non-toxic compounds exhibited distinct angles within a high-dimensional space. Although the Mahalanobis distance of the compounds in relation to DMSO did not reveal a marked variance between toxic and non-toxic compounds, we noted that the Mahalanobis distance for varying concentrations within a single compound performed that the distance value augmented concomitant with concentration and these values were utilized to deduce the IC50 values for each compound (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb). A comparative analysis of X-Profiler with CellProfiler and DeepProfiler is presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec. The distribution of the absolute discrepancies between true and predicted IC50 values underscores X-Profiler's superior precision, with a mean and standard deviation for all compounds being 2.66 and 2.94, respectively. In contrast, DeepProfiler and CellProfiler yielded mean values of 4.45\u0026thinsp;\u0026plusmn;\u0026thinsp;4.84 and 3.48\u0026thinsp;\u0026plusmn;\u0026thinsp;3.56, respectively. The comparative analysis of IC50 prediction curves for the compound carvedilol by X-Profiler, DeepProfiler, and CellProfiler serves as a compelling exemplar, highlighting X-Profiler's preeminent predictive accuracy in determining IC50 values (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed). Utilizing the calculated feature distances, we inferred the IC50 values of the compounds, thereby demonstrating that X-Profiler has great potential to replace traditional IC50 detection methods.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eMultiple cell slices enable accurate grasp of the compound property\u003c/h2\u003e \u003cp\u003eDuring the feature extraction phase employed by X-Profiler, we obtained a class embedding for a patch of images, referred to as \u0026lsquo;multiple cells features\u0026rsquo; (cls embedding), along with the embedding for each individual image in the patch, termed \u0026lsquo;single cells features\u0026rsquo;. Our research conducted separate analyses on multiple cells and single cells image features in compound classification tasks. Additionally, we compared the landscape of X-Profiler feature with DeepProfiler and CellProfiler. Herein, the multiple cells features aggregate the mean of 100 single cell slice attributes, whereas the single cells features are a stochastic selection from the entirety of single cell slices (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea to Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ef). We selected the first ten compounds (alphabetically sorted) from a set of 100 compounds for the t-SNE plot illustration and presented compound distance in two-dimensional space.\u003c/p\u003e \u003cp\u003eWithin our results, we observed that CellProfiler lacks the capability to differentiate changes in cellular phenotype between compounds, regardless of whether they form single cells features clusters or multiple cells features clusters (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ee, Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ef). Additionally, the single cells feature clusters generated by DeepProfiler also failed to distinguish between compounds. (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ed). The distance between compounds features from X-Profiler and DeepProfiler is more explicit in the landscape of multiple cell features than in that of single ones. For instance, For instance, the presence of benzethonium chloride and betrixaban compounds is evident within the single cells feature landscape as a cluster (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea), while in landscape of multiple cells, they evolve to form discrete clusters (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb). This suggests that integrating a greater number of image slices can yield more comprehensive features to accurately predict compound properties. Furthermore, in comparison to X-Profiler, the multiple cells features of DeepProfiler merely attain the level of single-cell clustering as exhibited by X-Profiler (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec). We believe that the deployment of convolutional networks within X-Profiler and DeepProfiler might engender more efficacious features for the analysis of cell painting data than traditional statistical algorithms such as CellProfiler, notwithstanding the limited interpretability inherent to these features. Nevertheless, X-Profiler exhibits superior predictive capacity in contrast to DeepProfiler. The incorporation of an attention mechanism within X-Profiler augments its cellular feature aggregation, outperforming DeepProfiler which only calculates the mean of individual cell features. This underscores the importance of leveraging transformer architecture to enhance feature extraction capabilities in intricate and unstructured data.\u003c/p\u003e \u003cp\u003eOur study conducted a comparative examination of single cell-slice image from four compounds (specifically bedaquiline, belzutifan, carvedilol, and daclatasvir) as selected from Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea. The landscape representation of t-SNE revealed that the cellular embeddings of the drugs belzutifan, carvedilol, and daclatasvir formed a tight cluster, markedly distinct from bedaquiline. Concurrently, the cellular images of the compounds belzutifan, carvedilol, and daclatasvir appeared indistinguishable upon visual inspection (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eg), and displayed comparable intensity in the computation of Reactive Oxygen Species (ROS) and Mitochondrial Membrane Potential (MMP) under varying concentrations, a characteristic not shared with the compound bedaquiline (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eh).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIn an endeavor to comprehend the results more profoundly, we researched the mechanism of action (MOA) of the three compounds. Despite diverging therapeutic indications - belzutifan for Von Hippel-Lindau (VHL) syndrome-associated clear-cell renal cell carcinoma (ccRCC), carvedilol for hypertension, and daclatasvir for Chronic hepatitis C genotype 3 (GT-3 HCV) - their MOAs suggest that they may exert similar effects on cardiomyocytes. Belzutifan inhibits hypoxia-inducible factor 2α (HIF-2α), and research suggests that HIF-2α inhibitors can reverse pulmonary hypertension and that there are shared pathophysiologic mechanisms between cancer and heart failure\u003csup\u003e40,41\u003c/sup\u003e. Carvedilol, a nonselective beta-adrenergic antagonist, has anti-free radical and antioxidant effects, and can resist oxidation and reduce ROS production\u003csup\u003e42\u003c/sup\u003e. It further impedes the cardiac Mitochondrial Permeability Transition (MPT), which can depolarize mitochondrial membranes and uncouple oxidative phosphorylation (OXPHOS)\u003csup\u003e42\u003c/sup\u003e. Daclatasvir, a pangenotypic NS5A replication complex inhibitor with a dual antiviral effect, inhibits RNA replication and viral assembly. Cellular ROS levels rise during HCV infection, and evidence suggests that anti-RNA viral drugs are associated with intracellular ROS levels\u003csup\u003e43,44\u003c/sup\u003e. Given that cell phenotype reflects the proper classification of drug mechanisms, the application of deep learning holds substantial promise for future endeavors in drug discovery and therapeutic indication investigations.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eX-Profiler can act as an effective cell-slice filter\u003c/h2\u003e \u003cp\u003eTo showcase the capability of X-Profiler in distilling the most salient single-cell images, we generated an attention heatmap using X-Profiler, which highlighted the cells pertinent to the task throughout the entirety of the image concerning the cardiotoxicity task (Supplementary Fig.\u0026nbsp;1a and Supplementary Fig.\u0026nbsp;1b). As indicated by the self-attention mechanism, the similarity between the embeddings of single cell-slice image and the embeddings of the CLS token is positively associated with the classification task\u003csup\u003e45\u003c/sup\u003e. In the heatmap, cells that are more relevant to the downstream task are represented by brighter and warmer pink dots, and these dots clearly illustrate the differences between different cells in terms of task correlation. In assessing the Transformer's capacity to segregate specific cell slices, we analyzed the attributes of task-relevant single-cell slice images alongside those deemed irrelevant. We presented the landscape of relevant and irrelevant single cells image features in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea, in which the dots symbolize task-related images, while inverted triangles represent those unrelated to the task. The single cells image features unrelated to the downstream tasks were indistinguishable among multiple compounds, while the single cells image features related to tasks were separately clustered and widely spaced apart in accordance with their respective compounds. This demonstrates that the multiple cell slices encoder can operate as a cell filter. Two ancillary observations emerged from this clustering exercise: 1) In cellular experiments involving various compounds with distinct properties, there are often both dead cells and unaffected normal cells that form a common group and do not contribute to classification. After passing through the single-cell slice encoder, the features of these cells are similar; 2) Upon exposure to different compounds, the cell phenotypes undergo more significant changes, making them markedly distinct from each other.\u003c/p\u003e \u003cp\u003eTo further demonstrate the screening capabilities of X-Profiler for cell phenotype, we compared cell-slice images that were considered relevant and irrelevant based on the X-Profiler attention map (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb). Each cell-slice image was comprised of three channels: nucleus (DNA), mitochondrial membrane potential (MMP) and mitochondrial reactive oxygen species (ROS). Task relevant images exhibited more accurate staining, higher fluorescence intensity, greater contrast between channels compared to the task irrelevant images. Moreover, relevant images exhibited more distinct cell structures in the ROS and MMP channels, leading to more accurate cell identification. In terms of image quality, task relevant images were lucid and free of impurities, in stark contrast to the irrelevant images, which exhibited a halo on the image surface caused by optical structural artifacts, resulting in unclear images and high noise levels.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eCell phenotype images are distinct from general images in two key ways. Primarily, they contain a high degree of redundancy, with dark backgrounds and bright cells dominating most of the image. As a result, it can be challenging to distinguish differences between cells using generic training. Secondly, cell phenotype images are often besieged with noise, resulting from experimental manipulations and batch-to-batch variability, necessitating varied concentration gradients and experimental duplicates to mitigate stochastic influences. Prevalent models and processing techniques have struggled to address these challenges, often yielding poor results. Consequently, the attention mechanism employed in X-Profiler enables the extraction of highly informative, accurate, and structurally intact single-cell slice images, while filtering out low-informative, erroneous, and poor-quality images. This selective extraction has precipitated an increase in the model's prowess following image preprocessing.\u003c/p\u003e \u003cp\u003eThanks to the advancement of current high-throughput and high-content techniques, enormous biological data are generated every moment to help us gain better understanding of lifes. However, significant noise accompanies the true signal in these data that are generated. For example, cell painting technique allows us to stain various organelles simultaneously, but the capability to use these images to establish biologically relevant models is largely affected by the algorithm we use to distinguish the information from the noise. Current methods, in general, treat every slice of the data, such as each single cell image, equally, and have their limitations in fully utilizing the high content data. Here we propose a deep learning model that has the transformer architecture at its core to effectively assign different weights on different pieces of data. By doing this, we are able to build models that pay more attention to high quality data and less attention to low quality data, which equivalently suppress the affection of noise. The results in this stuty showed that by putting different attentions on different pieces of data, our models achieve better performances in the earlier parts of this study. It is also noted the same approach can be applied to more broader areas to improve the performance of current approaches.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eModel parameter selection\u003c/h2\u003e \u003cp\u003eNumerous parameters can influence the performance of a model, such as epochs, training batches, learning rate, optimizers, and transformer attention layers, to name a few. In this segment of our research, we evaluated a selection of critical parameters, excluding subjective human expertise. Initially, we explored the variable of patch numbers, a key element of our study, by conducting trials during the training and validation phases with patch numbers ranging from 8 to 12, and during the prediction phase with a broader range from 10 to 750. Our findings indicated that the optimal patch number for training and validation was 8, while during the prediction phase, 100 proved to be most effective. This was evidenced by the superior performance metrics, including accuracy, area under the curve (AUC), and recall, which peaked when the training and validation patch number were set to 8 and demonstrated a plateau in improvement as the prediction patch number reached 100 (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ea). Regarding the consumption of computing resources, the quantity of patches used during the training and validation phases exhibited no tangible impact on the duration of processing or GPU utilization. Nevertheless, a discernible improvement in resource usage was correlated with an increase in the number of patches designated for the prediction phase (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003eb, Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec). Additionally, our study encompassed trials concerning the batch size and the attention layers. The outcomes revealed that the batch size bore no significant consequence on the model's performance and the optimal configuration for the attention layers was determined to be three (Supplementary Table\u0026nbsp;3).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion and Future directions","content":"\u003cp\u003eCompared to DeepProfiler and CellProfiler, X-Profiler is a novel HCA process that leverages both our understanding of cellular images and advanced deep learning algorithms. It excels in screening and utilizing informative and high-quality cell-slice images to extract task-relevant cellular features, resulting in superior performance in three drug property-related tasks: 1) determination of drug inhibition on hERG ion channels, 2) prediction of drug-induced mitochondrial toxicity, and 3) classification of compounds. In addition, X-Profiler can concatenate different compounds based on similar cell phenotypes, highlighting its potential in the field of drug repurposing and indication expansion.\u003c/p\u003e \u003cp\u003eIn conclusion, HCS is an emerging field that is still rapidly evolving in terms of experimental implementation and analytical methods and has the potential to address diverse biological problems. The key to achieving credible results in downstream tasks lies in the ability of analytical tools or computational models to select useful parts from a large number of cell images of varying quality. Among the tested solutions for downstream tasks, CellProfiler and DeepProfiler performed mediocrely, while X-Profiler showed promising results. This is because X-Profiler, which leverages deep learning to use multiple cell slices for model training and verification simultaneously, can accurately capture key information and eliminate noise interference. Additionally, X-Profiler can distinguish between cells with similar phenotypes but perturbed by different compounds, suggesting many possibilities for downstream applications. Overall, the conclusion emphasizes the potential of X-Profiler as a valuable tool for high-content cell image analysis, which can contribute to various fields such as drug discovery and development, disease diagnosis, and personalized medicine.\u003c/p\u003e \u003cp\u003eIn the next phase of our research, we plan to expand the application of X-Profiler to more cell phenotyping tasks and high-content images induced by other perturbations, such as RNA interference (RNAi) or CRISPR Cas9. Our goal is to demonstrate the versatility and effectiveness of our model in analyzing various types of cellular image data. However, we also acknowledge that algorithms based on cell phenotype images have limitations, particularly when the changes in cell phenotype induced by a compound are minimal, which may make it difficult for the model to accurately assess its toxicity and other characteristics. To address this issue, we intend to explore the integration of other high-level information sources, such as transcriptome data or videos of cells captured under brightfield conditions, using multimodal learning techniques. We believe that incorporating multiple sources of information will enhance the accuracy and robustness of our model, enabling it to tackle a wider array of biological problems.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eConstructing the cardiotoxicity high-content image data\u003c/h2\u003e \u003cp\u003eTo obtain an appropriate image dataset for the hERG inhibition task, we selected 100 compounds for HCS experiments and generated corresponding high-content images using a high-content imager. Supplementary Table\u0026nbsp;2 provides detailed information for each compound, including plate name, drug name, hERG IC50 value range, and cardiotoxicity label. We first downloaded all compounds with hERG IC50 values using the python API interface of the chEMBL Database and subsequently selected 100 compounds by deduplicating and filtering drug molecules with ambiguous hERG IC50 values. The compounds were then labeled as cardiotoxic and non-cardiotoxic based on their hERG IC50 values using a threshold of 10 \u0026micro;M (less than or equal to 10 \u0026micro;M for toxic labeling and greater than 10 \u0026micro;M for nontoxic labeling). This resulted in 47 cardiotoxic and 53 non-cardiotoxic compounds, on which we performed cellular experiments using hiPSC-CMs.\u003c/p\u003e \u003cp\u003eThe hiPSC-CMs at day 30 after cardiac induction were cryopreserved as Cauliscell hiPSC-CMs (Cauliscell Company, Nanjing, China) and thawed in a 37\u0026deg;C water bath with gentle shaking. After centrifugation and counting, the cells were added to 384-well plates precoated with 10 \u0026micro;g/ml recombinant human vitronectin at 12,000 cells/well (Cauliscell) in cardiomyocyte plating medium (Cauliscell). After 24 hours, the volume was replaced with cardiomyocyte maintenance medium, which was changed every other day. Once the cells started to beat rhythmically, we added the compounds, which were selected from the FDA-approved compound library (FDA-approved drug library, MedChemExpress) for cardiotoxicity testing. The working concentrations of each compound were 10 \u0026micro;M, 3.33 \u0026micro;M, 1.11 \u0026micro;M, 0.37 \u0026micro;M, 0.12 \u0026micro;M, and 0.04 \u0026micro;M, with 3 replicate instances set for each compound at each concentration. The control group was set with 0.1% DMSO (Sigma), and the blank control was set with water. After incubating the cells with compounds for 72 hours, working concentration dyes of CM-H2DCFDA (Thermo) at 5 \u0026micro;M, TMRM (Thermo) at 20 nM, and Hoechst33342 (Thermo) at 5 \u0026micro;g/ml were prepared with maintenance medium (Cauliscell) and added to the cells for 30 min. The cells were washed with HBSS (Beyotime Biotechnology), and YOYO-3 (Thermo) was added at a final concentration of 2 nM, followed by incubation at room temperature for 10 min and washing with HBSS (Beyotime Biotechnology). Finally, the 384-well plate (PerkinElmer) was placed on a high-content imager (Operetta CLS, PerkinElmer), and images were taken with a 20x water objective in the confocal model with 9 fields of view for one well, corresponding to the filter parameters shown in Supplementary Table\u0026nbsp;1.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eConstructing the mitochondrial toxicity high-content image data\u003c/h2\u003e \u003cp\u003eFor the mitochondrial toxicity task, we utilized open-source high-content images from Bray et al. [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. A total of fifty-five plate high-content images with numerous compounds were downloaded, and the mitochondrial toxicity labels, obtained from the PubChem Database (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://pubchem.ncbi.nlm.nih.gov/assay/pcget.cgi?query=download\u0026amp;record_type=datatable\u0026amp;actvty=all\u0026amp;response_type=save\u0026amp;aid=720637\u003c/span\u003e\u003cspan address=\"https://pubchem.ncbi.nlm.nih.gov/assay/pcget.cgi?query=download\u0026amp;record_type=datatable\u0026amp;actvty=all\u0026amp;response_type=save\u0026amp;aid=720637\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e), were assigned to each compound, and the assay is qHTS assay for small molecule disruptors of the mitochondrial membrane potential from tox21 program.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eCellProfiler and DeepProfiler data processing\u003c/h2\u003e \u003cp\u003eFor the cardiotoxicity and mitochondrial toxicity high-content images, we extracted cell phenotype features using standard procedures of CellProfiler (version 4.2.4). Subsequently, we utilized the machine learning model called LGBMClassifier from scikit-learn to perform classification tasks on toxicity data, after dealing with cell phenotype features. Once the locations of single cells were identified in the images by CellProfiler, we inputted the images and locations to DeepProfiler (version 0.3.1). During classification tasks, we employed a pretrained model named EfficientNet, which was deployed within DeepProfiler, and calculated embeddings using DeepProfiler. The CellProfiler-LightGBM and DeepProfiler results were evaluated and compared with X-Profiler to assess their accuracy and generalizability. In all three models (X-Profiler, DeepProfiler and CellProfiler-LightGBM), we partitioned the 285 compounds labeled with mitochondrial toxicity into a training set comprising 190 compounds and a test set containing 95 compounds. For the 100 cardiotoxic compounds, the training and testing sets contained 68 and 32 compounds, respectively.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eROS and MMP Measurement\u003c/h2\u003e \u003cp\u003eTo measure ROS and M MP, we utilized cell phenotype features calculated from our high-content images of 100 compounds, which were used for cardiotoxicity assessment. We designed two CellProfiler analysis protocols that can automatically detect and quantify fluorescence intensity, which proved to be useful for analyzing large image datasets. The ROS and MMP measurements were obtained from the suppressed fluorescent channels of the H2DCFDA and TMRM cell-based assay kit. We computed a reduced dataset with the well-mean feature vector per well, followed by normalizing all features by subtracting the mean of each plate layout from each feature.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eX-Profiler model Design\u003c/h2\u003e \u003cp\u003eX-Profiler is a plug-and-play flexible model framework that is not restricted to a particular backbone. The model processes cell-slice images by normalizing and grouping them by compound and concentration. First, single cell locations are extracted from CellProfiler by calculating the center coordinate of the nucleus. Second, images are rescaled with the global fluorescence intensity, and third, DeepProfiler crops cell slices from 3 channels based on the x and y coordinates for the center of a single nucleus. In all experiments, cell slices were cropped from a region of 96 x 96 pixels centered on the nucleus without resizing. The resulting 96 x 288-pixel images were preserved for model construction.\u003c/p\u003e \u003cp\u003eCell-slice images stained with different dyes were integrated as different channels into a single cell-slice multichannel image. This approach reduces noise interference and allows the model to perform the classification task using a few representative cells. Furthermore, the training phase inputs a limited number of cell images at a time to prevent the model from overfitting prematurely, which brought by complex information of too many cell images.\u003c/p\u003e \u003cp\u003eThe X-Profiler model consists of three main parts: a single cell-slice encoder, a multiple cell slices encoder, and a classifier. The single cell-slice encoder uses a uniform CNN network (ResNet18) to obtain an unbiased latent representation of single cell-slice image and improve the generalization of the latent representation. The multiple cell slices encoder is built by Transformer, which has strong contextualization capabilities to integrate information from each element in the sequence. A CLS token is added to the top of the sequence to filter out irrelevant cell representations and improve model robustness. The self-attentive mechanism of Transformer is used to fuse the latent representations corresponding to the CLS token, which are then fed into the classifier constructed by a linear layer for classification. The overall flow of prediction is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003eX-Profiler Training and Inference\u003c/h2\u003e \u003cp\u003eDuring the training process, we used the standard supervised training method with cross-entropy as the loss function, along with learning rate warm-up and cosine decay techniques. In order to account for various experimental and computational factors, as well as potential inaccuracies in the labels themselves, we also employed label smoothing to reduce the model's confidence.\u003c/p\u003e \u003cp\u003eIn the testing phase, we made a slight modification to the training approach. While in training we randomly selected 8 cell-slice images to form multiple cell-slices sequences, in testing we increased the number of cell slices included in each sequence, the number of each sequence were 12, 50, 100, 150, 200, 250, 300, 350, 400 450, 500, 550, 600, 650, 700, 750, respectively. In order to take into account the performance of the model and the consumption of the calculation at the same time, we randomly sampled 100 sets of multiple cell slices and computed the average classification result from those sequences as the final outcome. This approach offers the advantage of improving the chances of selecting valuable cell-slice images while also reducing computational costs by not evaluating all individual cells. The optimized X-Profiler models were trained up to 50 epochs. To prevent an overfitting problem, dropout and adamw regularization methods were used. Model training was implemented using pytorch (1.12.0\u0026thinsp;+\u0026thinsp;cu116). A workstation with an Intel(R) Xeon(R) Gold 5320 (104 core processors, 2.2 GHz) CPU and NVIDIA A100 graphics-processing unit was used for model training.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003eX-Profiler attention map\u003c/h2\u003e \u003cp\u003eAs we utilized the CLS token embeddings generated by Transformer for classification purposes, we obtained the attention map by calculating the dot product between the query representation and key representation among the tokens in Transformer. By applying the Softmax function, each element of the attention map was assigned a value ranging from 0 to 1.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section2\"\u003e \u003ch2\u003eCluster and statistical analysis\u003c/h2\u003e \u003cp\u003eTo evaluate the performance of feature extraction methods for cell-slice image analysis, we employed the t-SNE algorithm in the python sklearn package to reduce the dimensionality of features or embeddings computed by X-Profiler, CellProfiler, and DeepProfiler. We then examined the resulting component distributions to investigate the relationship between X-Profiler and the cell phenotype observed in images, as well as the effectiveness of X-Profiler compared to CellProfiler or DeepProfiler in capturing useful information. To assess the clustering performance, we used the silhouette score and adjusted random score method in sklearn to calculate the silhouette coefficient. We applied the Kruskal‒Wallis test to test the null hypothesis that the MMP or ROS intensity of images treated with compound at six different concentrations (0.04 \u0026micro;M, 0.12 \u0026micro;M, 0.37 \u0026micro;M, 0.11 \u0026micro;M, 3.33 \u0026micro;M, and 10 \u0026micro;M) were equal, setting significance at p\u0026thinsp;\u0026lt;\u0026thinsp;0.05.\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no conflict of interest.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eXiangrui Guo, Xiaodong Wang and Lipeng Lai conceived this research. Fan Zhang, Mengcheng Yao and Xueyu Guo curated the dataset. Xiangrui Gao, Fan Zhang and Xueyu Guo performed data analysis. Xiangrui.Gao devised deep learning algorithms. Xiaoxiao Wang ang Dong Chen conducted the HCS experiments. Xiangrui Gao, Xueyu Guo, Fan Zhang, Mengcheng Yao and Xiaoxiao Wang wrote and modified the paper. Xiaodong Wang and Lipeng Lai supervised this work.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAshburn, T.T., Thor, K.B.: Drug repositioning: identifying and developing new uses for existing drugs. Nat. Rev. Drug Discovery. \u003cb\u003e3\u003c/b\u003e, 673\u0026ndash;683 (2004)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNi, J., et al.: Obstacles and opportunities in Chinese pharmaceutical innovation. Globalization health. \u003cb\u003e13\u003c/b\u003e, 1\u0026ndash;9 (2017)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePerakakis, N., Yazdani, A., Karniadakis, G.E., Mantzoros, C.: Vol. 87 A1\u0026ndash;A9 (Elsevier, (2018)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePaananen, J., Fortino, V.: An omics perspective on drug target discovery platforms. Brief. Bioinform. \u003cb\u003e21\u003c/b\u003e, 1937\u0026ndash;1953 (2019). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/bib/bbz122\u003c/span\u003e\u003cspan address=\"10.1093/bib/bbz122\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLusher, S.J., McGuire, R., van Schaik, R.C., Nicholson, C.D., de Vlieg, J.: Data-driven medicinal chemistry in the era of big data. Drug discovery today. \u003cb\u003e19\u003c/b\u003e, 859\u0026ndash;868 (2014)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMehonic, A., et al.: Memristors\u0026mdash;From in-memory computing, deep learning acceleration, and spiking neural networks to the future of neuromorphic and bio‐inspired computing. Adv. Intell. Syst. \u003cb\u003e2\u003c/b\u003e, 2000085 (2020)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, H.: Machine learning for information retrieval: Neural networks, symbolic learning, and genetic algorithms. J. Am. Soc. Inform. Sci. \u003cb\u003e46\u003c/b\u003e, 194\u0026ndash;216 (1995)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTian, G., Harrison, P.J., Sreenivasan, A.P., Carreras-Puigvert, J., Spjuth, O.: Combining molecular and Cell Painting image data for mechanism of action prediction. Artif. Intell. Life Sci. \u003cb\u003e3\u003c/b\u003e, 100060 (2023)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBickle, M.: The beautiful cell: high-content screening in drug discovery. Anal. Bioanal. Chem. \u003cb\u003e398\u003c/b\u003e, 219\u0026ndash;226 (2010)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePerlman, Z.E., et al.: Multidimensional drug profiling by automated microscopy. Science. \u003cb\u003e306\u003c/b\u003e, 1194\u0026ndash;1198 (2004)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSnijder, B., Pelkmans, L.: Origins of regulated cell-to-cell variability. Nat. Rev. Mol. Cell Biol. \u003cb\u003e12\u003c/b\u003e, 119\u0026ndash;125 (2011)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLjosa, V., et al.: Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J. BioMol. Screen. \u003cb\u003e18\u003c/b\u003e, 1321\u0026ndash;1329 (2013)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYoung, D.W., et al.: Integrating high-content screening and ligand-target prediction to identify mechanism of action. Nat. Chem. Biol. \u003cb\u003e4\u003c/b\u003e, 59\u0026ndash;68 (2008)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSundaramurthy, V., et al.: Integration of chemical and RNAi multiparametric profiles identifies triggers of intracellular mycobacterial killing. Cell. host microbe. \u003cb\u003e13\u003c/b\u003e, 129\u0026ndash;142 (2013)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAkbarzadeh, M., et al.: Morphological profiling by means of the Cell Painting assay enables identification of tubulin-targeting compounds. \u003cem\u003eCell Chemical Biology\u003c/em\u003e 29, 1053\u0026ndash;1064. e1053 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCastoreno, A.B., et al.: Small molecules discovered in a pathway screen target the Rho pathway in cytokinesis. Nat. Chem. Biol. \u003cb\u003e6\u003c/b\u003e, 457\u0026ndash;463 (2010)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLejal, V., Cerisier, N., Rouqui\u0026eacute;, D., Taboureau, O.: Assessment of Drug-Induced Liver Injury through Cell Morphology and Gene Expression Analysis. Chem. Res. Toxicol. \u003cb\u003e36\u003c/b\u003e, 1456\u0026ndash;1470 (2023)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoo, L.-H., et al.: An approach for extensibly profiling the molecular states of cellular subpopulations. Nat. Methods. \u003cb\u003e6\u003c/b\u003e, 759\u0026ndash;765 (2009)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeldman, D., et al.: Pooled genetic perturbation screens with image-based phenotypes. Nat. Protoc. \u003cb\u003e17\u003c/b\u003e, 476\u0026ndash;512 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeldman, D., et al.: Optical pooled screens in human cells. Cell. \u003cb\u003e179\u003c/b\u003e, 787\u0026ndash;799 (2019)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFunk, L., et al.: The phenotypic landscape of essential human genes. Cell. \u003cb\u003e185\u003c/b\u003e, 4634\u0026ndash;4653 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiberali, P., Snijder, B., Pelkmans, L.: A hierarchical map of regulatory genetic interactions in membrane trafficking. Cell. \u003cb\u003e157\u003c/b\u003e, 1473\u0026ndash;1487 (2014)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFischer, B., et al.: A map of directional genetic interactions in a metazoan cell. Elife. \u003cb\u003e4\u003c/b\u003e, e05464 (2015)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYin, Z., et al.: A screen for morphological complexity identifies regulators of switch-like transitions between discrete cell shapes. Nat. Cell Biol. \u003cb\u003e15\u003c/b\u003e, 860\u0026ndash;871 (2013)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChandrasekaran, S.N., et al.: JUMP Cell Painting dataset: morphological impact of 136,000 chemical and genetic perturbations. \u003cem\u003ebioRxiv\u003c/em\u003e, 2003. 2023.534023 (2023). (2023)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eStirling, D.R., et al.: CellProfiler 4: improvements in speed, utility and usability. BMC Bioinform. \u003cb\u003e22\u003c/b\u003e, 1\u0026ndash;11 (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDao, D., et al.: CellProfiler Analyst: interactive data exploration, analysis and classification of large biological image sets. Bioinformatics. \u003cb\u003e32\u003c/b\u003e, 3210\u0026ndash;3212 (2016)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRocha, L., Velho, L., Carvalho, P.C.P.: in \u003cem\u003eProceedings. XV Brazilian Symposium on Computer Graphics and Image Processing.\u003c/em\u003e 99\u0026ndash;105 (IEEE)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaralick, R.M., Shanmugam, K., Dinstein, I.H.: Textural features for image classification. IEEE Trans. Syst. man. cybernetics, 610\u0026ndash;621 (1973)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMoshkov, N., et al.: Learning representations for image-based profiling of perturbations. \u003cem\u003eBiorxiv\u003c/em\u003e, 2022.2008. 503783 (2022). (2012)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlber, M., et al.: Integrating machine learning and multiscale modeling\u0026mdash;perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences. NPJ Digit. Med. \u003cb\u003e2\u003c/b\u003e, 115 (2019)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoo, L.-H., Wu, L.F., Altschuler, S.J.: Image-based multivariate profiling of drug responses from single cells. Nat. Methods. \u003cb\u003e4\u003c/b\u003e, 445\u0026ndash;453 (2007)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVaswani, A., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30 (2017)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRichard, A.M., et al.: The Tox21 10K compound library: collaborative chemistry advancing toxicology. Chem. Res. Toxicol. \u003cb\u003e34\u003c/b\u003e, 189\u0026ndash;216 (2020)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBegriche, K., Massart, J., Robin, M.-A., Borgne-Sanchez, A., Fromenty, B.: Drug-induced toxicity on mitochondria and lipid metabolism: mechanistic diversity and deleterious consequences for the liver. J. Hepatol. \u003cb\u003e54\u003c/b\u003e, 773\u0026ndash;794 (2011)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHargreaves, I.P., Al Shahrani, M., Wainwright, L., Heales, S.J.: Drug-induced mitochondrial toxicity. Drug Saf. \u003cb\u003e39\u003c/b\u003e, 661\u0026ndash;674 (2016)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTang, X., Wang, Z., Hu, S., Zhou, B.: Assessing drug-induced mitochondrial toxicity in cardiomyocytes: implications for preclinical cardiac safety evaluation. Pharmaceutics. \u003cb\u003e14\u003c/b\u003e, 1313 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaghighi, M., Caicedo, J.C., Cimini, B.A., Carpenter, A.E., Singh, S.: High-dimensional gene expression and morphology profiles of cells across 28,000 genetic and chemical perturbations. Nat. Methods. \u003cb\u003e19\u003c/b\u003e, 1550\u0026ndash;1557 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMeyer, J.N., Hartman, J.H., Mello, D.F.: Mitochondrial Toxicity. Toxicol. Sci. \u003cb\u003e162\u003c/b\u003e, 15\u0026ndash;23 (2018). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/toxsci/kfy008\u003c/span\u003e\u003cspan address=\"10.1093/toxsci/kfy008\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ede Wit, S., Glen, C., de Boer, R.A.: Lang, N. N. Mechanisms shared between cancer, heart failure, and targeted anti-cancer therapies. Cardiovascular. Res. \u003cb\u003e118\u003c/b\u003e, 3451\u0026ndash;3466 (2022)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNiemann, B., et al.: Caloric restriction reduces sympathetic activity similar to beta-blockers but conveys additional mitochondrio-protective effects in aged myocardium. Sci. Rep. \u003cb\u003e11\u003c/b\u003e, 1931 (2021)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePereira, C.: Drug-induced cardiac mitochondrial toxicity and protection: from doxorubicin to carvedilol. Curr. Pharm. Design. \u003cb\u003e17\u003c/b\u003e, 2113\u0026ndash;2129 (2011)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLozano-Sep\u0026uacute;lveda, S.A., Rinc\u0026oacute;n-Sanchez, A.R., Rivas-Estilla, A.M.: Antioxidants benefits in hepatitis C infection in the new DAAs era. Ann. Hepatol. \u003cb\u003e18\u003c/b\u003e, 410\u0026ndash;415 (2019)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReshi, M.L., Su, Y.-C., Hong, J.-R.: RNA viruses: ROS-mediated cell death. Int. J. cell. biology \u003cb\u003e2014\u003c/b\u003e (2014)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGhosh, M.C., et al.: Therapeutic inhibition of HIF-2α reverses polycythemia and pulmonary hypertension in murine models of human diseases. Blood J. Am. Soc. Hematol. \u003cb\u003e137\u003c/b\u003e, 2509\u0026ndash;2519 (2021)\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3933549/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3933549/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eHigh-content analysis (HCA) holds enormous potential for drug discovery and research, but widely used methods can be cumbersome and yield inaccurate results. Noisy and redundant signals in cell images impede accurate deep learning-based image analysis. To address these issues, we introduce X-Profiler, a novel HCA method that combines cellular experiments, image processing, and deep learning modeling. X-Profiler combines the convolutional neural network and Transformer to encode high-content images, effectively filtering out noisy signals and precisely characterizing cell phenotypes. In comparative tests on drug-induced cardiotoxicity, mitochondrial toxicity classification, and compound classification, X-Profiler outperformed both DeepProfiler and CellProfiler, as two highly recognized and representative methods in this field. Our results demonstrate the utility and versatility of X-Profiler, and we anticipate its wide application in HCA for advancing drug development and disease research.\u003c/p\u003e","manuscriptTitle":"Attention-based deep learning for accurate cell image analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-29 16:30:09","doi":"10.21203/rs.3.rs-3933549/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"05f40364-91ba-4e83-9526-84fe396db1b7","owner":[],"postedDate":"February 29th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":29004303,"name":"Biological sciences/Computational biology and bioinformatics/High-throughput screening"},{"id":29004304,"name":"Biological sciences/Drug discovery/Drug screening/High-throughput screening"}],"tags":[],"updatedAt":"2024-06-25T07:08:21+00:00","versionOfRecord":[],"versionCreatedAt":"2024-02-29 16:30:09","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3933549","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3933549","identity":"rs-3933549","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}