A Multi-stage Neural Network Approach for Coronary 3D Reconstruction from Uncalibrated X-ray Angiography Images

A Multi-stage Neural Network Approach for Coronary 3D Reconstruction from Uncalibrated X-ray Angiography Images | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A Multi-stage Neural Network Approach for Coronary 3D Reconstruction from Uncalibrated X-ray Angiography Images Kritika Iyer, Brahmajee K. Nallamothu, C. Alberto Figueroa, Raj R. Nadakuditi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2782923/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 16 Oct, 2023 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract We present a multi-stage neural network approach for 3D reconstruction of coronary artery trees from uncalibrated 2D X-ray angiography images. This method uses several binarized images from different angles to reconstruct a 3D coronary tree without any knowledge of image acquisition parameters. The method consists of a single backbone network and separate stages for vessel centerline and radius reconstruction. The output is an analytical matrix representation of the coronary tree suitable for downstream applications such as hemodynamic modeling of local vessel narrowing (i.e., stenosis). The network was trained using a dataset of synthetic coronary trees from a vessel generator informed by both clinical image data and literature values on coronary anatomy. Our multi-stage network achieved sub-pixel accuracy in reconstructing vessel radius (RMSE = 0.16 ± 0.07mm) and stenosis radius (MAE = 0.27 ± 0.18mm), the most important feature used to inform diagnostic decisions. The network also led to 52% and 38% reduction in vessel centerline reconstruction errors compared to a single-stage network and projective geometry-based methods, respectively. Our method demonstrated robustness to overcome challenges such as vessel foreshortening or overlap in the input images. This work is an important step towards automated analysis of anatomic and functional disease severity in the coronary arteries. Biological sciences/Physiology/Cardiovascular biology/Cardiovascular diseases/Acute coronary syndromes Physical sciences/Mathematics and computing/Computer science Physical sciences/Engineering/Biomedical engineering Physical sciences/Physics/Fluid dynamics Physical sciences/Mathematics and computing/Computational science Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 1. Introduction The coronary circulation is divided between the right and left coronary trees, each of which supply blood to different regions of the heart muscle [ 1 ]. Coronary Artery Disease (CAD) occurs when atherosclerotic plaque accumulates in the coronary arteries, leading to a local narrowing known as stenosis [ 2 ]. The most common imaging modality used to diagnose CAD is X-ray angiography, in which 2D X-ray images are acquired as radio-opaque dye is injected into the coronaries. This allows the cardiologist to visualize and estimate diameter reduction at regions of stenosis. Although anatomical assessment of stenosis severity is the most common diagnostic practice, functional metrics which take hemodynamic parameters into account through the use of invasive pressure wires placed in the coronaries, such as fractional flow reserve (FFR) and instantaneous wave-free ratio (iFR), have led to better diagnostic outcomes [ 3 ], [ 4 ]. In recent years, there have been numerous efforts to derive computational estimates of those metrics or to propose new metrics such as quantitative flow ratio (QFR) that do not rely on invasive pressure wires [ 5 ]–[ 10 ]. A key step in generating computational estimates through modeling approaches is defining the coronary tree geometry of the patient. Creating such a 3D model is relatively straightforward when using 3D medical image data such as computed tomography angiography (CTA). However, 2D X-ray angiography requires a method to accurately reconstruct the 3D geometry from a series of images. Many groups have proposed coronary reconstruction algorithms using the principles of projective geometry and stereovision [ 11 ]–[ 15 ]. An in-depth review of these techniques can be found in [ 16 ]. Despite their success, projection-based methods have several limitations such as their reliance on user input to identify corresponding vessels or features in all the input images. They also either require precise hardware calibration during image acquisition or algorithms to correct the recorded acquisition angles, since recorded angles and distances can have up to a 10% tolerance [ 17 ]. Furthermore, projection-based methods are susceptible to how clearly visible all coronary branches are in each angiography image. For example, overlapping branches and vessel foreshortening are two challenges which can introduce uncertainty into the reconstructed 3D geometry. Recently, a few groups have performed 3D reconstruction of coronary trees from X-ray angiography images using a combination of machine learning and projective geometry [ 18 ] [ 9 ]. While the vessel segmentation component of these methods utilized neural networks, the 3D reconstruction component relied on stereovision and projective geometry. A fully machine learning-based method has the potential to perform automated reconstruction without the need for imaging calibration. The main contribution of this work is a multi-stage neural network for 3D reconstruction from 2D binarized X-ray angiographic images which outputs an analytical representation of a 3D coronary tree. To our knowledge, this work is the first purely machine learning approach for 3D coronary tree reconstruction using X-ray angiography alone. The neural network was trained using analytical 3D coronary trees from a synthetic vessel generator. The advantage of such an analytical representation of the geometry is that it makes it easy to perform parametric hemodynamic analyses. A proof-of-concept example of such analysis is also included here. In this work, we demonstrate smaller reconstruction errors compared to a single-stage neural network (Section 3.1 ) and traditional projection-based methods (Section 3.2 ). Proof-of-concept hemodynamic analyses showed that acceptable errors were achieved for the clinical quantities of interest despite modest errors in the 3D geometric reconstruction (Section 3.3 ). 2. Methods 2.1 Multi-stage Neural Network Design We present a multi-stage neural network for 3D reconstruction of coronary trees. The input to the network is two or three binarized angiograms for a given coronary tree, with a Euclidean distance transform applied to create a smooth field that implicitly encodes vessel radii (see Fig. 1 ). In this work, binarized angiograms were directly created by our synthetic data generator (Section 2.2 ). For clinical angiograms, we have previously developed a vessel segmentation neural network known as AngioNet [ 19 ] which can convert the desired clinical images into binarized angiograms. Since no angle or distance information is provided to the neural network as an input, image calibration or parameter correction algorithms are not required. The multi-stage neural network was designed to reconstruct both vessel centerlines (stage 1) and radii (stage 2) for each branch in the coronary tree. The centerline and radius stages both employed a convolutional neural network backbone to learn relevant features of the coronary tree from the input images. In this work, we chose to use ResNet101 [ 20 ] as the backbone network. While the convolutional layers can learn image-based features relevant to the vessel geometry, a multilayer perceptron (MLP) is better suited to solve the regression problems of identifying the 3D coordinates of the vessel centerline and corresponding radii. Therefore, we replaced the final layer of the backbone network with separate MLPs for the centerline and radius stages, as follows. In the centerline stage, the final fully connected layer of the backbone network was replaced by a multilayer perceptron (MLP) with ReLU activation and batch normalization between layers. The MLP was composed of 4 hidden layers, where the first 3 layers had 1024 neurons and the last layer had 512 neurons. The output of the centerline MLP was a \(M*N*3\) linear layer, containing \(N\) centerline points for each of the \(M\) branches in the binarized angiogram. This output vector was reshaped into a matrix before computing the loss. Meanwhile, the radius stage replaced the final layer of the backbone with a separate MLP for each branch, for a total of \(M\) MLPs. The radius MLPs were composed of 3 hidden layers with 128 neurons each, with batch normalization and ReLU activation between each hidden layer. The output of each MLP was a vector of radii of dimension \(N\) . The MLP for each branch was trained separately to improve the network’s ability to capture sudden reductions in vessel radii at regions of stenosis. Without this step, stenoses are likely to be overlooked since they make up a small portion of the points in the coronary tree. The outputs of both stages were concatenated to form an \(M\times N\times 4\) matrix, where the last dimension encodes the 3D centerline coordinate and radius for each point as \((x,y,z,r)\) . The radius stage was trained using the mean squared error as the loss function: $$\begin{array}{c}\frac{1}{n}\sum _{i=0}^{n}{\left({y}_{i}- {\widehat{y}}_{i}\right)}^{2} \#\left(1\right)\end{array}$$ where \(y\) is the ground truth and \(\widehat{y}\) is the neural network prediction. Conversely, the centerline stage was trained using the same loss function with an additional length regularization term: $$\begin{array}{c}\frac{1}{n}\sum _{i=0}^{n}{\left({y}_{i}- {\widehat{y}}_{i}\right)}^{2}+ \lambda \sum {S}_{y}- {S}_{\widehat{y}}\#\left(2\right)\end{array}$$ where \(\lambda\) is the regularization rate, \({S}_{y}\) is the arclength of the ground truth branches, and \({S}_{\widehat{y}}\) is the arclength of the predicted branches. This vessel length regularization term was included because vessel length is an important determinant of the pressure gradient through a vessel, an important indicator of disease severity. An ADAM optimizer with learning rate 5e-4 and weight decay (L2) regularization was used to train both stages. Batch size was set to 8 and the multi-stage network was trained for 300 epochs. For the radius MLPs, the backbone was frozen after the initial 300 epochs and each branch MLP was trained for an additional 50 epochs. We did not observe a notable improvement in accuracy when retraining the backbone network for the centerline and radius tasks. We now present the single-stage counterpart to our multi-stage network for comparison. The single-stage network architecture was composed of the backbone network and a single MLP which outputs an \(M\times N\times 4\) matrix containing both the centerlines and their radii (Fig. 1 ). The loss function of the single-stage network was a weighted MSE loss: $$\begin{array}{c}\frac{1}{n}\sum _{i=0}^{n}{\left({y}_{i}- {\widehat{y}}_{i}\right)}^{2}+{\mu ({r}_{i}-\widehat{{r}_{i}})}^{2} \#\left(3\right)\end{array}$$ Here, \(y\) and \(\widehat{y}\) represent the ground truth and predicted centerline coordinates while \(r\) and \(\widehat{r}\) represent the radii along the centerlines. The regularization parameter \(\mu\) was chosen such that the centerline and radius terms were of the same order of magnitude. The single-stage network was trained using the same hyperparameters as the multi-stage network to make a fair comparison: ADAM optimizer with learning rate 5e-4, L2 regularization, and a batch size of 8. 2.2 Synthetic Dataset Generation To train the proposed multi-staged neural network, we require hundreds or thousands of ground truth 3D coronary trees and their corresponding segmented 2D angiograms. In practice, this means that we must identify thousands of patients with both 3D CTA data and 2D X-ray angiograms, which is typically not feasible in single-center studies such as ours. Another challenge of using clinical image data as input is that the coronaries deform in each frame of an X-ray angiography series due to the contraction of the heart. This necessitates temporal registration of frames from multiple angiographic series in order to create a valid set of input images for 3D coronary tree reconstruction. To produce a large enough dataset and eliminate external sources of error such as temporal registration, we devised a method to produce a sufficiently large training dataset consisting of 5,000 static 3D coronary tree geometries and their corresponding sets of 2D projections. While we have used synthetic data to train and validate our 3D reconstruction network, the use of synthetic projection images as input does not preclude future clinical application. A segmentation algorithm or neural network such as AngioNet [ 19 ] could be used to convert clinical angiograms obtained during routine patient care in the future into a suitable input for our network. This work focuses on 3D reconstruction of the right coronary tree as the large anatomical variation in the left coronary arteries [ 21 ] makes reconstruction more challenging. The method includes two steps: 1) a synthetic 3D coronary tree generator, and 2) a projection algorithm to create sets of segmented angiograms. A brief description of these steps is provided next. Step 1: Synthetic 3D coronary tree generator : Fig. 2 provides an overview of the coronary tree generator. A distribution of patient-specific centerlines was obtained from 10 CTAs for the four main branches of the right coronary tree, namely the right main coronary artery (RCA), sino-atrial node branch (SA), acute marginal branch (AM), and posterior-descending artery (PDA). The posterolateral ventricular branch (PLV) is implicitly included as part of the RCA, which bifurcates into the PDA and PLV branches. These data were used to identify a distribution of controls points and their standard deviations in 3D space for each branch of the coronary tree (see Fig. 2 A). From this distribution, new vessels can be generated via uniform random sampling. Linearly tapering radius was assigned to each branch, and stenoses with a gaussian profile were randomly introduced (Fig. 2 B). Branches were combined into a tree and augmented with random rotation, shear, and/or warping (Fig. 2 C). This algorithm was refined through repeated iterations with a board-certified interventional cardiologist to generate realistic trees. Further details of the clinical and mathematical assumptions used to inform synthetic data generation are given in Appendix A. Step 2: Projection algorithm : Cone-beam projections of each coronary tree were generated from 5 views to mimic the X-ray angiogram acquisition process. Image acquisition angles were randomly sampled from 20-degree windows around commonly used clinical values. Out of the 5 views, 3 were randomly chosen for training (Fig. 3 ). A Euclidean distance transform was applied to the projection images, which were then input into the neural network. The data generated in this fashion were split into 4,500 coronary trees and their corresponding projections for training and 500 for validation (90 − 10 training split). 3. Results In this section, we present several methods to evaluate the performance of the proposed method. We first compared the reconstruction error of our multi-stage network against a single-stage neural network to demonstrate the advantages of the multi-stage approach. We then performed a head-to-head comparison between our neural network reconstruction method and a projection-based method. Both comparisons were performed on single vessel geometries for simplicity. Next, we examined the performance of the method in reconstructing right coronary trees. Lastly, an analysis of how geometric reconstruction error affects hemodynamics is presented. 3.1 Multi-stage versus Single-Stage Reconstruction performance A key design feature of our network is its multi-stage nature, which considers the tasks of centerline and radius reconstruction separately. The importance of this feature is illustrated in this section via a comparison between the single- and multi-stage networks. The performance of both networks was evaluated using 10 distinct RCA synthetic vessels. For each synthetic vessel, stenoses between 20–90%, with increments of 5%, were introduced, for a total of 150 unique vessel geometries. Reconstruction error was assessed for each geometry. An example of a vessel geometry with 70% stenosis and its single- and multi-stage reconstructions is shown in Fig. 4 A. On average, the root mean squared error (RMSE) of the multi-stage network centerline was 52% lower than the single-stage centerline RMSE (0.83 ± 0.29mm vs 1.73 ± 0.42mm), see Fig. 4 B. As for radius reconstruction accuracy, mean absolute error (MAE) at the stenosis was evaluated instead of RMSE in the whole vessel due to the importance of accurately predicting stenosis severity. The MAE across all vessels was 0.117 ± 0.068mm for the multi-stage network and 0.927 ± 0.436mm for the single stage network. For both networks, the MAE increased with the stenosis severity (Fig. 4 B). However, the single-stage network effectively failed to detect the stenosis for all levels of stenosis (Fig. 4 A), resulting in extremely large MAE errors (over 1.5 mm) for severe stenoses with over 80% diameter reduction. 3.2 Multi-stage Centerline reconstruction against a Projection-based Method We now compare the performance of our multi-stage neural network against a projection-based method in the simplest case of a single vessel centerline reconstruction from 2 projection images. In this example, the projection-based approach was a combination of the algorithms proposed by Banerjee et al [ 11 ] and Vukicevic et al [ 12 ]. Briefly, a point cloud approach [ 11 ] was used to automatically identify up to 10 possible corresponding points for each centerline point in the reference projection image. The set of possible corresponding points was further refined using the reprojection error and ordering constraint cost function proposed in [ 12 ]. Matched points from both images were then back-projected and interpolated with a b-spline to create a 3D centerline. The dataset of synthetic projection images for this comparison was defined as follows. 5 synthetic RCA vessels were first created. We considered a spherical coordinate system \((\theta ,\varphi )\) , where \(\theta\) and \(\varphi\) are the azimuthal and elevation angles, respectively. For a given vessel, the first projection image in every pair was fixed at \(\theta =-45^\circ , \varphi =0\) . The second image was defined using six different intervals \(\varDelta \theta \in [15^\circ , 90^\circ ]\) , for a total of 30 different pairs of projection images (5 different vessels, 6 different angles between projection images for each vessel), see Fig. 5 A. The reconstruction error as a function of the angle between projection images, \(\varDelta \theta\) , was compared between both methods. As seen in Fig. 5 B, the centerline RMSE and standard deviation was lower in the neural network approach compared to the projection-based method for all tested angles and vessels: 2.12 ± 0.05mm and 3.49 ± 0.44mm, respectively. Figure 5 C shows two examples of the same vessel reconstructed using 2 pairs of projection images with different \(\varDelta \theta\) . The ground truth centerline (blue), neural network centerline (red), and projection-based centerline (brown) are shown. The neural network centerline followed the average path of the ground truth centerline, although it failed to capture some of the bends along its path (red arrows). Meanwhile, the brown arrows indicate gaps or regions of uncertainty in the centerline reconstructed using projection-based methods. 3.3 Reconstruction of coronary trees In this section, we measure the accuracy of the proposed method for coronary tree reconstruction, using the validation set of 500 synthetic coronary trees defined in Section 2.2 . Figure 6 shows several examples of reconstructed coronary tree centerlines (red) and their corresponding ground truth (blue). We observed that the neural network learned the mean centerline path of each vessel in the tree; however, as in the single vessel case, the predicted vessels were not as tortuous as the ground truth centerlines. The RMSE between the ground truth and predicted centerline points in the validation set was 2.57 ± 0.78mm. The MAE in vessel length was 8.83 ± 4.81mm. Optimal values for vessel length reconstruction were obtained with a regularization length parameter \(\lambda =0.1\) (see Eq. (2)), which led to a 47% decrease in vessel length error (16.36 ± 2.88 mm) compared to the same network trained without length regularization in the loss function. Larger values of \(\lambda\) resulted in over-constraining the vessel length and inaccurate paths for the different branches, whereas smaller values of \(\lambda\) resulted in weaker enforcement of the vessel length. We now consider the error in vessel radius along the centerline for all 2,000 branches. The RMSE of the vessel radius was 0.16 ± 0.07mm, which corresponds to sub-pixel resolution. The error in radius was larger when comparing minimum stenosis diameter (MAE = 0.27 ± 0.18mm), particularly for severe stenoses greater than 70% diameter reduction, consistent with the behavior reported for single vessel reconstruction in Section 3.1 . Figure 7 shows examples of radius reconstruction (red dots) along normalized centerline position of the main RCA (blue) in vessels without stenosis (top panel), two cases of a single stenosis (mid panel), and a case with two stenoses (bottom panel). Panel A demonstrates that the neural network accurately captures the tapering of the vessel along its length. Panel B shows that the neural network accurately predicts the severity and location of the stenosis, although with some oscillations apparent in regions outside the stenosis, or a slight shift in the stenosis location. Lastly, in panel C the neural network approximates serial stenoses as a single, longer stenosis with an error of 0.3mm in stenosis severity. Some oscillations remain apparent in the region outside the stenosis. 3.4 Effect of reconstruction error on hemodynamics In the previous examples, we established the performance of the multi-stage neural network to reconstruct centerline and radius of vessels in coronary artery trees. However, as stated in the introduction, functional assessment of vessel disease has recently received significant attention. In this section, we investigate the quality of our proposed multi-stage neural network from a functional standpoint. Towards that end, we simulate the physics of blood flow and pressure in the reconstructed coronary trees using computational fluid dynamics (CFD) simulations and compare the results against known ground truth CFD data. Specifically, we compare distributions of pressure down the RCA vessel in ground truth coronary trees (known synthetic geometries) and their reconstructions. The pressure field is the basis to calculate well-established functional metrics of CAD such as FFR, iFR, and QFR [ 4 ], [ 6 ], [ 22 ]. CFD simulations with consistent inflow and outflow boundary conditions were run using the validated open-source software CRIMSON [ 23 ]. Two examples are discussed: 1) a healthy coronary tree without stenosis, and 2) a diseased tree with one stenosis. Details of simulation parameters (e.g., inflow and outflow boundary conditions, fluid properties such as density and viscosity, etc.) are given in Appendix B. Results are summarized in Fig. 8 . Results for the healthy coronary tree are given in the top panel of Fig. 8 .The left column depicts solution maps of pressure calculated via CFD analysis for ground truth and reconstructed geometries. The center and right columns show the radius and pressure, respectively, versus normalized centerline position in the ground truth (blue) and reconstructed (red) geometries. In the top panel, there is no stenosis and the changes in pressure down the RCA are small. The pressure gradients over the ground truth and reconstructed RCA vessels are DP GT = 9.8 mmHg and DP R = 8.4 mmHg, resulting in a difference between pressure gradient estimates of \({\Delta }P={\Delta }{P}_{GT}-{\Delta }{P}_{R}\) = 1.4 mmHg (or only 1.2% of the ground truth inflow pressure). In the diseased coronary tree, the pressure gradients over the ground truth and reconstructed RCA vessels are DP GT = 18.8 mmHg and DP R = 14.3 mmHg, resulting in an error of \({\Delta }P\) = 4.5 mmHg. Additionally the pressures in the reconstructed geometry were approximately 10mmHg lower than the pressures in the ground truth. This is due to the differences in reconstructed radius in the proximal portion of the vessel, as well as the decreased tortuosity (and therefore decreased resistance) of the reconstructed coronary tree compared to the ground truth. The larger pressure error in the stenosis case compared to the healthy case can be attributed to the radius reconstruction error at the stenosis. Given a vessel with radius \(a\) , its resistance \(R\) is inversely proportional to its radius to the power of four ( \(R \tilde 1⁄{a}^{4}\) ). Thus, relatively small errors in radius reconstruction have substantial impacts on vessel resistance and therefore pressure drop across the vessel. The ground truth stenotic geometry had serial 51% and 57% stenoses. However, the neural network approximated these 2 stenoses as a single, longer, 53% stenosis, corresponding to a radius error of 0.07mm (see center column Fig. 8 ). The ratio of pressures on either side of the stenosis (a surrogate of FFR), was 0.886 in the ground truth and 0.895 in the reconstructed vessel. Therefore, despite the 10mmHg difference in inlet pressure, there was only a 1% error in the clinical quantity of interest. 4. Discussion Reconstructing 3D geometries from sets of 2D image data is a challenging task that often necessitates accurate knowledge of imaging parameters such as angles and distances during image acquisition. Uncertainty of these parameters poses a large challenge for projective geometry approaches to reconstruct the coronaries from X-ray angiography images; therefore, we have developed and tested a novel multi-stage neural network method for 3D coronary reconstruction. 4.1 Considerations of Neural Network Design The most salient feature of the proposed network is its multi-stage nature, which enables accurate reconstruction of centerline paths and vessel radii. When considering the single-stage approach to reconstructing vessel geometry, we observed that the network was able to accurately capture the trend of linearly decreasing radius along the vessel but could not capture the stenoses (Section 3.1 ). This is likely because the stenosis radii make up less than 2% of the \(N\times 4\) vessel matrix; the stenoses had such a small contribution to the loss function that the network did not learn to predict them. In the multi-stage approach, the stenosis radii had a much larger contribution to the loss function since the mean squared error of the radii was optimized separately from the centerlines. The network was therefore able to learn and accurately reconstruct stenosis severity. This observation motivated going one step further for the coronary trees and using a separate MLP to learn the radii for each vessel. This was necessary since not all vessel branches contained stenoses; if the network was trained to predict the radii of all branches at the same time, the stenoses would once again make up less than 2% of the \(M\times N\) output matrix. A major challenge of reconstructing stenoses is that, while they are the most important diagnostic feature of the image, they only make up a small proportion of the vessel matrix. The multi-stage approach described in this work amplifies the impact of the stenoses and thus improves prediction accuracy. The size of the MLPs for the centerline and radius tasks was determined by testing networks with a varying number of hidden layers and neurons per layer. Networks with 128, 256, 512, 1024, 2048, and 4096 neurons in each of 1–4 hidden layers were evaluated for both centerline and radius accuracy. For the centerline MLPs, the minimum error was achieved when using 4 hidden layers. The number of neurons per layer did not greatly affect mean squared error, which ranged from 2.72mm to 2.91mm in the validation set of 500 synthetic coronary trees. Therefore, a network with 1024 neurons per layer was chosen as it was of the same order of magnitude as the network output size ( \(M\) * \(N\) *3). For the radius MLPs, the lowest error was achieved in networks with 3 hidden layers. Interestingly, although overall mean squared error for the radii was similar for networks with different widths (0.02-0.03mm), smaller networks performed better at regions of stenosis. The MAE at the stenosis for a network with 128 neurons was 0.27mm, compared to 0.40mm and 0.42mm for networks with 2048 and 4096 neurons, respectively. Thus, we determined that the optimal size for the radius network was 3 hidden layers with 128 neurons each. 4.2 Comparison with Projection-based Methods of Reconstruction 3D reconstruction of coronary trees from 2D X-ray angiographic images has been generally performed using projective geometry and stereovision techniques [ 11 ]–[ 15 ]. Despite their success, projection-based methods have limitations such as the need to identify corresponding features in the input images, and precise information on the image acquisition angles. Overlapping branches and foreshortening are two key challenges leading to uncertainty in the reconstructed geometry using projective geometry methods. The key contribution of this paper is a neural network for reconstructing 3D coronary trees that aims to overcome the limitations of projective geometry methods. Our network was trained over a wide range of angles between images \(\varDelta \theta\) with the purpose of making it robust (and therefore relatively insensitive to this parameter). In Section 3.2 , we compared the performance of our multi-stage neural network against a projection-based method in a synthetic dataset of pairs of projection images with angles between images \(\varDelta \theta \in \left[15^\circ , 90^\circ \right]\) . Our network’s RMSE was on average 39% lower than the projection method across all values of \(\varDelta \theta ,\) even though the neural network was not trained on images at these specific projection angles. Together with the lower standard deviation, this could indicate that the neural network is more robust to different characteristics of the input images. For instance, different amounts of foreshortening or overlap may occur for a given vessel when projections are taken at different angles. While the projection method reconstructions are less accurate when the input images have high levels of foreshortening or overlap, the neural network could learn universal characteristics of the RCA vessel shape to overcome these challenges and maintain similar accuracy for all input images. The higher robustness to foreshortening and overlap in the input images of the neural network method is further supported by the qualitative examples in Fig. 5 C. The brown arrows, which point to gaps or regions of uncertainty in the projection method centerline, correspond to regions on the input image where the vessel is highly foreshortened or bending at an angle that obscures other parts of the vessel. The neural network interpolates the 3D centerline in these regions, resulting in a continuous path. The main limitation of the neural network compared to projection methods is that it does not capture all the bends in the vessel centerline path despite accurately following the average path. The lack of tortuosity leads to a shorter reconstructed vessel than the ground truth, despite the length regularization term in the loss function (Eq. 2). This behavior could be a consequence of the network learning the features of a typical 3D RCA shape. While these features ensure that the network can interpolate the 3D centerline even when the input images are challenging, they also may cause the network to predict similar, smooth centerline paths for all inputs. Meanwhile, the projection-based method can accurately capture the details specific to each centerline path using the features of the input images. The neural network was trained using a MSE loss, which only accounts for point-wise differences between ground truth and reconstructed centerlines. A loss function that incorporates more global shape information such as curvature in addition to the vessel length regularization could improve the centerline path accuracy. 4.3 Reconstruction of Coronary Trees When applying the multi-stage neural network to reconstructing a whole coronary tree, the reconstructions captured the average path of all branches in the tree but under-estimated their tortuosity, similar to the behavior observed for single-vessel reconstruction. This appears to correspond to a well-known curve-fitting regression phenomenon: when the network is not large enough to capture the full complexity of a dataset, it tends to predict its average behavior. We hypothesized that increasing the complexity of the neural network by increasing the size of the MLP may alleviate this issue. We tested this hypothesis by performing a scaling study in Section 4.1 to test the performance of networks with 128 to 4096 neurons in each of 1–4 hidden layers. From this scaling study, we observed 1% decrease in RMSE between the 4096x4 network and 128x1 network. Therefore, the computational overhead of increasing the MLP size did not justify the modest decrease in error. As discussed in Section 4.2 , a loss function which incorporates global shape may be a more effective strategy to improve centerline accuracy. Another challenge of coronary tree reconstruction compared to single vessel reconstruction was to accurately predict stenoses radius. As described in Section 4.1 , employing a multi-stage approach, and training a separate MLP for each vessel branch enabled the network to learn how to predict stenoses. Despite this, the MAE at the stenosis was higher for coronary trees (0.27mm) compared to a single vessel (0.12mm). This may be due to the random projection angles, as overlapping branches may have obscured the stenosis in some of the input images. 4.4 Effect of reconstruction error on hemodynamics The last application example explored the impact of geometric reconstruction errors on hemodynamic indices such as pressure. The simple test examples demonstrated that gradients of pressure (a surrogate of FFR) compared well between ground truth and reconstructed geometries, even though the neural network centerlines were not as tortuous as the ground truth. The error in radius had a much larger impact on the pressure drop than the centerline error, particularly errors in stenosis radius. As seen in the stenosis example, the neural network approximated two serial stenoses (51% and 57% radius reduction) as a single stenosis with 53% radius reduction. Despite only a 0.07mm difference in stenosis radius, there was a noticeable difference in pressure profiles across the stenosis between the 2 geometries. However, the gradients of pressure across the stenosis were relatively similar between the 2 cases (less than 1% difference). We hypothesize that accurately capturing the overall vessel lengths and stenosis radii contributed to the accurate estimation of gradients across the vessels. However, at higher levels of stenosis, even small errors in reconstructed radius can lead to large differences in pressure gradients across the vessels. Given the high sensitivity of vessel resistance to the reconstructed vessel radius ( \(R\tilde1/{a}^{4}\) ), it is critically important to accurately determine vessel radius at the stenosis. The main limitation of improving radius estimation is the resolution of the X-ray angiography image. In an angiogram, the difference between a 70% and 75% stenosis cannot be distinguished since it is smaller than the pixel resolution, and thus the projections of both levels of stenosis appear identical. It is therefore difficult to differentiate between stenoses at a higher resolution even with a deep neural network. Super-resolution algorithms are a potential solution, as they have successfully been used to overcome the limitations of pixel size in magnetic resonance imaging [ 24 ]–[ 26 ]. 4.5 Future Work Future work will focus on increasing the scope of the dataset to match real-world data more closely. In terms of synthetic data generation, we will expand our tree generator to incorporate further variations in the number and types of branches in the right and left coronary trees. We will also explore alternate strategies to encode the 3D coronary geometry and stenoses, since training a separate MLP to learn the radius along each branch may become infeasible in tree structures with a large number of branches. For example, the vessel radii could be encoded in a vector containing the proximal radius, distal radius, and parametric position and severity of the stenoses for each branch. Besides the synthetic projection images considered in this paper, our multi-stage neural network can be applied to clinical X-ray angiograms that have been processed with a segmentation algorithm to create binary input images. Clinical angiograms bring additional challenges, such as the motion of the coronary tree during the cardiac cycle. Care must be taken to identify angiographic frames acquired at the same point in the cardiac cycle to reconstruct a valid 3D coronary tree from clinical images. In the catheterization lab, it is common practice to move the patient table during image acquisition to follow the flow of injected dye. Since the motion is controlled via a joystick and not often recorded, clinical angiograms are challenging to use as inputs for projection-based reconstruction methods because they require precise knowledge of image acquisition parameters. Neural networks may therefore provide a robust alternative for 3D reconstruction of coronary trees when those parameters are unknown. In conclusion, in this work we have presented a proof-of-concept of a multi-stage neural network that can be used for 3D reconstruction of coronary trees from sets of uncalibrated X-ray angiography images, with sub-pixel resolution of vessel radius. We have demonstrated that reconstruction error is at an acceptable level to accurately model hemodynamic quantities such as pressure gradients across a vessel. Declarations Acknowledgments The authors acknowledge the Precision Health at the University of Michigan for providing cost-free, member access to GPUs and computing resources on the Armis2 cluster—necessary resources to support the neural network training reported in this publication. Computing resources for hemodynamics simulations were provided by the National Science Foundation [Grant 1531752], Acquisition of Conflux. This work was supported by the National Science Foundation Graduate Research Fellowship Program [DGE1841052] and Rackham Merit Fellowship. Data and Code Availability The code for the synthetic coronary generator can be found on GitHub: https://github.com/kritiyer/vessel_tree_generator/ Competing Interest Statement CAF and BKN are founders of AngioInsight, Inc, a startup company which uses machine learning and signal processing to assist physicians with angiography interpretation. AngioInsight did not sponsor this work. This work is a part of a patent which has been filed by the University of Michigan, US patent 11,386,563 Anatomical and Functional Assessment of Coronary Artery Disease Using Machine Learning . All other authors declare no competing interests. References W. W. Nichols, M. F. O’Rourke, C. Vlachopoulos, and D. A. McDonald, McDonald’s Blood Flow in Arteries : Theoretical, Experimental and Clinical Principles . London: Hodder Arnold, 2011. Available: https://www.crcpress.com/McDonalds-Blood-Flow-in-Arteries-Theoretical-Experimental-and-Clinical/Vlachopoulos-ORourke-Nichols/p/book/9780340985014 N. H. Pijls, J. A. van Son, R. L. Kirkeeide, B. De Bruyne, and K. L. Gould, “Experimental basis of determining maximum coronary, myocardial, and collateral blood flow by pressure measurements for assessing functional stenosis severity before and after percutaneous transluminal coronary angioplasty.,” Circulation , vol. 87, no. 4, pp. 1354–67, Apr. 1993, doi: 10.1161/01.CIR.87.4.1354. W. F. Fearon et al. , “Clinical Outcomes and Cost-Effectiveness of Fractional Flow Reserve–Guided Percutaneous Coronary Intervention in Patients With Stable Coronary Artery Disease: Three-Year Follow-Up of the FAME 2 Trial (Fractional Flow Reserve Versus Angiography for Multivessel Evaluation),” Circulation , vol. 137, no. 5, pp. 480–487, Jan. 2018, doi: 10.1161/CIRCULATIONAHA.117.031907. M. Götberg et al. , “Instantaneous Wave-free Ratio versus Fractional Flow Reserve to Guide PCI,” New England Journal of Medicine , vol. 376, no. 19, pp. 1813–1823, May 2017, doi: 10.1056/NEJMoa1616540. W. F. Fearon et al. , “Accuracy of Fractional Flow Reserve Derived From Coronary Angiography,” Circulation , Jan. 2019, doi: 10.1161/CIRCULATIONAHA.118.037350. B. Xu et al. , “Diagnostic Accuracy of Angiography-Based Quantitative Flow Ratio Measurements for Online Assessment of Coronary Stenosis,” Journal of the American College of Cardiology , vol. 70, no. 25, pp. 3077–3087, 2017, doi: 10.1016/j.jacc.2017.10.035. J. M. Carson, C. Roobottom, R. Alcock, and P. Nithiarasu, “Computational instantaneous wave-free ratio (IFR) for patient-specific coronary artery stenoses using 1D network models,” International Journal for Numerical Methods in Biomedical Engineering , vol. 35, no. 11, p. e3255, 2019, doi: 10.1002/cnm.3255. C. A. Taylor, T. A. Fonte, and J. K. Min, “Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: Scientific basis,” Journal of the American College of Cardiology , vol. 61, no. 22, pp. 2233–2241, 2013, doi: 10.1016/j.jacc.2012.11.083. J. Jiang et al. , “Fractional flow reserve for coronary stenosis assessment derived from fusion of intravascular ultrasound and X-ray angiography,” Quant Imaging Med Surg , vol. 11, no. 11, pp. 4543–4555, Nov. 2021, doi: 10.21037/qims-20-1324. J. Li et al. , “Accuracy of computational pressure-fluid dynamics applied to coronary angiography to derive fractional flow reserve: FLASH FFR,” Cardiovascular Research , vol. 116, no. 7, pp. 1349–1356, Jun. 2020, doi: 10.1093/cvr/cvz289. A. Banerjee, F. Galassi, E. Zacur, G. L. D. Maria, R. P. Choudhury, and V. Grau, “Point-Cloud Method for Automated 3D Coronary Tree Reconstruction From Multiple Non-Simultaneous Angiographic Projections,” IEEE Transactions on Medical Imaging , vol. 39, no. 4, pp. 1278–1290, Apr. 2020, doi: 10.1109/TMI.2019.2944092. A. M. Vukicevic, S. Çimen, N. Jagic, G. Jovicic, A. F. Frangi, and N. Filipovic, “Three-dimensional reconstruction and NURBS-based structured meshing of coronary arteries from the conventional X-ray angiography projection images,” Scientific Reports , vol. 8, no. 1, Dec. 2018, doi: 10.1038/s41598-018-19440-9. C. V. Bourantas et al. , “A method for 3D reconstruction of coronary arteries using biplane angiography and intravascular ultrasound images,” Computerized Medical Imaging and Graphics , vol. 29, no. 8, pp. 597–606, Dec. 2005, doi: 10.1016/j.compmedimag.2005.07.001. S. J. Chen and J. D. Carroll, “3-D Reconstruction of Coronary Arterial Tree to Optimize Angiographic Visualization,” IEEE Transactions on Medical Imaging , vol. 19, no. 4, pp. 318–336, Apr. 2000, doi: 10.1109/42.848183. J. Yang, W. Cong, Y. Chen, J. Fan, Y. Liu, and Y. Wang, “External force back-projective composition and globally deformable optimization for 3-D coronary artery reconstruction,” Phys. Med. Biol. , vol. 59, no. 4, pp. 975–1003, Feb. 2014, doi: 10.1088/0031-9155/59/4/975. S. Çimen, A. Gooya, M. Grass, and A. F. Frangi, “Reconstruction of coronary arteries from X-ray angiography: A review.,” Medical image analysis , vol. 32, pp. 46–68, Aug. 2016, doi: 10.1016/j.media.2016.02.007. Siemens AG, “Artis Q/Q.zen/zeego System Owners Manual.” 2015. D. M. Bappy, A. Hong, E. Choi, J.-O. Park, and C.-S. Kim, “Automated three-dimensional vessel reconstruction based on deep segmentation and bi-plane angiographic projections,” Computerized Medical Imaging and Graphics , vol. 92, p. 101956, Sep. 2021, doi: 10.1016/j.compmedimag.2021.101956. K. Iyer et al. , “AngioNet: a convolutional neural network for vessel segmentation in X-ray angiography,” Sci Rep , vol. 11, Sep. 2021, doi: 10.1038/s41598-021-97355-8. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Dec. 2015. Available: http://arxiv.org/abs/1512.03385 F. Cademartiri et al. , “Prevalence of anatomical variants and coronary anomalies in 543 consecutive patients studied with 64-slice CT coronary angiography,” Eur Radiol , vol. 18, no. 4, pp. 781–791, Apr. 2008, doi: 10.1007/s00330-007-0821-9. N. H. Pijls et al. , “Measurement of fractional flow reserve to assess the functional severity of coronary-artery stenoses,” N Engl J Med , vol. 334, no. 26, pp. 1703–1708, Jun. 1996, doi: 10.1056/NEJM199606273342604. C. J. Arthurs et al. , “CRIMSON: An open-source software framework for cardiovascular integrated modelling and simulation,” PLOS Computational Biology , vol. 17, no. 5, p. e1008881, May 2021, doi: 10.1371/journal.pcbi.1008881. E. Ferdian et al. , “Cerebrovascular super-resolution 4D Flow MRI – using deep learning to non-invasively quantify velocity, flow, and relative pressure.” bioRxiv, p. 2021.08.25.457611, Aug. 27, 2021. doi: 10.1101/2021.08.25.457611. Y. Chen, F. Shi, A. G. Christodoulou, Y. Xie, Z. Zhou, and D. Li, “Efficient and Accurate MRI Super-Resolution Using a Generative Adversarial Network and 3D Multi-level Densely Connected Network,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 . Springer International Publishing, 2018, pp. 91–99. doi: 10.1007/978-3-030-00928-1_11. H. Greenspan, G. Oz, N. Kiryati, and S. Peled, “MRI inter-slice reconstruction using super-resolution,” Magnetic Resonance Imaging , vol. 20, no. 5, pp. 437–446, Jun. 2002, doi: 10.1016/S0730-725X(02)00511-8. Additional Declarations Competing interest reported. CAF and BKN are founders of AngioInsight, Inc, a startup company which uses machine learning and signal processing to assist physicians with angiography interpretation. AngioInsight did not sponsor this work. This work is a part of a patent which has been filed by the University of Michigan, US patent 11,386,563 Anatomical and Functional Assessment of Coronary Artery Disease Using Machine Learning. All other authors declare no competing interests. Supplementary Files Appendicesscireports.docx Cite Share Download PDF Status: Published Journal Publication published 16 Oct, 2023 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Major revision 28 Jul, 2023 Reviews received at journal 07 Jul, 2023 Reviewers agreed at journal 06 Jul, 2023 Reviewers agreed at journal 02 Jul, 2023 Reviewers agreed at journal 02 Jul, 2023 Reviewers invited by journal 19 Jun, 2023 Editor assigned by journal 19 Jun, 2023 Editor invited by journal 16 Apr, 2023 Submission checks completed at journal 16 Apr, 2023 First submitted to journal 05 Apr, 2023 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2782923","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":192257965,"identity":"c533882a-2e5a-4f61-a234-6fb1962a6e38","order_by":0,"name":"Kritika Iyer","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA10lEQVRIiWNgGAWjYBACNiBmbGCQYGBjbwAyDSyI1yLBxnMApEWCOJuAWoDWSCSA2ERo4ZNIPvZx5h6LOj7J51c3/CiQYOBv707A7zCJtOSZG54BHSadU3azB+gwiTNnNxDQkmPM+OAAWEvaDR6gFgOJXEJa8j9DtEieSbv5hzgtOcyMG0BaJNiP3SbOFp5nxowzDkhItvHksN2WMZDgIegX+fbkx4w9B+r45duPP7v55o+NHH97L34tDAIJMBaPAZjErxwE+A/AWOwPCKseBaNgFIyCEQkAQbM/giIDu4AAAAAASUVORK5CYII=","orcid":"","institution":"University of Michigan–Ann Arbor","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Kritika","middleName":"","lastName":"Iyer","suffix":""},{"id":192257966,"identity":"c76d1885-5b3a-41c7-bdc3-f55383542068","order_by":1,"name":"Brahmajee K. Nallamothu","email":"","orcid":"","institution":"University of Michigan–Ann Arbor","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Brahmajee","middleName":"K.","lastName":"Nallamothu","suffix":""},{"id":192257967,"identity":"c3981108-695e-41a4-b1e6-ad71a8518ebd","order_by":2,"name":"C. Alberto Figueroa","email":"","orcid":"","institution":"University of Michigan–Ann Arbor","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"C.","middleName":"Alberto","lastName":"Figueroa","suffix":""},{"id":192257968,"identity":"c2250f2a-31f7-43cd-adcf-847126ac416b","order_by":3,"name":"Raj R. Nadakuditi","email":"","orcid":"","institution":"University of Michigan–Ann Arbor","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Raj","middleName":"R.","lastName":"Nadakuditi","suffix":""}],"badges":[],"createdAt":"2023-04-05 23:29:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2782923/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2782923/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-023-44633-2","type":"published","date":"2023-10-16T15:01:44+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":35944775,"identity":"8356d395-933a-4dd3-8ea0-ecd2413f2de3","added_by":"auto","created_at":"2023-04-18 14:52:16","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":206875,"visible":true,"origin":"","legend":"\u003cp\u003eNeural network architecture. The input is a set of binarized X-ray angiography images with a Euclidean distance transform applied for mathematical smoothness. The multi-stage network is composed of a backbone network and separate multi-layer perceptrons which predict vessel centerlines and radii. In contrast, the single-stage network is composed of the backbone network and a single MLP which predicts both vessel centerlines and radii simultaneously. In this paper, the angiography images are created from ground truth synthetic data (section 2.2), although the network is capable of taking segmented clinical angiograms as input data as well.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/75cc37ba7ff3155100c23e4d.png"},{"id":35942706,"identity":"1ed1feef-65bf-44b2-bb9a-15ba5442fc1e","added_by":"auto","created_at":"2023-04-18 14:36:16","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":131370,"visible":true,"origin":"","legend":"\u003cp\u003eDiagrams of synthetic 3D coronary tree generation. A: Centerlines derived from patient image data (CTA) are aligned to determine a distribution of control points. Control points are then randomly sampled from this distribution to produce new centerlines. B: Schematic of a gaussian stenosis profile introduced in a tapered vessel C: Examples of a real patient coronary tree from CTA and synthetically generated coronary trees. The synthetic trees mimic the structure and shape of the CTA-derive tree while being uniquely different.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/7b2c3f226ac35528e0f9ce90.png"},{"id":35944079,"identity":"354ce9ee-c98b-4455-80ba-5ade826e5cdd","added_by":"auto","created_at":"2023-04-18 14:44:16","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":132238,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic of Synthetic X-ray Projection Generation. Projections were obtained as a cone beam of X-rays from a point source hitting a detector at a series of sampled random positions. The algorithm produces a set of binary angiograms. A Euclidean Distance Transform is applied to the projection images to create the input for the multi-stage neural network.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/707922189fc1bdb7243183b6.png"},{"id":35942710,"identity":"07d42f26-a4b0-4cd8-a01b-96d9641b8820","added_by":"auto","created_at":"2023-04-18 14:36:16","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":180942,"visible":true,"origin":"","legend":"\u003cp\u003eSingle- and multi-stage neural network reconstruction results. 2A shows an example vessel with a 70% stenosis and corresponding reconstructions. The single-stage network fails to capture the stenosis while the multi-stage network prediction does capture the stenosis and more closely follows the vessel centerline path. 2B shows reconstruction error for vessel centerlines (RMSE) and radii (stenosis MAE) in 150 synthetically generated vessels with varying levels of stenosis from 20-90%. The multi-stage network had a 52% lower centerline error compared to the single-stage network. Furthermore, the single-stage network failed to capture the stenosis, as demonstrated by the increasing radius error for increasing stenosis severity.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/4ea35d3072363c23e7601704.png"},{"id":35944082,"identity":"cd18e0c2-5c72-4941-91ff-b6e58c5cfeb4","added_by":"auto","created_at":"2023-04-18 14:44:16","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":206451,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of neural network and projection-based reconstructions of a single vessel from different input projection images. A) Sample 3D RCA geometry and corresponding projection images. B) The neural network has on average 38% lower reconstruction error for all values of Δθ, the angle between input projection images. C) Examples of the same 3D geometry reconstructed from different input projection images. Red arrows indicate regions where the neural network centerline does not follow the ground truth path while brown arrows indicate regions of uncertainty and gaps in the projection-based centerline.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/c2e79b1ca22c4264df256ed6.png"},{"id":35942713,"identity":"082ca0d3-5667-4eb9-90dd-3d9d7bd521c1","added_by":"auto","created_at":"2023-04-18 14:36:16","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":170405,"visible":true,"origin":"","legend":"\u003cp\u003eExamples ground truth centerlines (blue), reconstructed centerlines (red), and their corresponding input images. As in the single vessel case, the neural network predictions follow the average path of all vessels but do not have the same tortuosity as the ground truth centerlines.\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/104fb5eb1ef6710776a97569.png"},{"id":35945376,"identity":"f8778d45-9f9c-4122-b0ca-5b477e440d4d","added_by":"auto","created_at":"2023-04-18 15:00:16","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":91218,"visible":true,"origin":"","legend":"\u003cp\u003eExamples of radius reconstruction (red) along normalized centerline position of the main RCA (blue) in a vessel without stenosis, vessels with a single stenosis, and two stenoses. For the vessel without stenosis, the neural network captures the linear tapering of the vessel well. For the vessels with a single stenosis, the neural network accurately predicts the severity and location of the stenosis, although with some oscillations apparent in regions outside the stenosis, or a slight shift in the stenosis location. Finally, in the multi-stenosis case, the neural network approximates serial stenoses as a single stenosis with an error of 0.3mm in stenosis severity.\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/50ee904fc7a28f63a59e3337.png"},{"id":35942711,"identity":"c94488b7-c89d-41e7-9752-0fdc1a7b425f","added_by":"auto","created_at":"2023-04-18 14:36:16","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":206461,"visible":true,"origin":"","legend":"\u003cp\u003eComparison of pressures in ground truth and reconstructed coronary trees. The top and bottom panels contains results for a healthy tree without stenosis and a disease tree with two stenoses, respectively. The left column shows solution maps of pressure calculated via CFD in ground truth and reconstructed geometries. The center and right columns show plots of reconstructed radius and CFD-derived pressure down the RCA for the ground truth (blue) and reconstructed (red) cases.\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/a0a9a2a5e0d10f3cc10b2662.png"},{"id":45091264,"identity":"5fd453f0-95d6-4c9f-aa00-8eeb652279f9","added_by":"auto","created_at":"2023-10-23 15:09:17","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1876743,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/f42a4bf7-a596-4135-a190-bc999d5ae3dc.pdf"},{"id":35944777,"identity":"3fe29ddd-920f-450b-bf34-493768efd484","added_by":"auto","created_at":"2023-04-18 14:52:16","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":26576,"visible":true,"origin":"","legend":"","description":"","filename":"Appendicesscireports.docx","url":"https://assets-eu.researchsquare.com/files/rs-2782923/v1/ccb4b82e608d6bf6f50c2286.docx"}],"financialInterests":"Competing interest reported. CAF and BKN are founders of AngioInsight, Inc, a startup company which uses machine learning and signal processing to assist physicians with angiography interpretation. AngioInsight did not sponsor this work. This work is a part of a patent which has been filed by the University of Michigan, US patent 11,386,563 Anatomical and Functional Assessment of Coronary Artery Disease Using Machine Learning. All other authors declare no competing interests.","formattedTitle":"A Multi-stage Neural Network Approach for Coronary 3D Reconstruction from Uncalibrated X-ray Angiography Images","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eThe coronary circulation is divided between the right and left coronary trees, each of which supply blood to different regions of the heart muscle [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Coronary Artery Disease (CAD) occurs when atherosclerotic plaque accumulates in the coronary arteries, leading to a local narrowing known as stenosis [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. The most common imaging modality used to diagnose CAD is X-ray angiography, in which 2D X-ray images are acquired as radio-opaque dye is injected into the coronaries. This allows the cardiologist to visualize and estimate diameter reduction at regions of stenosis.\u003c/p\u003e \u003cp\u003eAlthough anatomical assessment of stenosis severity is the most common diagnostic practice, functional metrics which take hemodynamic parameters into account through the use of invasive pressure wires placed in the coronaries, such as fractional flow reserve (FFR) and instantaneous wave-free ratio (iFR), have led to better diagnostic outcomes [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e], [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In recent years, there have been numerous efforts to derive computational estimates of those metrics or to propose new metrics such as quantitative flow ratio (QFR) that do not rely on invasive pressure wires [\u003cspan additionalcitationids=\"CR6 CR7 CR8 CR9\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u0026ndash;[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. A key step in generating computational estimates through modeling approaches is defining the coronary tree geometry of the patient. Creating such a 3D model is relatively straightforward when using 3D medical image data such as computed tomography angiography (CTA). However, 2D X-ray angiography requires a method to accurately reconstruct the 3D geometry from a series of images.\u003c/p\u003e \u003cp\u003eMany groups have proposed coronary reconstruction algorithms using the principles of projective geometry and stereovision [\u003cspan additionalcitationids=\"CR12 CR13 CR14\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u0026ndash;[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. An in-depth review of these techniques can be found in [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. Despite their success, projection-based methods have several limitations such as their reliance on user input to identify corresponding vessels or features in all the input images. They also either require precise hardware calibration during image acquisition or algorithms to correct the recorded acquisition angles, since recorded angles and distances can have up to a 10% tolerance [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Furthermore, projection-based methods are susceptible to how clearly visible all coronary branches are in each angiography image. For example, overlapping branches and vessel foreshortening are two challenges which can introduce uncertainty into the reconstructed 3D geometry.\u003c/p\u003e \u003cp\u003eRecently, a few groups have performed 3D reconstruction of coronary trees from X-ray angiography images using a combination of machine learning and projective geometry [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. While the vessel segmentation component of these methods utilized neural networks, the 3D reconstruction component relied on stereovision and projective geometry. A fully machine learning-based method has the potential to perform automated reconstruction without the need for imaging calibration.\u003c/p\u003e \u003cp\u003eThe main contribution of this work is a multi-stage neural network for 3D reconstruction from 2D binarized X-ray angiographic images which outputs an analytical representation of a 3D coronary tree. To our knowledge, this work is the first purely machine learning approach for 3D coronary tree reconstruction using X-ray angiography alone. The neural network was trained using analytical 3D coronary trees from a synthetic vessel generator. The advantage of such an analytical representation of the geometry is that it makes it easy to perform parametric hemodynamic analyses. A proof-of-concept example of such analysis is also included here.\u003c/p\u003e \u003cp\u003eIn this work, we demonstrate smaller reconstruction errors compared to a single-stage neural network (Section \u003cspan refid=\"Sec6\" class=\"InternalRef\"\u003e3.1\u003c/span\u003e) and traditional projection-based methods (Section \u003cspan refid=\"Sec7\" class=\"InternalRef\"\u003e3.2\u003c/span\u003e). Proof-of-concept hemodynamic analyses showed that acceptable errors were achieved for the clinical quantities of interest despite modest errors in the 3D geometric reconstruction (Section \u003cspan refid=\"Sec8\" class=\"InternalRef\"\u003e3.3\u003c/span\u003e).\u003c/p\u003e"},{"header":"2. Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Multi-stage Neural Network Design\u003c/h2\u003e \u003cp\u003eWe present a multi-stage neural network for 3D reconstruction of coronary trees. The input to the network is two or three binarized angiograms for a given coronary tree, with a Euclidean distance transform applied to create a smooth field that implicitly encodes vessel radii (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). In this work, binarized angiograms were directly created by our synthetic data generator (Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e). For clinical angiograms, we have previously developed a vessel segmentation neural network known as AngioNet [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] which can convert the desired clinical images into binarized angiograms. Since no angle or distance information is provided to the neural network as an input, image calibration or parameter correction algorithms are not required.\u003c/p\u003e \u003cp\u003eThe multi-stage neural network was designed to reconstruct both vessel centerlines (stage 1) and radii (stage 2) for each branch in the coronary tree. The centerline and radius stages both employed a convolutional neural network backbone to learn relevant features of the coronary tree from the input images. In this work, we chose to use ResNet101 [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] as the backbone network. While the convolutional layers can learn image-based features relevant to the vessel geometry, a multilayer perceptron (MLP) is better suited to solve the regression problems of identifying the 3D coordinates of the vessel centerline and corresponding radii. Therefore, we replaced the final layer of the backbone network with separate MLPs for the centerline and radius stages, as follows.\u003c/p\u003e \u003cp\u003eIn the centerline stage, the final fully connected layer of the backbone network was replaced by a multilayer perceptron (MLP) with ReLU activation and batch normalization between layers. The MLP was composed of 4 hidden layers, where the first 3 layers had 1024 neurons and the last layer had 512 neurons. The output of the centerline MLP was a \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M*N*3\\)\u003c/span\u003e\u003c/span\u003e linear layer, containing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N\\)\u003c/span\u003e\u003c/span\u003e centerline points for each of the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\)\u003c/span\u003e\u003c/span\u003e branches in the binarized angiogram. This output vector was reshaped into a matrix before computing the loss.\u003c/p\u003e \u003cp\u003eMeanwhile, the radius stage replaced the final layer of the backbone with a separate MLP for each branch, for a total of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\)\u003c/span\u003e\u003c/span\u003e MLPs. The radius MLPs were composed of 3 hidden layers with 128 neurons each, with batch normalization and ReLU activation between each hidden layer. The output of each MLP was a vector of radii of dimension \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N\\)\u003c/span\u003e\u003c/span\u003e. The MLP for each branch was trained separately to improve the network\u0026rsquo;s ability to capture sudden reductions in vessel radii at regions of stenosis. Without this step, stenoses are likely to be overlooked since they make up a small portion of the points in the coronary tree.\u003c/p\u003e \u003cp\u003eThe outputs of both stages were concatenated to form an \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\times N\\times 4\\)\u003c/span\u003e\u003c/span\u003e matrix, where the last dimension encodes the 3D centerline coordinate and radius for each point as \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((x,y,z,r)\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe radius stage was trained using the mean squared error as the loss function:\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}\\frac{1}{n}\\sum _{i=0}^{n}{\\left({y}_{i}- {\\widehat{y}}_{i}\\right)}^{2} \\#\\left(1\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(y\\)\u003c/span\u003e\u003c/span\u003e is the ground truth and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\widehat{y}\\)\u003c/span\u003e\u003c/span\u003e is the neural network prediction. Conversely, the centerline stage was trained using the same loss function with an additional length regularization term:\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}\\frac{1}{n}\\sum _{i=0}^{n}{\\left({y}_{i}- {\\widehat{y}}_{i}\\right)}^{2}+ \\lambda \\sum {S}_{y}- {S}_{\\widehat{y}}\\#\\left(2\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\lambda\\)\u003c/span\u003e\u003c/span\u003e is the regularization rate, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{y}\\)\u003c/span\u003e\u003c/span\u003e is the arclength of the ground truth branches, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{\\widehat{y}}\\)\u003c/span\u003e\u003c/span\u003e is the arclength of the predicted branches. This vessel length regularization term was included because vessel length is an important determinant of the pressure gradient through a vessel, an important indicator of disease severity.\u003c/p\u003e \u003cp\u003eAn ADAM optimizer with learning rate 5e-4 and weight decay (L2) regularization was used to train both stages. Batch size was set to 8 and the multi-stage network was trained for 300 epochs. For the radius MLPs, the backbone was frozen after the initial 300 epochs and each branch MLP was trained for an additional 50 epochs. We did not observe a notable improvement in accuracy when retraining the backbone network for the centerline and radius tasks.\u003c/p\u003e \u003cp\u003eWe now present the single-stage counterpart to our multi-stage network for comparison. The single-stage network architecture was composed of the backbone network and a single MLP which outputs an \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\times N\\times 4\\)\u003c/span\u003e\u003c/span\u003ematrix containing both the centerlines and their radii (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The loss function of the single-stage network was a weighted MSE loss:\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\begin{array}{c}\\frac{1}{n}\\sum _{i=0}^{n}{\\left({y}_{i}- {\\widehat{y}}_{i}\\right)}^{2}+{\\mu ({r}_{i}-\\widehat{{r}_{i}})}^{2} \\#\\left(3\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eHere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(y\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\widehat{y}\\)\u003c/span\u003e\u003c/span\u003e represent the ground truth and predicted centerline coordinates while \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(r\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\widehat{r}\\)\u003c/span\u003e\u003c/span\u003e represent the radii along the centerlines. The regularization parameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\mu\\)\u003c/span\u003e\u003c/span\u003e was chosen such that the centerline and radius terms were of the same order of magnitude. The single-stage network was trained using the same hyperparameters as the multi-stage network to make a fair comparison: ADAM optimizer with learning rate 5e-4, L2 regularization, and a batch size of 8.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Synthetic Dataset Generation\u003c/h2\u003e \u003cp\u003eTo train the proposed multi-staged neural network, we require hundreds or thousands of ground truth 3D coronary trees and their corresponding segmented 2D angiograms. In practice, this means that we must identify thousands of patients with both 3D CTA data and 2D X-ray angiograms, which is typically not feasible in single-center studies such as ours. Another challenge of using clinical image data as input is that the coronaries deform in each frame of an X-ray angiography series due to the contraction of the heart. This necessitates temporal registration of frames from multiple angiographic series in order to create a valid set of input images for 3D coronary tree reconstruction. To produce a large enough dataset and eliminate external sources of error such as temporal registration, we devised a method to produce a sufficiently large training dataset consisting of 5,000 static 3D coronary tree geometries and their corresponding sets of 2D projections. While we have used synthetic data to train and validate our 3D reconstruction network, the use of synthetic projection images as input does not preclude future clinical application. A segmentation algorithm or neural network such as AngioNet [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] could be used to convert clinical angiograms obtained during routine patient care in the future into a suitable input for our network.\u003c/p\u003e \u003cp\u003eThis work focuses on 3D reconstruction of the right coronary tree as the large anatomical variation in the left coronary arteries [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] makes reconstruction more challenging. The method includes two steps: 1) a synthetic 3D coronary tree generator, and 2) a projection algorithm to create sets of segmented angiograms. A brief description of these steps is provided next.\u003c/p\u003e \u003cp\u003e \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eStep 1: Synthetic 3D coronary tree generator\u003c/span\u003e: Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e provides an overview of the coronary tree generator. A distribution of patient-specific centerlines was obtained from 10 CTAs for the four main branches of the right coronary tree, namely the right main coronary artery (RCA), sino-atrial node branch (SA), acute marginal branch (AM), and posterior-descending artery (PDA). The posterolateral ventricular branch (PLV) is implicitly included as part of the RCA, which bifurcates into the PDA and PLV branches. These data were used to identify a distribution of controls points and their standard deviations in 3D space for each branch of the coronary tree (see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). From this distribution, new vessels can be generated via uniform random sampling. Linearly tapering radius was assigned to each branch, and stenoses with a gaussian profile were randomly introduced (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eB). Branches were combined into a tree and augmented with random rotation, shear, and/or warping (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eC). This algorithm was refined through repeated iterations with a board-certified interventional cardiologist to generate realistic trees. Further details of the clinical and mathematical assumptions used to inform synthetic data generation are given in Appendix A.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eStep 2: Projection algorithm\u003c/span\u003e: Cone-beam projections of each coronary tree were generated from 5 views to mimic the X-ray angiogram acquisition process. Image acquisition angles were randomly sampled from 20-degree windows around commonly used clinical values. Out of the 5 views, 3 were randomly chosen for training (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). A Euclidean distance transform was applied to the projection images, which were then input into the neural network. The data generated in this fashion were split into 4,500 coronary trees and their corresponding projections for training and 500 for validation (90\u0026thinsp;\u0026minus;\u0026thinsp;10 training split).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"3. Results","content":"\u003cp\u003eIn this section, we present several methods to evaluate the performance of the proposed method. We first compared the reconstruction error of our multi-stage network against a single-stage neural network to demonstrate the advantages of the multi-stage approach. We then performed a head-to-head comparison between our neural network reconstruction method and a projection-based method. Both comparisons were performed on single vessel geometries for simplicity. Next, we examined the performance of the method in reconstructing right coronary trees. Lastly, an analysis of how geometric reconstruction error affects hemodynamics is presented.\u003c/p\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Multi-stage versus Single-Stage Reconstruction performance\u003c/h2\u003e \u003cp\u003eA key design feature of our network is its multi-stage nature, which considers the tasks of centerline and radius reconstruction separately. The importance of this feature is illustrated in this section via a comparison between the single- and multi-stage networks. The performance of both networks was evaluated using 10 distinct RCA synthetic vessels. For each synthetic vessel, stenoses between 20\u0026ndash;90%, with increments of 5%, were introduced, for a total of 150 unique vessel geometries. Reconstruction error was assessed for each geometry. An example of a vessel geometry with 70% stenosis and its single- and multi-stage reconstructions is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eA. On average, the root mean squared error (RMSE) of the multi-stage network centerline was 52% lower than the single-stage centerline RMSE (0.83\u0026thinsp;\u0026plusmn;\u0026thinsp;0.29mm vs 1.73\u0026thinsp;\u0026plusmn;\u0026thinsp;0.42mm), see Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eB.\u003c/p\u003e \u003cp\u003eAs for radius reconstruction accuracy, mean absolute error (MAE) at the stenosis was evaluated instead of RMSE in the whole vessel due to the importance of accurately predicting stenosis severity. The MAE across all vessels was 0.117\u0026thinsp;\u0026plusmn;\u0026thinsp;0.068mm for the multi-stage network and 0.927\u0026thinsp;\u0026plusmn;\u0026thinsp;0.436mm for the single stage network. For both networks, the MAE increased with the stenosis severity (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eB). However, the single-stage network effectively failed to detect the stenosis for all levels of stenosis (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eA), resulting in extremely large MAE errors (over 1.5 mm) for severe stenoses with over 80% diameter reduction.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Multi-stage Centerline reconstruction against a Projection-based Method\u003c/h2\u003e \u003cp\u003eWe now compare the performance of our multi-stage neural network against a projection-based method in the simplest case of a single vessel centerline reconstruction from 2 projection images. In this example, the projection-based approach was a combination of the algorithms proposed by Banerjee et al [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] and Vukicevic et al [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Briefly, a point cloud approach [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] was used to automatically identify up to 10 possible corresponding points for each centerline point in the reference projection image. The set of possible corresponding points was further refined using the reprojection error and ordering constraint cost function proposed in [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Matched points from both images were then back-projected and interpolated with a b-spline to create a 3D centerline.\u003c/p\u003e \u003cp\u003eThe dataset of synthetic projection images for this comparison was defined as follows. 5 synthetic RCA vessels were first created. We considered a spherical coordinate system \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\((\\theta ,\\varphi )\\)\u003c/span\u003e\u003c/span\u003e, where \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\theta\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varphi\\)\u003c/span\u003e\u003c/span\u003e are the azimuthal and elevation angles, respectively. For a given vessel, the first projection image in every pair was fixed at \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\theta =-45^\\circ , \\varphi =0\\)\u003c/span\u003e\u003c/span\u003e. The second image was defined using six different intervals \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta \\in [15^\\circ , 90^\\circ ]\\)\u003c/span\u003e\u003c/span\u003e, for a total of 30 different pairs of projection images (5 different vessels, 6 different angles between projection images for each vessel), see Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eA. The reconstruction error as a function of the angle between projection images, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta\\)\u003c/span\u003e\u003c/span\u003e, was compared between both methods.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAs seen in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eB, the centerline RMSE and standard deviation was lower in the neural network approach compared to the projection-based method for all tested angles and vessels: 2.12\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05mm and 3.49\u0026thinsp;\u0026plusmn;\u0026thinsp;0.44mm, respectively. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eC shows two examples of the same vessel reconstructed using 2 pairs of projection images with different \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta\\)\u003c/span\u003e\u003c/span\u003e. The ground truth centerline (blue), neural network centerline (red), and projection-based centerline (brown) are shown. The neural network centerline followed the average path of the ground truth centerline, although it failed to capture some of the bends along its path (red arrows). Meanwhile, the brown arrows indicate gaps or regions of uncertainty in the centerline reconstructed using projection-based methods.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Reconstruction of coronary trees\u003c/h2\u003e \u003cp\u003eIn this section, we measure the accuracy of the proposed method for coronary tree reconstruction, using the validation set of 500 synthetic coronary trees defined in Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e. Figure\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows several examples of reconstructed coronary tree centerlines (red) and their corresponding ground truth (blue). We observed that the neural network learned the mean centerline path of each vessel in the tree; however, as in the single vessel case, the predicted vessels were not as tortuous as the ground truth centerlines. The RMSE between the ground truth and predicted centerline points in the validation set was 2.57\u0026thinsp;\u0026plusmn;\u0026thinsp;0.78mm. The MAE in vessel length was 8.83\u0026thinsp;\u0026plusmn;\u0026thinsp;4.81mm. Optimal values for vessel length reconstruction were obtained with a regularization length parameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\lambda =0.1\\)\u003c/span\u003e\u003c/span\u003e (see Eq.\u0026nbsp;(2)), which led to a 47% decrease in vessel length error (16.36\u0026thinsp;\u0026plusmn;\u0026thinsp;2.88 mm) compared to the same network trained without length regularization in the loss function. Larger values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\lambda\\)\u003c/span\u003e\u003c/span\u003e resulted in over-constraining the vessel length and inaccurate paths for the different branches, whereas smaller values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\lambda\\)\u003c/span\u003e\u003c/span\u003e resulted in weaker enforcement of the vessel length.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eWe now consider the error in vessel radius along the centerline for all 2,000 branches. The RMSE of the vessel radius was 0.16\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07mm, which corresponds to sub-pixel resolution. The error in radius was larger when comparing minimum stenosis diameter (MAE\u0026thinsp;=\u0026thinsp;0.27\u0026thinsp;\u0026plusmn;\u0026thinsp;0.18mm), particularly for severe stenoses greater than 70% diameter reduction, consistent with the behavior reported for single vessel reconstruction in Section \u003cspan refid=\"Sec6\" class=\"InternalRef\"\u003e3.1\u003c/span\u003e. Figure\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e shows examples of radius reconstruction (red dots) along normalized centerline position of the main RCA (blue) in vessels without stenosis (top panel), two cases of a single stenosis (mid panel), and a case with two stenoses (bottom panel). Panel A demonstrates that the neural network accurately captures the tapering of the vessel along its length. Panel B shows that the neural network accurately predicts the severity and location of the stenosis, although with some oscillations apparent in regions outside the stenosis, or a slight shift in the stenosis location. Lastly, in panel C the neural network approximates serial stenoses as a single, longer stenosis with an error of 0.3mm in stenosis severity. Some oscillations remain apparent in the region outside the stenosis.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Effect of reconstruction error on hemodynamics\u003c/h2\u003e \u003cp\u003eIn the previous examples, we established the performance of the multi-stage neural network to reconstruct centerline and radius of vessels in coronary artery trees. However, as stated in the introduction, functional assessment of vessel disease has recently received significant attention. In this section, we investigate the quality of our proposed multi-stage neural network from a functional standpoint. Towards that end, we simulate the physics of blood flow and pressure in the reconstructed coronary trees using computational fluid dynamics (CFD) simulations and compare the results against known ground truth CFD data. Specifically, we compare distributions of pressure down the RCA vessel in ground truth coronary trees (known synthetic geometries) and their reconstructions. The pressure field is the basis to calculate well-established functional metrics of CAD such as FFR, iFR, and QFR [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e], [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eCFD simulations with consistent inflow and outflow boundary conditions were run using the validated open-source software CRIMSON [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. Two examples are discussed: 1) a healthy coronary tree without stenosis, and 2) a diseased tree with one stenosis. Details of simulation parameters (e.g., inflow and outflow boundary conditions, fluid properties such as density and viscosity, etc.) are given in Appendix B. Results are summarized in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eResults for the healthy coronary tree are given in the top panel of Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e.The left column depicts solution maps of pressure calculated via CFD analysis for ground truth and reconstructed geometries. The center and right columns show the radius and pressure, respectively, versus normalized centerline position in the ground truth (blue) and reconstructed (red) geometries. In the top panel, there is no stenosis and the changes in pressure down the RCA are small. The pressure gradients over the ground truth and reconstructed RCA vessels are \u003cem\u003eDP\u003c/em\u003e\u003csub\u003e\u003cem\u003eGT\u003c/em\u003e\u003c/sub\u003e = 9.8 mmHg and \u003cem\u003eDP\u003c/em\u003e\u003csub\u003e\u003cem\u003eR\u003c/em\u003e\u003c/sub\u003e = 8.4 mmHg, resulting in a difference between pressure gradient estimates of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\Delta }P={\\Delta }{P}_{GT}-{\\Delta }{P}_{R}\\)\u003c/span\u003e\u003c/span\u003e = 1.4 mmHg (or only 1.2% of the ground truth inflow pressure).\u003c/p\u003e \u003cp\u003eIn the diseased coronary tree, the pressure gradients over the ground truth and reconstructed RCA vessels are DP\u003csub\u003eGT\u003c/sub\u003e = 18.8 mmHg and DP\u003csub\u003eR\u003c/sub\u003e = 14.3 mmHg, resulting in an error of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\Delta }P\\)\u003c/span\u003e\u003c/span\u003e= 4.5 mmHg. Additionally the pressures in the reconstructed geometry were approximately 10mmHg lower than the pressures in the ground truth. This is due to the differences in reconstructed radius in the proximal portion of the vessel, as well as the decreased tortuosity (and therefore decreased resistance) of the reconstructed coronary tree compared to the ground truth.\u003c/p\u003e \u003cp\u003eThe larger pressure error in the stenosis case compared to the healthy case can be attributed to the radius reconstruction error at the stenosis. Given a vessel with radius \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(a\\)\u003c/span\u003e\u003c/span\u003e, its resistance \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(R\\)\u003c/span\u003e\u003c/span\u003e is inversely proportional to its radius to the power of four (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(R \\tilde 1\u0026frasl;{a}^{4}\\)\u003c/span\u003e\u003c/span\u003e). Thus, relatively small errors in radius reconstruction have substantial impacts on vessel resistance and therefore pressure drop across the vessel. The ground truth stenotic geometry had serial 51% and 57% stenoses. However, the neural network approximated these 2 stenoses as a single, longer, 53% stenosis, corresponding to a radius error of 0.07mm (see center column Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e). The ratio of pressures on either side of the stenosis (a surrogate of FFR), was 0.886 in the ground truth and 0.895 in the reconstructed vessel. Therefore, despite the 10mmHg difference in inlet pressure, there was only a 1% error in the clinical quantity of interest.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eReconstructing 3D geometries from sets of 2D image data is a challenging task that often necessitates accurate knowledge of imaging parameters such as angles and distances during image acquisition. Uncertainty of these parameters poses a large challenge for projective geometry approaches to reconstruct the coronaries from X-ray angiography images; therefore, we have developed and tested a novel multi-stage neural network method for 3D coronary reconstruction.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Considerations of Neural Network Design\u003c/h2\u003e \u003cp\u003eThe most salient feature of the proposed network is its multi-stage nature, which enables accurate reconstruction of centerline paths and vessel radii. When considering the single-stage approach to reconstructing vessel geometry, we observed that the network was able to accurately capture the trend of linearly decreasing radius along the vessel but could not capture the stenoses (Section \u003cspan refid=\"Sec6\" class=\"InternalRef\"\u003e3.1\u003c/span\u003e). This is likely because the stenosis radii make up less than 2% of the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N\\times 4\\)\u003c/span\u003e\u003c/span\u003e vessel matrix; the stenoses had such a small contribution to the loss function that the network did not learn to predict them. In the multi-stage approach, the stenosis radii had a much larger contribution to the loss function since the mean squared error of the radii was optimized separately from the centerlines. The network was therefore able to learn and accurately reconstruct stenosis severity.\u003c/p\u003e \u003cp\u003eThis observation motivated going one step further for the coronary trees and using a separate MLP to learn the radii for each vessel. This was necessary since not all vessel branches contained stenoses; if the network was trained to predict the radii of all branches at the same time, the stenoses would once again make up less than 2% of the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\times N\\)\u003c/span\u003e\u003c/span\u003e output matrix. A major challenge of reconstructing stenoses is that, while they are the most important diagnostic feature of the image, they only make up a small proportion of the vessel matrix. The multi-stage approach described in this work amplifies the impact of the stenoses and thus improves prediction accuracy.\u003c/p\u003e \u003cp\u003eThe size of the MLPs for the centerline and radius tasks was determined by testing networks with a varying number of hidden layers and neurons per layer. Networks with 128, 256, 512, 1024, 2048, and 4096 neurons in each of 1\u0026ndash;4 hidden layers were evaluated for both centerline and radius accuracy. For the centerline MLPs, the minimum error was achieved when using 4 hidden layers. The number of neurons per layer did not greatly affect mean squared error, which ranged from 2.72mm to 2.91mm in the validation set of 500 synthetic coronary trees. Therefore, a network with 1024 neurons per layer was chosen as it was of the same order of magnitude as the network output size (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(M\\)\u003c/span\u003e\u003c/span\u003e*\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N\\)\u003c/span\u003e\u003c/span\u003e*3).\u003c/p\u003e \u003cp\u003eFor the radius MLPs, the lowest error was achieved in networks with 3 hidden layers. Interestingly, although overall mean squared error for the radii was similar for networks with different widths (0.02-0.03mm), smaller networks performed better at regions of stenosis. The MAE at the stenosis for a network with 128 neurons was 0.27mm, compared to 0.40mm and 0.42mm for networks with 2048 and 4096 neurons, respectively. Thus, we determined that the optimal size for the radius network was 3 hidden layers with 128 neurons each.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Comparison with Projection-based Methods of Reconstruction\u003c/h2\u003e \u003cp\u003e3D reconstruction of coronary trees from 2D X-ray angiographic images has been generally performed using projective geometry and stereovision techniques [\u003cspan additionalcitationids=\"CR12 CR13 CR14\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u0026ndash;[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. Despite their success, projection-based methods have limitations such as the need to identify corresponding features in the input images, and precise information on the image acquisition angles. Overlapping branches and foreshortening are two key challenges leading to uncertainty in the reconstructed geometry using projective geometry methods.\u003c/p\u003e \u003cp\u003eThe key contribution of this paper is a neural network for reconstructing 3D coronary trees that aims to overcome the limitations of projective geometry methods. Our network was trained over a wide range of angles between images \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta\\)\u003c/span\u003e\u003c/span\u003e with the purpose of making it robust (and therefore relatively insensitive to this parameter). In Section \u003cspan refid=\"Sec7\" class=\"InternalRef\"\u003e3.2\u003c/span\u003e, we compared the performance of our multi-stage neural network against a projection-based method in a synthetic dataset of pairs of projection images with angles between images \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta \\in \\left[15^\\circ , 90^\\circ \\right]\\)\u003c/span\u003e\u003c/span\u003e. Our network\u0026rsquo;s RMSE was on average 39% lower than the projection method across all values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varDelta \\theta ,\\)\u003c/span\u003e\u003c/span\u003eeven though the neural network was not trained on images at these specific projection angles. Together with the lower standard deviation, this could indicate that the neural network is more robust to different characteristics of the input images. For instance, different amounts of foreshortening or overlap may occur for a given vessel when projections are taken at different angles. While the projection method reconstructions are less accurate when the input images have high levels of foreshortening or overlap, the neural network could learn universal characteristics of the RCA vessel shape to overcome these challenges and maintain similar accuracy for all input images.\u003c/p\u003e \u003cp\u003eThe higher robustness to foreshortening and overlap in the input images of the neural network method is further supported by the qualitative examples in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eC. The brown arrows, which point to gaps or regions of uncertainty in the projection method centerline, correspond to regions on the input image where the vessel is highly foreshortened or bending at an angle that obscures other parts of the vessel. The neural network interpolates the 3D centerline in these regions, resulting in a continuous path.\u003c/p\u003e \u003cp\u003eThe main limitation of the neural network compared to projection methods is that it does not capture all the bends in the vessel centerline path despite accurately following the average path. The lack of tortuosity leads to a shorter reconstructed vessel than the ground truth, despite the length regularization term in the loss function (Eq.\u0026nbsp;2). This behavior could be a consequence of the network learning the features of a typical 3D RCA shape. While these features ensure that the network can interpolate the 3D centerline even when the input images are challenging, they also may cause the network to predict similar, smooth centerline paths for all inputs. Meanwhile, the projection-based method can accurately capture the details specific to each centerline path using the features of the input images.\u003c/p\u003e \u003cp\u003eThe neural network was trained using a MSE loss, which only accounts for point-wise differences between ground truth and reconstructed centerlines. A loss function that incorporates more global shape information such as curvature in addition to the vessel length regularization could improve the centerline path accuracy.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Reconstruction of Coronary Trees\u003c/h2\u003e \u003cp\u003eWhen applying the multi-stage neural network to reconstructing a whole coronary tree, the reconstructions captured the average path of all branches in the tree but under-estimated their tortuosity, similar to the behavior observed for single-vessel reconstruction. This appears to correspond to a well-known curve-fitting regression phenomenon: when the network is not large enough to capture the full complexity of a dataset, it tends to predict its average behavior. We hypothesized that increasing the complexity of the neural network by increasing the size of the MLP may alleviate this issue. We tested this hypothesis by performing a scaling study in Section \u003cspan refid=\"Sec11\" class=\"InternalRef\"\u003e4.1\u003c/span\u003e to test the performance of networks with 128 to 4096 neurons in each of 1\u0026ndash;4 hidden layers. From this scaling study, we observed 1% decrease in RMSE between the 4096x4 network and 128x1 network. Therefore, the computational overhead of increasing the MLP size did not justify the modest decrease in error. As discussed in Section \u003cspan refid=\"Sec12\" class=\"InternalRef\"\u003e4.2\u003c/span\u003e, a loss function which incorporates global shape may be a more effective strategy to improve centerline accuracy.\u003c/p\u003e \u003cp\u003eAnother challenge of coronary tree reconstruction compared to single vessel reconstruction was to accurately predict stenoses radius. As described in Section \u003cspan refid=\"Sec11\" class=\"InternalRef\"\u003e4.1\u003c/span\u003e, employing a multi-stage approach, and training a separate MLP for each vessel branch enabled the network to learn how to predict stenoses. Despite this, the MAE at the stenosis was higher for coronary trees (0.27mm) compared to a single vessel (0.12mm). This may be due to the random projection angles, as overlapping branches may have obscured the stenosis in some of the input images.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Effect of reconstruction error on hemodynamics\u003c/h2\u003e \u003cp\u003eThe last application example explored the impact of geometric reconstruction errors on hemodynamic indices such as pressure. The simple test examples demonstrated that gradients of pressure (a surrogate of FFR) compared well between ground truth and reconstructed geometries, even though the neural network centerlines were not as tortuous as the ground truth.\u003c/p\u003e \u003cp\u003eThe error in radius had a much larger impact on the pressure drop than the centerline error, particularly errors in stenosis radius. As seen in the stenosis example, the neural network approximated two serial stenoses (51% and 57% radius reduction) as a single stenosis with 53% radius reduction. Despite only a 0.07mm difference in stenosis radius, there was a noticeable difference in pressure profiles across the stenosis between the 2 geometries. However, the gradients of pressure across the stenosis were relatively similar between the 2 cases (less than 1% difference). We hypothesize that accurately capturing the overall vessel lengths and stenosis radii contributed to the accurate estimation of gradients across the vessels. However, at higher levels of stenosis, even small errors in reconstructed radius can lead to large differences in pressure gradients across the vessels.\u003c/p\u003e \u003cp\u003eGiven the high sensitivity of vessel resistance to the reconstructed vessel radius (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(R\\tilde1/{a}^{4}\\)\u003c/span\u003e\u003c/span\u003e), it is critically important to accurately determine vessel radius at the stenosis. The main limitation of improving radius estimation is the resolution of the X-ray angiography image. In an angiogram, the difference between a 70% and 75% stenosis cannot be distinguished since it is smaller than the pixel resolution, and thus the projections of both levels of stenosis appear identical. It is therefore difficult to differentiate between stenoses at a higher resolution even with a deep neural network. Super-resolution algorithms are a potential solution, as they have successfully been used to overcome the limitations of pixel size in magnetic resonance imaging [\u003cspan additionalcitationids=\"CR25\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e]\u0026ndash;[\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003e4.5 Future Work\u003c/h2\u003e \u003cp\u003eFuture work will focus on increasing the scope of the dataset to match real-world data more closely. In terms of synthetic data generation, we will expand our tree generator to incorporate further variations in the number and types of branches in the right and left coronary trees. We will also explore alternate strategies to encode the 3D coronary geometry and stenoses, since training a separate MLP to learn the radius along each branch may become infeasible in tree structures with a large number of branches. For example, the vessel radii could be encoded in a vector containing the proximal radius, distal radius, and parametric position and severity of the stenoses for each branch. Besides the synthetic projection images considered in this paper, our multi-stage neural network can be applied to clinical X-ray angiograms that have been processed with a segmentation algorithm to create binary input images. Clinical angiograms bring additional challenges, such as the motion of the coronary tree during the cardiac cycle. Care must be taken to identify angiographic frames acquired at the same point in the cardiac cycle to reconstruct a valid 3D coronary tree from clinical images.\u003c/p\u003e \u003cp\u003eIn the catheterization lab, it is common practice to move the patient table during image acquisition to follow the flow of injected dye. Since the motion is controlled via a joystick and not often recorded, clinical angiograms are challenging to use as inputs for projection-based reconstruction methods because they require precise knowledge of image acquisition parameters. Neural networks may therefore provide a robust alternative for 3D reconstruction of coronary trees when those parameters are unknown.\u003c/p\u003e \u003cp\u003eIn conclusion, in this work we have presented a proof-of-concept of a multi-stage neural network that can be used for 3D reconstruction of coronary trees from sets of uncalibrated X-ray angiography images, with sub-pixel resolution of vessel radius. We have demonstrated that reconstruction error is at an acceptable level to accurately model hemodynamic quantities such as pressure gradients across a vessel.\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003eAcknowledgments\u003c/p\u003e\n\u003cp\u003eThe authors acknowledge the Precision Health at the University of Michigan for providing cost-free, member access to GPUs and computing resources on the Armis2 cluster\u0026mdash;necessary resources to support the neural network training reported in this publication. Computing resources for hemodynamics simulations were provided by the National Science Foundation [Grant 1531752], Acquisition of Conflux. This work was supported by the National Science Foundation Graduate Research Fellowship Program [DGE1841052] and Rackham Merit Fellowship.\u003c/p\u003e\n\u003cp\u003eData and Code Availability\u003c/p\u003e\n\u003cp\u003eThe code for the synthetic coronary generator can be found on GitHub: https://github.com/kritiyer/vessel_tree_generator/\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCompeting Interest Statement\u003c/p\u003e\n\u003cp\u003eCAF and BKN are founders of AngioInsight, Inc, a startup company which uses machine learning and signal processing to assist physicians with angiography interpretation. AngioInsight did not sponsor this work. This work is a part of a patent which has been filed by the University of Michigan, US patent 11,386,563 \u003cem\u003eAnatomical and Functional Assessment of Coronary Artery Disease Using Machine Learning\u003c/em\u003e. All other authors declare no competing interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eW. W. Nichols, M. F. O\u0026rsquo;Rourke, C. Vlachopoulos, and D. A. McDonald, \u003cem\u003eMcDonald\u0026rsquo;s Blood Flow in Arteries : Theoretical, Experimental and Clinical Principles\u003c/em\u003e. London: Hodder Arnold, 2011. Available: https://www.crcpress.com/McDonalds-Blood-Flow-in-Arteries-Theoretical-Experimental-and-Clinical/Vlachopoulos-ORourke-Nichols/p/book/9780340985014\u003c/li\u003e\n\u003cli\u003eN. H. Pijls, J. A. van Son, R. L. Kirkeeide, B. De Bruyne, and K. L. Gould, \u0026ldquo;Experimental basis of determining maximum coronary, myocardial, and collateral blood flow by pressure measurements for assessing functional stenosis severity before and after percutaneous transluminal coronary angioplasty.,\u0026rdquo; \u003cem\u003eCirculation\u003c/em\u003e, vol. 87, no. 4, pp. 1354\u0026ndash;67, Apr. 1993, doi: 10.1161/01.CIR.87.4.1354.\u003c/li\u003e\n\u003cli\u003eW. F. Fearon \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Clinical Outcomes and Cost-Effectiveness of Fractional Flow Reserve\u0026ndash;Guided Percutaneous Coronary Intervention in Patients With Stable Coronary Artery Disease: Three-Year Follow-Up of the FAME 2 Trial (Fractional Flow Reserve Versus Angiography for Multivessel Evaluation),\u0026rdquo; \u003cem\u003eCirculation\u003c/em\u003e, vol. 137, no. 5, pp. 480\u0026ndash;487, Jan. 2018, doi: 10.1161/CIRCULATIONAHA.117.031907.\u003c/li\u003e\n\u003cli\u003eM. G\u0026ouml;tberg \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Instantaneous Wave-free Ratio versus Fractional Flow Reserve to Guide PCI,\u0026rdquo; \u003cem\u003eNew England Journal of Medicine\u003c/em\u003e, vol. 376, no. 19, pp. 1813\u0026ndash;1823, May 2017, doi: 10.1056/NEJMoa1616540.\u003c/li\u003e\n\u003cli\u003eW. F. Fearon \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Accuracy of Fractional Flow Reserve Derived From Coronary Angiography,\u0026rdquo; \u003cem\u003eCirculation\u003c/em\u003e, Jan. 2019, doi: 10.1161/CIRCULATIONAHA.118.037350.\u003c/li\u003e\n\u003cli\u003eB. Xu \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Diagnostic Accuracy of Angiography-Based Quantitative Flow Ratio Measurements for Online Assessment of Coronary Stenosis,\u0026rdquo; \u003cem\u003eJournal of the American College of Cardiology\u003c/em\u003e, vol. 70, no. 25, pp. 3077\u0026ndash;3087, 2017, doi: 10.1016/j.jacc.2017.10.035.\u003c/li\u003e\n\u003cli\u003eJ. M. Carson, C. Roobottom, R. Alcock, and P. Nithiarasu, \u0026ldquo;Computational instantaneous wave-free ratio (IFR) for patient-specific coronary artery stenoses using 1D network models,\u0026rdquo; \u003cem\u003eInternational Journal for Numerical Methods in Biomedical Engineering\u003c/em\u003e, vol. 35, no. 11, p. e3255, 2019, doi: 10.1002/cnm.3255.\u003c/li\u003e\n\u003cli\u003eC. A. Taylor, T. A. Fonte, and J. K. Min, \u0026ldquo;Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: Scientific basis,\u0026rdquo; \u003cem\u003eJournal of the American College of Cardiology\u003c/em\u003e, vol. 61, no. 22, pp. 2233\u0026ndash;2241, 2013, doi: 10.1016/j.jacc.2012.11.083.\u003c/li\u003e\n\u003cli\u003eJ. Jiang \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Fractional flow reserve for coronary stenosis assessment derived from fusion of intravascular ultrasound and X-ray angiography,\u0026rdquo; \u003cem\u003eQuant Imaging Med Surg\u003c/em\u003e, vol. 11, no. 11, pp. 4543\u0026ndash;4555, Nov. 2021, doi: 10.21037/qims-20-1324.\u003c/li\u003e\n\u003cli\u003eJ. Li \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Accuracy of computational pressure-fluid dynamics applied to coronary angiography to derive fractional flow reserve: FLASH FFR,\u0026rdquo; \u003cem\u003eCardiovascular Research\u003c/em\u003e, vol. 116, no. 7, pp. 1349\u0026ndash;1356, Jun. 2020, doi: 10.1093/cvr/cvz289.\u003c/li\u003e\n\u003cli\u003eA. Banerjee, F. Galassi, E. Zacur, G. L. D. Maria, R. P. Choudhury, and V. Grau, \u0026ldquo;Point-Cloud Method for Automated 3D Coronary Tree Reconstruction From Multiple Non-Simultaneous Angiographic Projections,\u0026rdquo; \u003cem\u003eIEEE Transactions on Medical Imaging\u003c/em\u003e, vol. 39, no. 4, pp. 1278\u0026ndash;1290, Apr. 2020, doi: 10.1109/TMI.2019.2944092.\u003c/li\u003e\n\u003cli\u003eA. M. Vukicevic, S. \u0026Ccedil;imen, N. Jagic, G. Jovicic, A. F. Frangi, and N. Filipovic, \u0026ldquo;Three-dimensional reconstruction and NURBS-based structured meshing of coronary arteries from the conventional X-ray angiography projection images,\u0026rdquo; \u003cem\u003eScientific Reports\u003c/em\u003e, vol. 8, no. 1, Dec. 2018, doi: 10.1038/s41598-018-19440-9.\u003c/li\u003e\n\u003cli\u003eC. V. Bourantas \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;A method for 3D reconstruction of coronary arteries using biplane angiography and intravascular ultrasound images,\u0026rdquo; \u003cem\u003eComputerized Medical Imaging and Graphics\u003c/em\u003e, vol. 29, no. 8, pp. 597\u0026ndash;606, Dec. 2005, doi: 10.1016/j.compmedimag.2005.07.001.\u003c/li\u003e\n\u003cli\u003eS. J. Chen and J. D. Carroll, \u0026ldquo;3-D Reconstruction of Coronary Arterial Tree to Optimize Angiographic Visualization,\u0026rdquo; \u003cem\u003eIEEE Transactions on Medical Imaging\u003c/em\u003e, vol. 19, no. 4, pp. 318\u0026ndash;336, Apr. 2000, doi: 10.1109/42.848183.\u003c/li\u003e\n\u003cli\u003eJ. Yang, W. Cong, Y. Chen, J. Fan, Y. Liu, and Y. Wang, \u0026ldquo;External force back-projective composition and globally deformable optimization for 3-D coronary artery reconstruction,\u0026rdquo; \u003cem\u003ePhys. Med. Biol.\u003c/em\u003e, vol. 59, no. 4, pp. 975\u0026ndash;1003, Feb. 2014, doi: 10.1088/0031-9155/59/4/975.\u003c/li\u003e\n\u003cli\u003eS. \u0026Ccedil;imen, A. Gooya, M. Grass, and A. F. Frangi, \u0026ldquo;Reconstruction of coronary arteries from X-ray angiography: A review.,\u0026rdquo; \u003cem\u003eMedical image analysis\u003c/em\u003e, vol. 32, pp. 46\u0026ndash;68, Aug. 2016, doi: 10.1016/j.media.2016.02.007.\u003c/li\u003e\n\u003cli\u003eSiemens AG, \u0026ldquo;Artis Q/Q.zen/zeego System Owners Manual.\u0026rdquo; 2015.\u003c/li\u003e\n\u003cli\u003eD. M. Bappy, A. Hong, E. Choi, J.-O. Park, and C.-S. Kim, \u0026ldquo;Automated three-dimensional vessel reconstruction based on deep segmentation and bi-plane angiographic projections,\u0026rdquo; \u003cem\u003eComputerized Medical Imaging and Graphics\u003c/em\u003e, vol. 92, p. 101956, Sep. 2021, doi: 10.1016/j.compmedimag.2021.101956.\u003c/li\u003e\n\u003cli\u003eK. Iyer \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;AngioNet: a convolutional neural network for vessel segmentation in X-ray angiography,\u0026rdquo; \u003cem\u003eSci Rep\u003c/em\u003e, vol. 11, Sep. 2021, doi: 10.1038/s41598-021-97355-8.\u003c/li\u003e\n\u003cli\u003eK. He, X. Zhang, S. Ren, and J. Sun, \u0026ldquo;Deep Residual Learning for Image Recognition,\u0026rdquo; Dec. 2015. Available: http://arxiv.org/abs/1512.03385\u003c/li\u003e\n\u003cli\u003eF. Cademartiri \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Prevalence of anatomical variants and coronary anomalies in 543 consecutive patients studied with 64-slice CT coronary angiography,\u0026rdquo; \u003cem\u003eEur Radiol\u003c/em\u003e, vol. 18, no. 4, pp. 781\u0026ndash;791, Apr. 2008, doi: 10.1007/s00330-007-0821-9.\u003c/li\u003e\n\u003cli\u003eN. H. Pijls \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Measurement of fractional flow reserve to assess the functional severity of coronary-artery stenoses,\u0026rdquo; \u003cem\u003eN Engl J Med\u003c/em\u003e, vol. 334, no. 26, pp. 1703\u0026ndash;1708, Jun. 1996, doi: 10.1056/NEJM199606273342604.\u003c/li\u003e\n\u003cli\u003eC. J. Arthurs \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;CRIMSON: An open-source software framework for cardiovascular integrated modelling and simulation,\u0026rdquo; \u003cem\u003ePLOS Computational Biology\u003c/em\u003e, vol. 17, no. 5, p. e1008881, May 2021, doi: 10.1371/journal.pcbi.1008881.\u003c/li\u003e\n\u003cli\u003eE. Ferdian \u003cem\u003eet al.\u003c/em\u003e, \u0026ldquo;Cerebrovascular super-resolution 4D Flow MRI \u0026ndash; using deep learning to non-invasively quantify velocity, flow, and relative pressure.\u0026rdquo; bioRxiv, p. 2021.08.25.457611, Aug. 27, 2021. doi: 10.1101/2021.08.25.457611.\u003c/li\u003e\n\u003cli\u003eY. Chen, F. Shi, A. G. Christodoulou, Y. Xie, Z. Zhou, and D. Li, \u0026ldquo;Efficient and Accurate MRI Super-Resolution Using a Generative Adversarial Network and 3D Multi-level Densely Connected Network,\u0026rdquo; in \u003cem\u003eMedical Image Computing and Computer Assisted Intervention \u0026ndash; MICCAI 2018\u003c/em\u003e. Springer International Publishing, 2018, pp. 91\u0026ndash;99. doi: 10.1007/978-3-030-00928-1_11.\u003c/li\u003e\n\u003cli\u003eH. Greenspan, G. Oz, N. Kiryati, and S. Peled, \u0026ldquo;MRI inter-slice reconstruction using super-resolution,\u0026rdquo; \u003cem\u003eMagnetic Resonance Imaging\u003c/em\u003e, vol. 20, no. 5, pp. 437\u0026ndash;446, Jun. 2002, doi: 10.1016/S0730-725X(02)00511-8.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-2782923/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2782923/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWe present a multi-stage neural network approach for 3D reconstruction of coronary artery trees from uncalibrated 2D X-ray angiography images. This method uses several binarized images from different angles to reconstruct a 3D coronary tree without any knowledge of image acquisition parameters. The method consists of a single backbone network and separate stages for vessel centerline and radius reconstruction. The output is an analytical matrix representation of the coronary tree suitable for downstream applications such as hemodynamic modeling of local vessel narrowing (i.e., stenosis). The network was trained using a dataset of synthetic coronary trees from a vessel generator informed by both clinical image data and literature values on coronary anatomy. Our multi-stage network achieved sub-pixel accuracy in reconstructing vessel radius (RMSE\u0026thinsp;=\u0026thinsp;0.16\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07mm) and stenosis radius (MAE\u0026thinsp;=\u0026thinsp;0.27\u0026thinsp;\u0026plusmn;\u0026thinsp;0.18mm), the most important feature used to inform diagnostic decisions. The network also led to 52% and 38% reduction in vessel centerline reconstruction errors compared to a single-stage network and projective geometry-based methods, respectively. Our method demonstrated robustness to overcome challenges such as vessel foreshortening or overlap in the input images. This work is an important step towards automated analysis of anatomic and functional disease severity in the coronary arteries.\u003c/p\u003e","manuscriptTitle":"A Multi-stage Neural Network Approach for Coronary 3D Reconstruction from Uncalibrated X-ray Angiography Images","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-04-18 14:36:11","doi":"10.21203/rs.3.rs-2782923/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2023-07-28T07:03:46+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2023-07-07T07:55:54+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"b772e496-7fe2-42f0-b34b-08a31417f671","date":"2023-07-06T16:18:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"6ac368d3-dda9-4fa4-a182-4ab7e4049751","date":"2023-07-02T21:49:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"16e00aa0-2482-4f52-8e48-2d63a050d056","date":"2023-07-02T18:56:32+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2023-06-19T14:45:34+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2023-06-19T14:43:49+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2023-04-16T11:00:43+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2023-04-16T10:51:34+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2023-04-05T23:28:57+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a1973c35-be70-4372-9661-d2b487985c87","owner":[],"postedDate":"April 18th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":20744391,"name":"Biological sciences/Physiology/Cardiovascular biology/Cardiovascular diseases/Acute coronary syndromes"},{"id":20744392,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":20744393,"name":"Physical sciences/Engineering/Biomedical engineering"},{"id":20744394,"name":"Physical sciences/Physics/Fluid dynamics"},{"id":20744395,"name":"Physical sciences/Mathematics and computing/Computational science"}],"tags":[],"updatedAt":"2023-10-23T15:06:47+00:00","versionOfRecord":{"articleIdentity":"rs-2782923","link":"https://doi.org/10.1038/s41598-023-44633-2","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2023-10-16 15:01:44","publishedOnDateReadable":"October 16th, 2023"},"versionCreatedAt":"2023-04-18 14:36:11","video":"","vorDoi":"10.1038/s41598-023-44633-2","vorDoiUrl":"https://doi.org/10.1038/s41598-023-44633-2","workflowStages":[]},"version":"v1","identity":"rs-2782923","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2782923","identity":"rs-2782923","version":["v1"]},"buildId":"rHA-KDH7Qsr4HCuvH75dn","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}