3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology

3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article 3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology Isaac Tian, Jason Liu, Michael Wong, Nisa Kelly, Yong Liu, Andrea Garber, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3935042/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 02 Feb, 2025 Read the published version in npj Digital Medicine → Version 1 posted 10 You are reading this latest preprint version Abstract Total and regional body composition are strongly correlated with metabolic syndrome and have been estimated non-invasively from 3D optical scans using linear parameterizations of body shape and linear regression models. Prior works produced accurate and precise predictions on many, but not all, body composition targets relative to the reference dual X-Ray absorptiometry (DXA) measurement. Here, we report the effects of replacing linear models with nonlinear parameterization and regression models on the precision and accuracy of body composition estimation in a novel application of deep 3D convolutional graph networks to human body composition modeling. We assembled an ensemble dataset of 4286 topologically standardized 3D optical scans from four different human body shape databases, DFAUST, CAESAR, Shape Up! Adults, and Shape Up! Kids and trained a parameterized shape model using a graph convolutional 3D autoencoder (3DAE) in lieu of linear PCA. We trained a nonlinear Gaussian process regression (GPR) on the 3DAE parameter space to predict body composition via correlations to paired DXA reference measurements from the Shape Up! scan subset. We tested our model on a set of 424 randomly withheld test meshes and compared the effects of nonlinear computation against prior linear models. Nonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Deep shape features produced 6-8% reduction in prediction error over linear PCA features for males only and a 4-14% reduction in precision error for both sexes. Our best performing nonlinear model predicting body composition from deep features outperformed prior work using linear methods on all tested body composition prediction metrics in both precision and accuracy. All coefficients of determination (R 2 ) for all predicted variables were above 0.86. We show that GPR is a more precise and accurate method for modeling body composition mappings from body shape features than linear regression. Deep 3D features learned by a graph convolutional autoencoder only improved male body composition accuracy but improved precision in both sexes. Our work achieved lower estimation RMSEs than all previous work on 10 metrics of body composition. Health sciences/Health care/Medical imaging/Three-dimensional imaging Health sciences/Health care/Weight management 3D Scanning Body Composition Machine Learning Autoencoder Gaussian Process Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 INTRODUCTION Total and regional body composition are correlated with many of the leading causes of death in the US and around the world. [ 1 – 4 ] High visceral fat deposits doubled the risk for metabolic syndrome in males with otherwise normal BMIs [ 4 ] and increased mortality rates from cancer by up to 63% [ 5 ]. Metabolic syndrome, a condition indicated by an array of biological and physical vital measurements such as blood glucose and waist circumference, is strongly associated with many chronic conditions such as cancer, heart failure, and diabetes [ 6 – 8 ]. Management of these conditions is also heavily impacted by body composition, specifically low lean mass, which predicts poor treatment outcomes [ 9 , 10 ] and up to 17-fold increased mortality [ 11 , 12 ]. However, body composition assessment historically required more advanced instruments such as dual X-ray absorptiometry (DXA) or air displacement plethysmography (ADP) [ 13 ]. Unlike ADP, DXA can measure regional compartmental fat and lean deposits but exposes participants to potentially harmful ionizing radiation and is not recommended for frequently repeated measurements, particularly in at-risk groups such as young children and pregnant women. Radiation exposure must be limited to exigent circumstances even in healthy adults. An ideal alternative assessment system should achieve accuracy and precision on total and regional body composition measurement comparable to DXA without utilizing ionizing radiation. The system should be relatively inexpensive and easy to use, requiring no special training or certifications to operate and returning results in a minute or less. Recent work showed that 3D optical (3DO) imaging can serve as an accurate and precise, low-cost, noninvasive surrogate to DXA imaging [ 14 – 16 ]. 3DO measures the surface geometry of the human body using light in the optical spectrum as opposed to the penetrating radiation of DXA. It does not require the injection of an isotope contrast like MRI and can scan an entire adult body in under one minute. Scanning systems cost on the order of $ 10,000 and are programmed by the manufacturer to operate automatically without the need for certified technicians. The external 3D shape of a human body contains strong signals about its internal structure and composition that can be learned by machine learning algorithms. Recent advances in 3D scanning technology have made 3DO scanning of human bodies more accessible and widely distributed than ever [ 16 , 17 ]. However, current work on learning body composition from 3DO scans relies on linear mathematical models, such as principal component analysis (PCA) and linear regression. These simplified linear assumptions impose potentially erroneous prior assumptions on the parameterization of both the 3D shape model and the mapping between shape parameters and body composition. Nonlinear methods that relax restrictive assumptions on the estimated functions may align better with the true relationship between shape and body composition and provide better prediction accuracy of target variables. An accurate model for estimating total and regional body composition metrics from 3DO scans could standardize body composition assessment by removing the costs and risks associated with clinical evaluation and irradiation. We propose to use deep, nonlinear methods to better estimate shape parameterization and body composition relative to prior work with linear baselines. In this work, we trained a deep 3D graph convolutional autoencoder on a diverse sample of full-body 3DO scans. A subset of these scans was captured with paired DXA scans for ground truth body composition training targets. We trained a regression model from extracted 3D graph convolutional features in the deep autoencoder network to DXA body composition variables using a nonlinear Gaussian process regression (GPR). We observed the effect of manipulating depths and dimensions of convolutional features on prediction accuracy and compared nonlinear estimates of 3D reconstruction error and body composition prediction precision and accuracy to results produced by linear methods from prior works. To our knowledge, this is the first application of a 3D graph convolutional autoencoder to body composition prediction. Our results indicate that nonlinear regression using a GPR with a squared dot product kernel provides greater accuracy and precision in body composition estimation on test data than previous linear methods. METHODS This work performs an ablation study between linear and nonlinear methods for shape modeling and body composition estimation of 3D human body shape. We assembled a composite dataset of 3DO human body shape from four independent sources and standardized them to a common topology. We trained a deep 3D autoencoder (3DAE) based on the work of Zhou et al. [ 18 ] on the composite dataset at multiple parameter sizes and evaluated its 3D reconstruction error on test data. 10 variables of body composition were studied consistent with prior work with reference values obtained with same-day DXA scans. The variables studied were: Total body fat mass Total body lean mass Visceral fat mass Arm lean mass Leg lean mass Percent body fat Trunk fat mass Trunk lean mass Arm fat Mass Arm lean mass We trained nonlinear GPR models from 3DAE deep features at multiple scales to predict reference DXA values. We then trained a PCA model as a linear baseline using the same training data and parameter counts as the nonlinear model as an ablation study against prior work. From these linear PCA features we trained linear least squares regression and nonlinear GPR models to create linear-linear and hybrid linear-nonlinear models for DXA body composition prediction. This iterative modeling allowed us to quantify the impact of novel applications of nonlinear methods against linear baselines. Experimental Cohort Four data sets were used for this study: CAESAR [ 19 ], Shape Up! Adults (SUA) (NIH R01 DK109008) [ 14 ], Shape Up! Kids (SUK) (NIH R01 DK111698) [ 20 ], and DFAUST [ 21 ]. Table 1 shows how this data was subdivided to train and test our models. The CAESAR 3DO scan dataset represented 2400 American and Canadian adults aged 18–65 3D scanned in a neutral A-pose, with legs and arms held fully extended and abducted ~ 30 degrees from the midline of the body. Participants were mostly unclothed except for form-fitting gray underwear and a swim cap to standardize hair appearance. Roughly half of the recruited cohort were female. Sex, height, and weight were the only demographic variables used to construct the shape model; no other demographic collected was used in this study. Subjects were scanned on a custom-built Cyberwar WB4 3D scanner. The SUA and SUK datasets (ClinicalTrials.gov ID NCT03706612 (SUK) and ID NCT03637855 (SUA)) were cross-sectional and stratified by age (SUK: 5–8, 9–12, 13–17 year., SUA: 18–39, 40–59, ≥ 60 year.), ethnicity (non-Hispanic white, non‐Hispanic black, Hispanic, Asian, and Native Hawaiian or other Pacific Islander (NHOPI)), sex, and BMI Z-score. Along with extensive demographics, quantitative measures included whole body DXA scans and 3DO scans. We acquired duplicate whole-body DXA scans of each participant on either a Hologic Horizon/A system (UCSF) or a Discovery/A system (PBRC and UHCC) (Hologic Inc., Marlborough, MA, USA). Participants were positioned and scanned according to guidelines specified by the respective system manufacturers. All scans were analyzed at UHCC by a single certified technologist using Hologic Apex version 5.6 with the National Health and Nutrition Examination Survey (NHANES) Body Composition Analysis calibration option disabled. DXA systems quality control was performed by monitoring the weekly values of the Hologic Whole Body Phantom. Two independent 3DO scans were acquired for each participant in up to three scanning devices: Fit3D Proscanner 4.x, Fit3D Inc, Redwood City, CA, USA, Styku S100 4.1, Styku LLC, Los Angeles, CA, USA, and Size Stream SS20, Size Stream, Cary, NC, USA. All 3DO scans were captured in a neutral A-pose that closely mirrored the pose of the CAESAR dataset. The DFAUST dataset was a 4D capture of human pose. A templated mesh was registered to over 41,000 snapshots from continuous sequence of 3D point clouds (3dMD LLC, Atlanta, GA) of 10 unique individuals performing dynamic movements captured at 60 frames per second. 4264 meshes were reserved for testing and were not used in this study. No clinical data such as body composition was reported. The DFAUST dataset was used to pretrain the 3DAE model. The pretraining step initialized the network weights to a state similar to the presentation in the original work of Zhou et al. [ 18 ] An ensemble dataset of CAESAR, SUA, and SUK was used as training data to finetune the model. During deep network training, a held-out data split, the evaluation set, was reserved to benchmark the performance of the model at the conclusion of each training epoch as determined by its geometric reconstruction loss. 20% of the CAESAR data was reserved for the finetuning evaluation steps (FT eval). To create a standardized benchmark between the methods, the same training and test split in Tian et al. [ 15 ] was preserved to investigate the performance of the new deep autoencoder and nonlinear GPR on the same test set. The test set was a random sample of 20% of the available Shape Up! Adults scans. Table 1 3D scan count and unique individual representation for the pretrain, finetune, evaluation, and test sets. The pretraining step used only DFAUST data and included its own evaluation split consisting of 20% of the DFAUST scans. The finetune step was an iterative training step performed using only CAESAR, SUA, and SUK data starting with the network weights determined by the pretraining step. The evaluation set was used to determine the best performing model state to save without causing overfitting to the training data. The test set was kept the same as prior work to create controlled comparisons. Dataset Total 3D scans / Total unique individuals Pretraining Finetuning FT Eval Test DFAUST 36,956 / 10 0 0 0 CAESAR 0 2140 / 2014 462 / 462 0 Shape Up! Adults 0 1500 / 542 0 424 / 336 Shape Up! Kids 0 646 / 200 0 0 Total - 4286 / 2882 - - To enforce topological consistency between all 3DO meshes, all scans in Table 1 were standardized to a constant topology containing 6890 vertices (referred to as the SMPL [ 22 ] topology) except for the DFAUST data, which is already in this format natively. We used the automated template fitting method of Tian et al. [ 15 ] and the nonrigid deformation of Allen et al [ 23 ] for this process. Implementation details are provided in the Supplementary Materials. A visual example of this process is shown in Fig. 1 . The ensemble 3DO data selected for this study collectively covered many of the limitations inherent to each individual dataset. Clinically sampled, multi-identity datasets like SUA and SUK are custom-made for studies modeling human shape variation and body composition estimation across a diverse, cross-sectional sample of the population. SUA and SUK contain 3D meshes and body composition reference values for adults and kids respectively but are low in participation due to the additional overhead and difficulty of collecting clinical data. CAESAR augments this dataset by doubling the number of single-pose 3DO scans available while almost quadrupling the number of unique individuals represented. However, since there is no clinical data associated with CAESAR, it can only be used to train the 3D shape model and not the mapping to body composition. Only CAESAR scans were held out for model evaluation during the finetuning step of 3DAE training to preserve as many SUA and SUK scans with paired DXA measurements as possible for body composition regression training and testing. The DFAUST data was different from the other data subsets as it contained very few unique individuals (10, 5 male and 5 female) but the largest number of unique 3DO scans (~ 37,000). This dataset varied pose instead of individual identity to create shape diversity. Modeling shape variation due to posing is not an objective of this study and DFAUST was not used to jointly train the 3DAE shape model with the other datasets. However, DFAUST was very useful in initializing the 3DAE weights to a better-than-random starting state for training on the CAESAR-SUA-SUK ensemble dataset. 3D Deep Autoencoder with Graph Convolutional Network Raw 3DO scans contain potentially hundreds of thousands of unorganized vertices and are not suitable inputs for regression algorithms without first processing then into standardized feature vectors for all dataset members. In this study, we use a 3D graph convolutional neural network adapted from Zhou et al. [ 18 ] to perform nonlinear dimensionality reduction and feature extraction on the 3D body shape space. This deep network is a 3D autoencoder that possesses many attributes inherent to deep convolutional neural networks (CNNs). A local kernel operator paired with layered feature pooling and unpooling, equivalent to multilevel filters in 2D image CNNs, gives this method local feature sensitivity while enabling the representation of nonlinear relationships between the latent encoding and the decoded shape. For a 3D autoencoder, the inputs are the 3D mesh vertex coordinates represented as 6890x3 sized tensors. The loss is a geometric mean absolute error (MAE) loss minimizing the reconstruction error against the original input coordinates. Unlike image convolution on a regular square grid, the 3D mesh graph is irregular with varying connectivity. This requires some architectural modifications in the network to apply convolutional operations to human body scans. Our chosen implementation of a 3DAE assumes topological consistency of all mesh inputs. This simplifies the network design by allowing us to determine the spatial down sampling and upsampling operations once per mesh template in a preprocessing step. We defined both the convolutional kernel radius and the spatial kernel stride as two to be consistent with Zhou et al. [ 18 ] A visualization of the precomputed graph down sampling for each layer of the autoencoder is shown in Fig. 2 . Additional implementation details are written in the Supplementary Materials and in Zhou et al. Data paucity was one of the primary reasons for using linear algorithms rather than a deep network in prior work as 3DO data paired with ground truth DXA measurements are absent in the literature outside of the limited SUA and SUK collections. We initialized our model with the DFAUST dataset as augmentation data to mitigate the lack of data availability. We pretrained our 3DAE on DFAUST for 200 epochs, then trained on a composite of SUA, SUK, and CAESAR for an additional 400 epochs excluding DFAUST. We trained 3DAE models bottleneck channel depths of 7, 43, 90, and 601. With 7 nodes at the bottleneck layer, these channel depths defined total latent feature vector sizes of 49, 301, and 630, and 4284 as shown in Fig. 1 . All models were trained with a kernel basis size of M = 37. The learning rate was set to 1e-4. Training batch size was set to at 16. (See Zhou et al. for implementation details) All model variations had the same architecture and training data aside from the bottleneck layer. Learning a nonlinear transfer function with GPR For body composition analysis, we performed nonlinear Gaussian process regression between the features extracted from the SUA dataset and their paired DXA measurements. Prior work on estimating DXA body composition from 3D body shape used least squares linear regression, which imposed a restrictive assumption of linearity on the relationship between body shape variables and body composition. GPR was previously used in Wang et al. [ 25 ] for visceral adipose tissue estimation from body circumferences and is a nonlinear, probabilistic generalization of linear regression that relaxes the imposition of linearity on the function between body shape features and body composition. GPR is more generalized than linear regression but not as unrestricted as a multi-layer perceptron network (MLP). The limited number of data observations in our dataset made GPR a more appropriate regression method than MLPs while still relaxing tight assumptions on the function shape. We trained a GPR model to learn a nonlinear mapping between the encoded latent vectors of the training data and the DXA measurements for 10 body composition measures. We chose a squared dot product kernel for GPR under the assumption that relationships between body shape features and body composition were nonlinear but monotonic in both the first and second derivatives. Our assumption of monotonicity between body shape and body composition is consistent with the observation that variables such as visceral fat and percent fat are positively correlated with anthropometric measurements such as waist circumference and body volume [ 16 ]. We experimented with other kernels such as the radial basis function (RBF) and higher powers of dot product kernels and found they performed worse than a squared dot product kernel. We present results for the squared dot product kernel in this paper. We trained GPR models for male and female participants separately. GPR derivation details can be referenced in the Appendix and in greater detail in [ 26 ] and [ 27 ]. We implemented GPR using scikit-learn [ 28 ]. We concatenated [height, weight, age] of each participant to the feature vector input to GPR in accordance with the procedures established in previous work. To comprehensively search the feature representation space across multiple scales for the best predictive inputs, we performed GPR on all intermediate feature layers of the 3DAE shape model to predict body composition targets. For each of the four levels of deep features shown in Fig. 2 , we performed GPR to body composition targets. We reported the prediction accuracies for the bottleneck layer containing 4284 (7x612) features and for the feature layer producing the lowest RMSE in GPR prediction. Comprehensive ablation trials vs. linear methods via model permutations PCA is a common linear approach to dimensionality reduction for 3D human body shape due to the widespread adoption of methods such as SMPL [ 22 ] and was the shape modeling method used in prior work on 3DO body composition estimation. PCA is a deterministic linear operation with a globally optimal solution and produces feature vectors over a space of orthogonal components, making it well-behaved even for datasets containing just tens to hundreds of scans [ 15 , 23 ]. To test the comparative performance of new nonlinear models against linear baselines established in previous work, we trained a PCA model using the same 4286 training meshes of the 3DAE. DFAUST was not used for training PCA shape models. Although the test set membership of this study is the same as that of Tian et al (2022) [ 15 ], the training set in this work is greatly expanded. Recreating PCA models with consistent data membership allowed us to isolate the effects of deep 3DAE shape encoding and nonlinear GPR prediction relative to a baseline of PCA and ordinary least squares (OLS) on the same data. Due to the predefined downsampling of the 6890-vertex mesh topology shown in Fig. 2 , the bottleneck layer of the 3DAE must be a multiple of 7. We set the maximum bottleneck (latent code) size of our 3DAE to 4284 as it was the closest multiple of 7 to 4286, the maximum number of possible PCA components (corresponding to the number of meshes in the training set). While the 3DAE bottleneck layer dimension could be increased to an arbitrarily high number, resulting in lower reconstruction error, there would not be an equivalent linear PCA model to function as a comparative baseline. To further test our hypothesis regarding the better prediction accuracy of nonlinear GPR relative to OLS, we trained a GPR model using the same PCA basis from Tian et al. (2022) [ 15 ], labeled PCA-GPR M391/F457 which indicates 391 and 457 components for males and females respectively. Statistical Analysis We measured the geometric reconstruction accuracy of both linear and nonlinear shape models at different model sizes to assess the marginal contribution of nonlinear autoencoders to shape modeling accuracy relative to a linear PCA method used in prior work. We then iterated through different model permutations consisting of feature extraction with both linear and nonlinear shape models followed by both linear and nonlinear regression to DXA body composition measurements. Model permutations demonstrated the marginal contributions of GPR and 3DAE to the precision and accuracy of body composition prediction relative to the baselines established by linear algorithms used in prior work. We compared 3D geometric reconstruction error between the PCA baseline and 3DAE shape models at four model sizes as the per-vertex mean absolute error (MAE). We compared the accuracy of body composition estimation for four model permutations: the GPR baseline estimated using only demographic features [height, weight, age] with no shape information, the linear baseline consisting of ordinary least squares regression from PCA shape features (PCA-OLS), hybrid consisting of GPR using PCA shape features (PCA-GPR), and fully nonlinear GPR from deep shape features (3DAE-GPR). We quantified estimation error with root-mean-squared-error (RMSE) relative to reference DXA measurements and plotted the normalized RMSE of each predicted metric as a percentage of the PCA-OLS fully linear baseline. The coefficient of determination (R 2 ) for agreement to DXA was assessed for the 3DAE-GPR model. We predicted body composition on test-retest data pairs to compare the precision of our model permutations to prior work and to DXA scanners using the coefficient of variation (%CV) of visceral fat and the repeat RMSE of percent fat. We compared the precision of 3DAE-GPR at two model sizes against the PCA-GPR and PCA-OLS permutations, trained and tested on the same training data and test-retest pairs. For 3DAE-GPR, we trained the GPR component on the bottleneck layer (301 and 4284 features for the small and large model sizes respectively) to enforce consistency with the parameter count of the PCA permutations. We compared the accuracy of our best model performance to a comprehensive list of prior work using percent fat and visceral fat prediction as the benchmark and showed that we can achieve state of the art results on 3DO body composition prediction using deep convolutional feature extraction and nonlinear regression. We conducted ablation studies to evaluate the effects of skipping model training steps or withholding training data on geometric reconstruction accuracy and body composition prediction accuracy to confirm that all training procedures and all subsets of the data described in the method positively contributed to the accuracy of our results. We recorded the reconstruction accuracy of the 3DAE using a random initialization without the DFAUST pretrained initialization. We then tested the inverse data withholding condition by training a 3DAE on DFAUST only with no further finetuning with our multi-identity ensemble dataset on SUA test data. We withheld CAESAR, SUK and both CAESAR and SUK data from the training sets to test if dissimilarities between data subsets may have reduced reconstruction or regression performance. We tested the geometric reconstruction accuracy and the body composition prediction accuracy for each exclusion scenario. Ablation trials were tested on a d = 301 sized bottleneck network due to its much lower training time. RESULTS The Shape Up! Adults study population has been previously described [15] and summarized in Table S1 in the Supplementary Material. Only this subset of the total data was used for body composition regression training and testing. Geometric reconstruction error for both 3DAE and PCA shape models of four increasing sizes are shown in Table 3. The dimensionality d represents either the number of PCA coefficients used to parameterize shape (linear model) or the number of latent variables in the bottleneck layer of the 3DAE (nonlinear model) connecting the encoder module to the symmetric decoder. Reconstruction error was calculated as the geometric mean absolute error (MAE) between original and reconstructed vertex 3D positions. As expected, larger models were able to reconstruct the test data with lower error. Both linear and deep methods are comparable in terms of geometric reconstruction accuracy for the first three model sizes. PCA achieved lower geometric reconstruction error at the highest parameter count while 3DAE reconstruction error leveled off at just above 2mm MAE. Table 2 . Per-vertex reconstruction error measured as mean absolute error (MAE) between input and decoded meshes for held-out test meshes from Shape Up! Adults (n=424). The dimensionality (d) represents the number of latent layer variables in the autoencoder bottleneck or the number of principal components used for a linear shape space reconstruction. D =49 d =301 d =630 d =4284 3DAE MAE (mm) 5.16 2.57 2.18 2.02 PCA MAE (mm) 5.26 2.41 2.23 1.0 Figure 3 depicts the body composition prediction results from d =4284 sized models using different permutations of linear and nonlinear shape feature extraction and regression methods. Excluding the baseline column, each subsequent column represents a change of exactly one model parameter holding all others constant; i.e. OLS to GPR, PCA to 3DAE, 4824 total parameters to 400x64 parameters. All model permutations were trained and tested on the exact same mesh data. The baseline GPR model accuracy is the RMSE resulting from a regression using only known features [height, weight, age] without any conditioning on the 3DO shape features as was previously done in [15] and [29]. This RMSE was higher than any subsequent regression model using shape features as input. The reduction of prediction error when shape information is introduced demonstrated that 3DO shape is a useful signal for body composition prediction even when nonlinear regression algorithms are employed. PCA-OLS is equivalent to the linear methods of prior work. This is a linear regression model for body composition prediction taking PCA features as inputs. The PCA model had 4284 parameters to stay exactly consistent with the dimensionality the 3DAE. PCA-GPR represents a hybrid pipeline predicting body composition with nonlinear GPR from linear PCA features. Nonlinear regression from linear shape features achieved lower RMSE on every predicted metric relative to linear regression except arm lean on females (which was equal) and leg lean on females (which was higher by 0.02 kg, or 5%). These results indicated that GPR was a more accurate regression method than OLS when all other factors were held constant for most body composition targets. 3DAE-GPR represents the fully nonlinear pipeline where body composition is predicted from GPR using 3DAE deep features as inputs. The results for both the bottleneck layer (7x612) and the layer with the lowest RMSE are shown. The third layer (dimensions 400x64) was the most accurate feature layer for males and the first layer (dimensions 6890x16) was the best for females. In males, 3DAE feature extraction lowered RMSE relative to the previous model permutations, but in females no level of feature extraction outperformed GPR on the original mesh coordinates. This result suggests the features extracted from female meshes were less informative relative to male features or were highly correlated regardless of the method and model size. GPR always improves accuracy relative to OLS, but 3DAE feature extraction only improved accuracy for males. OLS regression to body composition with 3DAE features resulted in very low correlations with DXA due to the nonlinearity of the deep features and was not reported. We also tested concatenating all feature layers into a single multi-scale feature vector for GPR regression but did not achieve lower errors in doing so. We rescaled the charts in Figure 3 by normalizing each column by the RMSE of the fully linear PCA_OLS model and plotted the values in Figure 4 for visual clarity. In males, all subsequent model permutations had RMSEs less than that of PCA_OLS (indicated by a normalized RMSE of 100%). For females, only leg lean increased in RMSE when moving from PCA_OLS to PCA_GPR. However, as previously observed most RMSEs increased in females when incrementing PCA to 3DAE. When comparing nonlinear methods to linear methods, GPR is more accurate for body composition prediction in most metrics, but 3DAE is not always a better feature extraction method for body composition prediction from shape relative to linear PCA in females. Test-retest precision of percent fat and visceral fat estimation is shown in Table 3. For visceral fat mass precision, Coefficient of Variation (%CV) was calculated according to the definition in Glüer et al. [30] between two scans of the same individual in the test set taken on the same day. RMSE between trial 1 and trial 2 was shown for percent fat precision as the %CV of a percentage measurement is not used in convention. GPR was more precise on retests than OLS as shown by the PCA-OLS versus PCA-GPR trials, where the latter resulted in up to a 30% decrease in precision error. 3DAE also decreased precision error relative to PCA in both sexes, as illustrated by the 3DAE-GPR versus PCA-GPR trials. 3DAE coupled with nonlinear GPR had the highest precision. Compared to DXA, percent fat precision error for 3DAE-GPR was roughly twice as high. However, visceral fat precision was lower than DXA, indicating the 3DAE model paired with GPR prediction is much more reliable on retest accuracy than prior work. The larger d =4284 3DAE model was comparable in precision to the smaller model. 3DAE was more precise than PCA, and GPR was more precise than OLS. Table 3. Test-retest precision between repeat 3DO scan pairs of each participant in the SUA test set on the same scan device. 3DAE models were benchmarked with GPR trained from their bottleneck layers to standardize the comparison between model sizes. 3DAE was more precise than PCA, and GPR was more precise than OLS. Visceral fat % CV 3DAE-GPR 301 3DAE-GPR 4284 PCA-GPR 4284 PCA-OLS 4284 Tian et al. 2022 [15] DXA (Criterion) Males (n = 143) 4.4% 5.0% 5.2% 7.2% 8.8% 4.8% Females (n=199) 5.5% 5.5% 6.2% 8.6% 12.3% 6.3% Percent fat precision RMSE 3DAE-GPR 301 3DAE-GPR 4284 PCA-GPR 4284 PCA-OLS 4284 Tian et al. 2022 [15] DXA (Criterion) Males (n = 143) 1.0% 0.9% 1.1% 1.6% 1.9% 0.5% Females (n=199) 1.2% 1.2% 1.4% 2.0% 2.9% 0.5% Comparisons of our 3DAE-GPR models against prior work are shown in Table 4 using the d =4284 latent size model and Gaussian process regression with the best performing feature layer (third for males, first for females). Percent fat (PFAT) and visceral fat mass (VFAT) were selected as the comparative target variable as they achieved the lowest accuracy in previous works based on linear models. GPR using the exact same PCA features of Tian et al. (PCA-GPR M391/F457) lowered the RMSE from OLS for percent fat in both sexes and in female visceral fat but increased it slightly in male visceral fat by 8%. The models presented in [15] were built on a dataset containing an order of magnitude fewer members than what we presented in this work. These results suggest linear pipelines may be more competitive with nonlinear methods when training data quantity is more limited. We included RMSEs for our best performing, maximum size PCA model (PCA-GPR 4284) to demonstrate that increased parameter count does not cause overfitting and degrade accuracy relative to a sparser model. Our fully nonlinear model, 3DAE-GPR, produced the lowest error on visceral fat and percent fat estimation compared to all prior work on 3DO body composition estimation (bolded). We note that compared to Tian et al (2022), our best 3DAE-GPR model achieved lower RMSE on all of the 10 body composition metrics measured in Figure 4, shown in Supplementary Table 2. However, predicted metrics other than percent fat and visceral fat already showed high correlation with DXA in prior work using linear methods. Thus, we focused the comparison to prior work in Table 4 on the metabolically significant and previously underperforming predictions of percent fat and visceral fat. The test set used in this work was held the same as [15]. However, the training dataset was greatly expanded in size and scope relative to prior works. Table 4. Root-mean-squared errors (RMSE) for predicted percent fat (PFAT) and visceral fat (VFAT) of all current 3D-optical body composition prediction literature on Shape Up! Adults compared to the 3DAE-GPR prediction of the d=301 and d=4284 models using the most accurate feature layer identified in Figure 3. Best performing values are bolded. Paper N test meshes PFAT RMSE (%) VFAT RMSE (kg) Ng et al (2019) Anthro only M: 177 F: 230 M: 4.03 F: 3.99 M: 0.15 F: 0.14 Ng et al (2019) [15] M: 177 F: 230 M: 3.55 F: 3.88 M: 0.14 F: 0.13 Tian et al (2020) [28] M: 31 F: 39 M: 3.90 F: 3.29 M: 0.15 F: 0.17 Wong et al (2021) [31] M: 159 F: 202 M: 2.73 F: 3.46 M: 0.13 F: 0.13 Tian et al (2022) [15] M: 182 F: 248 M: 3.24 F: 4.22 M: 0.12 F: 0.14 PCA-GPR M391/F457 [15] M: 182 F: 248 M: 2.79 F: 3.09 M: 0.13 F: 0.12 PCA-GPR 4284 M: 181 F: 239 M: 2.68 F: 2.85 M: 0.11 F: 0.11 3DAE-GPR 4284 M: 181 F: 239 M: 2.50 F: 2.81 M: 0.11 F: 0.11 Ablation results Figure 5 shows a mesh reconstruction using a d =4284 model trained with 400 epochs on the finetuning ensemble training data from 1) a random initialization state and 2) from a pretrained initialization trained with 200 epochs using DFAUST data only. The model trained from a random initialization achieved 22.7mm MAE on test data reconstruction, more than 10x the error of the error show in Table 3. The pretraining steps using 40,000 DFAUST meshes was essential for creating an accurate shape model using a nonlinear 3DAE. Figure 6 shows visualizations of geometric reconstruction error as heat maps for a male and female subject before and after fine-tuning the d =4284 3DAE model with high resolution Shape Up! and CAESAR data. Without fine-tuning, the 3DAE model was equivalent to the work presented in Zhou et al. [18] trained exclusively on DFAUST data and achieved a 3D reconstruction error of 8.7 mm, as opposed to the 2.0 mm shown in Table 2 for the finetuned model. This model generalized very poorly to unseen scans of many unique individuals such as in Shape Up!, as DFAUST only contained 10 unique individuals captured in thousands of different poses. A 3DAE model for clinical machine learning applications needs to generalize well to any individual scanned from the general population in a neutral pose. Our fine-tuned model represented the non-rigid, identity-dependent deformations of unique individuals much more accurately. Withholding non-SUA data from 3DAE training did not improve 3D reconstruction error on SUA test meshes. In all three ablation trials, reconstruction accuracy on the same SUA test set (2.58mm, 2.71mm, 2.76mm for removing only SUK, only CAESAR, and both, respectively) was worse than the model trained on all data combined (reconstruction accuracy of 2.57) in Table 3. This ablation result validates the inclusion of a diverse dataset across multiple collection protocols. GPR models trained on the 3DAE model excluding SUK data showed 1% difference in RMSE in both directions. Not all target variables were uniformly lower in error when including versus excluding SUK. The variation could be attributed to noise and does not justify excluding SUK. Excluding CAESAR data from the training scans had similarly negligible effects on male prediction accuracy but increased female RMSEs without exception by up to 4%. Excluding both SUK and CAESAR produced results similar to the case where only CAESAR was excluded, but with even higher errors in females. Overall, we found that including all available 3D mesh data when training shape and regression models produced the lowest errors for 3D reconstruction and body composition prediction. DISCUSSION This study showed three primary findings. First, GPR generally improves body composition prediction accuracy and precision relative to linear regression regardless of whether the feature domain is linear (PCA) or nonlinear (3DAE). By comprehensively exploring model permutations and changing exactly one variable at each iteration, we were able to demonstrate that nonlinear regression with GPR performed better than OLS for every body composition target except female leg lean. Second, 3DAE feature extraction improved body composition prediction for males. However, deep features were not more informative than the unencoded mesh geometry in females and did not outperform PCA in geometric reconstruction at high parameter counts. Third, both 3DAE and GPR achieved higher precision than their linear counterparts in both males and females when measured on percent fat and visceral fat mass. A combination of an expanded and diversified training set with deep feature extraction and nonlinear GPR prediction allowed us to build an end-to-end model that outperformed all prior works on body composition prediction accuracy. The improved accuracy and precision demonstrated here builds on our prior work [ 14 , 15 ] and further establishes 3DO as a reliable and accessible clinical tool for the assessment of body composition. Such a tool has promising clinical implications for the management of chronic disease, where body composition has known associations with morbidity and mortality. To our knowledge this was the first application of deep nonlinear autoencoder networks to 3DO body composition estimation. We explicitly chose a spatial graph convolutional network for our autoencoder approach. Other autoencoder approaches, such as implicit surface encoding [ 32 ], non-convolutional variational architectures [ 33 ], and spectral graph convolution [ 34 ] have been used to study body shape, but without associations to clinical outcomes. Future work may find that these other approaches have advantages to the work presented here. We hypothesized that lower geometric reconstruction error in a shape autoencoder model was correlated with higher body composition prediction accuracy during regression from the extracted features. However, although a linear PCA model of size d = 4284 produced the lowest shape reconstruction error, it did not outperform 3DAE on body composition prediction accuracy or precision. Geometric reconstruction accuracy may be affected by shape deformations irrelevant to body composition variation such as pose or face detail. PCA may be outperforming 3DAE in shape reconstruction at high parameter counts due to 3DAE potentially learning redundant and correlated features when the model size is large. Unlike 3DAE, PCA is guaranteed to learn uncorrelated, orthogonal features due to its mathematical construction. Future work should investigate the relationship between geometric reconstruction accuracy and body composition prediction accuracy by controlling for uncorrelated geometric variations and using different autoencoder architectures. PCA’s high performance on shape reconstruction validate the methods of past work built on PCA models with linear regression [ 14 , 15 , 20 ] despite the restrictive linear assumptions of the algorithm. Prior work restricted the shape model and body composition prediction features to a sparse subset of the total number of PCA components to avoid overfitting during regression. Our extensive testing with different model sizes and permutations does not support the assumption that large parameter counts overfit on test data. PCA-GPR model using 4284 components achieved lower RMSEs on percent fat and visceral fat on both males and females relative to models trained with 391 or 457 features. Restricting shape and regression models to the first n components that describe 95% or 99% of the shape variance may not be justified in future work as it potentially handicaps prediction accuracies unnecessarily. Our chosen autoencoder architecture does not explicitly disentangle pose deformation from identity-dependent deformations, such as in Wong et al. [ 31 ] or Jiang et al [ 33 ]. Factoring out slight pose variation across our dataset may improve the reconstruction accuracy and regression accuracy of our method. Increased sensitivity to small changes in body shape may allow our method to improve monitoring body composition change over time in the same individual [ 35 ]. As our dataset was transformed to adopt the mesh topology of SMPL [ 22 ] while preserving the geometric surface detail of the original 3DO scans, it may be straightforward in future work to “unpose” every mesh in our 4286-member ensemble dataset to a neutral T-pose like that of Wong et al. using the skinning weights and joint position definitions of the SMPL template. The accuracy of this procedure will depend on how anatomically consistent the template correspondences are between our dataset and the meshes in Loper et al. Future work can explore the impact of different nonlinear regression methods on body composition prediction accuracy. GPR regularized the shape of the regression function to its kernel. A less restricted but more flexible regression method such as a multi-layer perceptron (MLP) may achieve better prediction error with proper regularization; however, deeper models run the risk of overfitting to the training data especially if the latent parameter count is large. We observed that GPR using a radial basis function kernel (RBF) already exhibits symptoms of overfitting, showing zero error on the training data but greatly increased error on the test data. Like prior works using linear models, this work separately trains a shape feature extractor followed by a body composition regression from features. Future work may integrate the current two-step pipeline into a single end-to-end deep network. This can be achieved by connecting the 3DAE encoder layers directly to a neural network regression model that targets body composition as its output instead of self-reconstruction. This end-to-end model can be initialized to state reported in this work by importing our trained 3DAE weights into the encoder of the network and the weights of a separately trained regression model into the neural network regression layers. GPR may be recreated as a neural network in PyTorch with GPyTorch [ 36 ]. This implementation could allow the combined network to achieve greater accuracy by starting from the state presented in this work and further optimizing all features to target body composition prediction with no intermediate objectives. Our 3D deep shape model was trained on the largest collection of high quality, multi-identity 3DO scan data currently available, spanning CAESAR, SUA, SUK, and DFAUST over more than a 20-year period. However, with 2,900 unique individuals, this dataset is still orders of magnitude smaller than those used in analogous networks for deep 2D image learning. This restriction on dataset size and population variation sampling density can create gaps in our model that exhibit low reconstruction and prediction accuracy where training data was not available or under sampled, especially at the extremes of body shapes as shown in Fig. 7 . Ongoing data collection of high resolution 3DO scans will improve the reconstruction accuracy of our method. Improvement of the nonlinear regression model will require additional paired DXA scans. The CAESAR data used in this work was built from the original 3D scan data collected by Robinette et al. [ 19 ] and not on a derived reconstruction such as the MPI CAESAR dataset produced by Pishchulin et al [ 37 ]. The MPI dataset is a template standardization of CAESAR using a limited number of PCA features for reconstruction. The resulting shapes are linearly projected compressions of the original 3D scan data and do not preserve original high-resolution detail well, as shown in Fig. 8 . Including these shapes into our training database would bias our deep nonlinear model towards an approximation of a linear solution. Our remeshing of the CAESAR scans into the SMPL format includes a nonrigid surface-to-surface deformation that produced templated meshes consistent with the original scan geometry and did not constrain the training data to projections onto a linear subspace. Future work studying human shape variation as a function of varying identity rather than pose may build upon our results by utilizing the higher fidelity standardized templates of our dataset rather than training on potentially degraded 3D shape geometry. CONCLUSION A comprehensive comparison of nonlinear methods for shape feature extraction and body composition regression against previous linear algorithms showed that nonlinear GPR improved body composition prediction accuracy and precision relative to linear regression for 10 metrics of body composition in males and for 9 in females. Nonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Relative to linear PCA feature computation, feature extraction with a deep 3D autoencoder provided marginal improvements to prediction accuracy for males but did not supersede the performance of GPR on raw mesh vertex positions for females. Deep shape features produced 6–8% reduction in prediction error over linear PCA features for males. However, deep 3DAE features reduced precision error between 4–14% relative to linear features with all other variables held equal. Our best performing nonlinear pipeline using 3DAE-GPR outperformed prior works on body composition prediction accuracy for all metrics. Precision error of our method is within 1-2x that of DXA, the gold standard for compartmental body composition measurement. Agreement with DXA as measured by R 2 values were greater than or equal to 0.86 for all predicted metrics. These findings improve the clinical utility of 3DO as accurate and accessible tool for the assessment of body composition. Future work in this space should explore the effects of disentangling pose variations from 3DO scans along with different deep network architectures such as variational latent encodings, end-to-end deep models, and predicting longitudinal change over time in individual subjects. Abbreviations 3 Dimensional Optical (3DO) 60 001 vertex mesh template (60k) body mass index (BMI) dual X ray absorptiometry (DXA) principal component analysis (PCA) Declarations Author Contributions: Isaac Y. Tian: Methodology, Software, Formal Analysis, Investigation, Writing – Original Jason Liu: Validation, Data Curation, Writing – R&E Michael C. Wong: Methodology, Investigation, Writing – R&E Nisa N. Kelly: Investigation, Writing – R&E, Data Curation, Project administration Yong E. Liu: Investigation, Writing – R&E, Data Curation Andrea K. Garber: Investigation, Writing – R&E, Funding Acquisition Steven B. Heymsfield: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing – R&E Brian Curless: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing – R&E John A. Shepherd: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing – R&E Funding Disclosure: AKG: NICHD #R01HD082166, National Institutes of Health NORC Center Grants (P30DK072476, Pennington/Louisiana and P30DK040561, Harvard); JAS: National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (R01DK109008 and R01DK111698). Conflict of Interest Statement: JAS is a scientific advisor for Styku LLC and owns stock options. BC is employed by Google LLC and owns stock options. SBH is on the medical advisory boards of Tanita Corp. and Medifast Corp. and is a former employee of Merck & Co. SBH is an Amazon Scholar and owns stock options. Styku LLC and Fit3D Inc. contributed scanning equipment in support of NIH studies. Data & Code Availability: The datasets generated and/or analyzed during the current study are not publicly available due to the inclusion of protected health information (PHI) but are available from the corresponding author on reasonable request. The underlying code for this study [and training/validation datasets] is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author. References Wilson JP, Kanaya AM, Fan B, Shepherd JA. Ratio of trunk to leg volume as a new body shape metric for diabetes and mortality. PLoS ONE 2013;8. doi:10.1371/journal.pone.0068716. Kootaka Y, Kamiya K, Hamazaki N, et al. The GLIM criteria for defining malnutrition can predict physical function and prognosis in patients with cardiovascular disease. Clin Nutr. 2021;40(1):146-152. Au PC, Li HL, Lee GK, et al. Sarcopenia and mortality in cancer: A meta-analysis. Osteoporos Sarcopenia. 2021;7(Suppl 1):S28-S33. Goodpaster BH, Krishnaswami S, Harris TB, Katsiaras A, Kritchevsky SB, Simonsick EM, et al. Obesity, regional body fat distribution, and the metabolic syndrome in older men and women. Archives of Internal Medicine 2005;165:777. doi:10.1001/archinte.165.7.777. Zhang C, Rexrode KM, van Dam RM, Li TY, Hu FB. Abdominal obesity and the risk of all-cause, cardiovascular, and cancer mortality. Circulation 2008;117:1658–67. doi:10.1161/circulationaha.107.739714. Hui WS, Liu Z, Ho SC. Metabolic syndrome and all-cause mortality: A meta-analysis of prospective cohort studies. European Journal of Epidemiology 2010;25:375–84. doi:10.1007/s10654-010-9459-z. Dulloo AG, Jacquet J, Solinas G, Montani J-P, Schutz Y. Body composition phenotypes in pathways to obesity and the metabolic syndrome. International Journal of Obesity 2010;34. doi:10.1038/ijo.2010.234. Wilson JP, Kanaya AM, Fan B, Shepherd JA. Ratio of trunk to leg volume as a new body shape metric for diabetes and mortality. PLoS ONE 2013;8. doi:10.1371/journal.pone.0068716. Galindo Martin CA, Aportela Vazquez VA, Becerril Hernandez F, et al. The GLIM criteria for adult malnutrition and its relation with adverse outcomes, a prospective observational study. Clin Nutr ESPEN. 2020;38:67-73. Sanchez-Rodriguez D, Locquet M, Bruyere O, et al. Prediction of 5-year mortality risk by malnutrition according to the GLIM format using seven pragmatic approaches to define the criterion of loss of muscle mass. Clin Nutr. 2021;40(4):2188-2199. Marco E, Sanchez-Rodriguez D, Davalos-Yerovi VN, et al. Malnutrition according to ESPEN consensus predicts hospitalizations and long-term mortality in rehabilitation patients with stable chronic obstructive pulmonary disease. Clin Nutr. 2019;38(5):2180-2186. Rondel A, Langius JAE, de van der Schueren MAE, Kruizenga HM. The new ESPEN diagnostic criteria for malnutrition predict overall survival in hospitalised patients. Clin Nutr. 2018;37(1):163-168 Fosbøl MØ, Zerahn B. Contemporary methods of body composition measurement. Clinical Physiology and Functional Imaging 2014;35:81–97. doi:10.1111/cpf.12152. Ng BK, Sommer MJ, Wong MC, Pagano I, Nie Y, Fan B, et al. Detailed 3-dimensional body shape features predict body composition, blood metabolites, and functional strength: The shape up! studies. The American Journal of Clinical Nutrition 2019;110:1316–26. doi:10.1093/ajcn/nqz218. Tian IY, Wong MC, Kennedy S, Kelly NN, Liu YE, Garber AK, et al. A device‐agnostic shape model for automated body composition estimates from 3D optical scans. Medical Physics 2022;49:6395–409. doi:10.1002/mp.15843. Tinsley GM, Moore ML, Benavides ML, Dellinger JR, Adamson BT. 3-dimensional optical scanning for body composition assessment: A 4-component model comparison of four commercially available scanners. Clinical Nutrition 2020;39:3160–7. doi:10.1016/j.clnu.2020.02.008. Wenninger S, Achenbach J, Bartl A, Latoschik ME, Botsch M. Realistic virtual humans from smartphone videos. 26th ACM Symposium on Virtual Reality Software and Technology 2020. doi:10.1145/3385956.3418940. Y. Zhou, C. Wu, Z. Li, C. Cao, Y. Ye, J. Saragih, H. Li, and Y. Sheikh, “Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels,” in Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS'20), 2020, pp. 9251–9262. Robinette KM, Blackwell S, Daanen H, Boehmer M, Fleming S. Civilian American and European Surface Anthropometry Resource (CAESAR), final report. volume 1. summary 2002. doi:10.21236/ada406704. Wong MC, Ng BK, Kennedy SF, Hwaung P, Liu EY, Kelly NN, et al. Children and adolescents’ anthropometrics body composition from 3‐D optical surface scans. Obesity 2019;27:1738–49. doi:10.1002/oby.22637. Bogo F, Romero J, Pons-Moll G, Black MJ. Dynamic FAUST: Registering human bodies in Motion. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017. doi:10.1109/cvpr.2017.591. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL,” ACM Transactions on Graphics, vol. 34, no. 6, pp. 1–16, 2015. Allen B, Curless B, Popović Z. The space of human body shapes. ACM Transactions on Graphics 2003;22:587–94. doi:10.1145/882262.882311. Hahner S, Garcke J. Mesh convolutional Autoencoder for semi-regular meshes of different sizes. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2022. doi:10.1109/wacv51458.2022.00240. Wang Q, Lu Y, Zhang X, Hahn JK. A novel hybrid model for visceral adipose tissue prediction using shape descriptors. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2019. doi:10.1109/embc.2019.8857092. Schulz E, Speekenbrink M, Krause A. A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions. Journal of Mathematical Psychology. 2018;85:1-16. doi:https://doi.org/10.1016/j.jmp.2018.03.001 Guo S. Implement A Gaussian Process From Scratch. Medium. Published January 26, 2021. Accessed May 10, 2023. https://towardsdatascience.com/implement-a-gaussian-process-from-scratch-2a074a470bce Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research. 2011;12(85):2825-2830. Tian IY, Ng BK, Wong MC, Kennedy S, Hwaung P, Kelly N, et al. Predicting 3D body shape and body composition from conventional 2D photography. Medical Physics 2020;47:6232–45. doi:10.1002/mp.14492. Glüer C-C, Blake G, Lu Y, Blunt1 BA, Jergas1 M, Genant1 HK. Accurate assessment of precision errors: How to measure the reproducibility of bone densitometry techniques. Osteoporosis International 1995;5:262–70. doi:10.1007/bf01774016. Wong MC, Ng BK, Tian I, Sobhiyeh S, Pagano I, Dechenaud M, et al. A pose‐independent method for accurate and precise body composition from 3D optical scans. Obesity 2021;29:1835–47. doi:10.1002/oby.23256. T. Alldieck, H. Xu, and C. Sminchisescu, “ImGHUM: Implicit generative models of 3D human shape and articulated pose,” 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021. B. Jiang, J. Zhang, J. Cai, and J. Zheng, “Disentangled human body embedding based on deep hierarchical neural network,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 8, pp. 2560–2575, 2020. C. Lemeunier, F. Denis, G. Lavoué, and F. Dupont, “Representation learning of 3D meshes using an Autoencoder in the spectral domain,” Computers & Graphics, vol. 107, pp. 131–143, 2022. Wong MC, Bennett JP, Leong LT, Tian IY, Liu YE, Kelly NN, et al. Monitoring body composition change for intervention studies with advancing 3D optical imaging technology in comparison to dual-energy X-ray absorptiometry. The American Journal of Clinical Nutrition,Volume 117, Issue 4, 2023, Pages 802-813, ISSN 0002-9165, https://doi.org/10.1016/j.ajcnut.2023.02.006. Gardner JR, Pleiss G, Weinberger KQ, Bindel D, Wilson A. GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. Neural Information Processing Systems. 2018;31:7576-7586. Pishchulin L, Wuhrer S, Helten T, Theobalt C, Schiele B. Building statistical shape spaces for 3D human modeling. Pattern Recognition 2017;67:276–86. doi:10.1016/j.patcog.2017.02.018. Additional Declarations There is a conflict of interest JAS is a scientific advisor for Styku LLC and owns stock options. BC is employed by Google LLC and owns stock options. SBH is on the medical advisory boards of Tanita Corp. and Medifast Corp. and is a former employee of Merck & Co. SBH is an Amazon Scholar and owns stock options. Styku LLC and Fit3D Inc. contributed scanning equipment in support of NIH studies. Cite Share Download PDF Status: Published Journal Publication published 02 Feb, 2025 Read the published version in npj Digital Medicine → Version 1 posted Editorial decision: revise 07 Nov, 2024 Review # 2 received at journal 06 Nov, 2024 Reviewer # 3 agreed at journal 31 Oct, 2024 Reviewer # 2 agreed at journal 28 Oct, 2024 Review # 1 received at journal 23 Aug, 2024 Reviewer # 1 agreed at journal 14 Aug, 2024 Reviewers invited by journal 10 Feb, 2024 Editor assigned by journal 07 Feb, 2024 Submission checks completed at journal 07 Feb, 2024 First submitted to journal 06 Feb, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3935042","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":272120992,"identity":"743662ee-ce10-4791-9820-d04c7702aeea","order_by":0,"name":"Isaac Tian","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4ElEQVRIiWNgGAWjYBACNiA+AGIYMCSDaAkZIrQww7SkJYC08BBhDzOEMmDIMQDRhLXwSeQfPPBzB0OeOXvO51c3aix4GNgPH92A12ESyQwHe88wFFv2vN1mnXMM6DCetLQbhLQc4G1jSNxwI3ebcQ4bUIsEjxlBLQf/grXkPDPO+UeklsMQW3KYH+e2EaOF57HBYdk2iWKDM8/MmHP7JHjYCPlFvj3x8ce3bTZ5BseTH3/O+VYnx89++BheLQwCCSBSIgHsSLC9eJWDAP8BMAXSwvyBoOpRMApGwSgYkQAARDBHUCJsnRUAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-4975-6290","institution":"University of Washington","correspondingAuthor":true,"prefix":"","firstName":"Isaac","middleName":"","lastName":"Tian","suffix":""},{"id":272120993,"identity":"bbce241b-b570-411c-bd10-ad5d5558186b","order_by":1,"name":"Jason Liu","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"prefix":"","firstName":"Jason","middleName":"","lastName":"Liu","suffix":""},{"id":272120994,"identity":"cd75f921-a314-4ef8-8153-994fc474da02","order_by":2,"name":"Michael Wong","email":"","orcid":"https://orcid.org/0000-0003-2392-9253","institution":"University of Hawaii Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Wong","suffix":""},{"id":272120995,"identity":"2822ed9b-7d83-4d23-9f2d-e7e6216f5db4","order_by":3,"name":"Nisa Kelly","email":"","orcid":"","institution":"University of Hawaii - Manoa","correspondingAuthor":false,"prefix":"","firstName":"Nisa","middleName":"","lastName":"Kelly","suffix":""},{"id":272120996,"identity":"a86440e7-6307-4d4f-99c5-a56b7acb1ac8","order_by":4,"name":"Yong Liu","email":"","orcid":"","institution":"University of Hawaii - Manoa","correspondingAuthor":false,"prefix":"","firstName":"Yong","middleName":"","lastName":"Liu","suffix":""},{"id":272120997,"identity":"62763669-7216-4212-903d-65649d19b164","order_by":5,"name":"Andrea Garber","email":"","orcid":"","institution":"University of California - San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Andrea","middleName":"","lastName":"Garber","suffix":""},{"id":272120998,"identity":"cf803caa-422a-4c12-bf04-0de552491700","order_by":6,"name":"Steven Heymsfield","email":"","orcid":"https://orcid.org/0000-0003-1127-9425","institution":"Pennington Biomedical Research Center","correspondingAuthor":false,"prefix":"","firstName":"Steven","middleName":"","lastName":"Heymsfield","suffix":""},{"id":272120999,"identity":"dd75f9c4-94d1-4c94-a53a-3179b8c3372f","order_by":7,"name":"Brian Curless","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"prefix":"","firstName":"Brian","middleName":"","lastName":"Curless","suffix":""},{"id":272121000,"identity":"c26bc4ad-03bd-4752-994d-f44f5558c3fc","order_by":8,"name":"John Shepherd","email":"","orcid":"","institution":"University of Hawaii - Manoa","correspondingAuthor":false,"prefix":"","firstName":"John","middleName":"","lastName":"Shepherd","suffix":""}],"badges":[],"createdAt":"2024-02-06 22:21:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3935042/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3935042/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41746-025-01469-6","type":"published","date":"2025-02-02T05:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":51083706,"identity":"94eae3b7-5789-450c-b869-86eb74eed1a6","added_by":"auto","created_at":"2024-02-13 19:42:19","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":655474,"visible":true,"origin":"","legend":"\u003cp\u003eLeft: a raw CAESAR scan with unordered vertices and incomplete surfaces. Center: automatic 60k template registration using the methods of Tian et al. Right: SMPL (6890) topology transformation of the 60k fit, created by multiplying the 60k mesh with a sparse nearest-neighbor matrix \u003cstrong\u003eC \u003c/strong\u003eand deforming the result to align with the 60k mesh.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/5d7f6bce19c9fbe86da52554.png"},{"id":51083705,"identity":"0c349dbd-0c71-4758-9d98-8fe8deaf8170","added_by":"auto","created_at":"2024-02-13 19:42:19","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":745433,"visible":true,"origin":"","legend":"\u003cp\u003eSuccessive convolutional downsampling of a human mesh with a radius and stride of 2. The images represent in order the input mesh (6890x3) and the intermediate graph topologies containing (# vertices x # channels): (6890x16), (1925x32), (400x64), (54x128), (7 x f). The final layer is the bottleneck layer, whose channel count f is variable in our experiments.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/12b3dd7ed13449fbb41c6255.png"},{"id":51083708,"identity":"645d7c8d-5f1d-45e6-b033-b0df4f6554e8","added_by":"auto","created_at":"2024-02-13 19:42:19","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":147743,"visible":true,"origin":"","legend":"\u003cp\u003eRMSE prediction errors for different model permutations. Every column represents exactly one modification to the pipeline parameters from the previous column (i.e. OLS to GPR) excluding the baseline column. RMSE is shown in kg except for PFAT, which is expressed as a percentage. Baseline is GPR prediction from known priors [height, weight, age] only. PCA_OLS is linear regression from linear PCA features. This result was identical to OLS from the 3DO vertex coordinates. PCA_GPR is nonlinear GPR prediction from linear PCA features. Mesh_GPR is GPR prediction from 3DO vertex coordinates with no feature extraction. 3DAE_GPR_X is the most accurate GPR model trained from any feature layer of the 3DAE network with the feature dimension indicated as X.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/743bc86afe53c201d0abd8d5.png"},{"id":51085142,"identity":"d6133899-422d-469f-9892-152742c6568f","added_by":"auto","created_at":"2024-02-13 19:50:20","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":184228,"visible":true,"origin":"","legend":"\u003cp\u003eNormalized RMSE values from Figure 3 as a percentage of the linear baseline (PCA_OLS), shown in parentheses below every body composition metric.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/7b6a240a1d4fe0805df98cd4.png"},{"id":51083707,"identity":"b59ebd90-723b-4d4f-99b7-805ab526ba2c","added_by":"auto","created_at":"2024-02-13 19:42:19","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":307074,"visible":true,"origin":"","legend":"\u003cp\u003ePretraining on 40,000 DFAUST meshes improves reconstruction fidelity in single pose, multi-identity data. \u0026nbsp;Left: Evaluation set mesh reconstruction after training 400 epochs from random initialization. Test set reconstruction error was 22.7mm. Right: Same mesh reconstruction using a model pretrained first on 200 epochs with DFAUST only followed by 400 epochs of finetuning. Test set reconstruction error was 2.0mm.\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/93c2f552407604120c2a4f52.png"},{"id":51083710,"identity":"445e78a3-2cd4-4ca6-aaed-61bb73d6b3ce","added_by":"auto","created_at":"2024-02-13 19:42:20","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":388787,"visible":true,"origin":"","legend":"\u003cp\u003eFinetune training on 4286-member multi-identity ensemble dataset reduces reconstruction error on test set meshes relative to 3DAE models trained on DFAUST only. Left: input 3DO test mesh. Center: reconstruction error heatmap using model state following 200 epochs of pretraining on DFAUST only. Test set MAE was 8.7mm. Right: heatmap on same test mesh after an additional 400 epochs of finetuning on multi-identity, single pose ensemble dataset. Error heatmap is in the range of [0,20] mm.\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/cb44f667f78c655c9e124d8f.png"},{"id":51083712,"identity":"60f8bdcf-03e5-4425-8cf7-ac8d6c1e8032","added_by":"auto","created_at":"2024-02-13 19:42:20","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":313916,"visible":true,"origin":"","legend":"\u003cp\u003eLeft: an input test mesh with high BMI; right, the heat map of the reconstruction showing where the greatest errors occurred. The numerous folds in the abdomen of this individual (47.3% body fat) were unique to only a few scans in the dataset and were possibly insufficiently modeled in the latent encoding.\u003c/p\u003e","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/b40dd906a237d3c52198e7e5.png"},{"id":51083713,"identity":"1888bfcb-a042-4881-9411-0dae2c3150da","added_by":"auto","created_at":"2024-02-13 19:42:20","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":353551,"visible":true,"origin":"","legend":"\u003cp\u003eMPI CAESAR mesh (Left) compared to our remeshing of the same individual (Right). The MPI shape was projected onto a linear PCA basis and lost considerable identifying detail.\u003c/p\u003e","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/dec8204a4546484d680617c3.png"},{"id":75303766,"identity":"df3adc35-0e5d-42e3-8b24-3948d3bcc26a","added_by":"auto","created_at":"2025-02-03 08:06:36","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4641427,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3935042/v1/dae05c1c-5bc2-4732-9039-87b087e91ff1.pdf"}],"financialInterests":"There is a conflict of interest\nJAS is a scientific advisor for Styku LLC and owns stock options. BC is employed by Google LLC and owns stock options. SBH is on the medical advisory boards of Tanita Corp. and Medifast Corp. and is a former employee of Merck \u0026 Co. SBH is an Amazon Scholar and owns stock options. Styku LLC and Fit3D Inc. contributed scanning equipment in support of NIH studies.","formattedTitle":"3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology","fulltext":[{"header":"INTRODUCTION","content":"\u003cp\u003eTotal and regional body composition are correlated with many of the leading causes of death in the US and around the world. [\u003cspan additionalcitationids=\"CR2 CR3\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] High visceral fat deposits doubled the risk for metabolic syndrome in males with otherwise normal BMIs [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] and increased mortality rates from cancer by up to 63% [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Metabolic syndrome, a condition indicated by an array of biological and physical vital measurements such as blood glucose and waist circumference, is strongly associated with many chronic conditions such as cancer, heart failure, and diabetes [\u003cspan additionalcitationids=\"CR7\" citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Management of these conditions is also heavily impacted by body composition, specifically low lean mass, which predicts poor treatment outcomes [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] and up to 17-fold increased mortality [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. However, body composition assessment historically required more advanced instruments such as dual X-ray absorptiometry (DXA) or air displacement plethysmography (ADP) [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. Unlike ADP, DXA can measure regional compartmental fat and lean deposits but exposes participants to potentially harmful ionizing radiation and is not recommended for frequently repeated measurements, particularly in at-risk groups such as young children and pregnant women. Radiation exposure must be limited to exigent circumstances even in healthy adults. An ideal alternative assessment system should achieve accuracy and precision on total and regional body composition measurement comparable to DXA without utilizing ionizing radiation. The system should be relatively inexpensive and easy to use, requiring no special training or certifications to operate and returning results in a minute or less.\u003c/p\u003e \u003cp\u003eRecent work showed that 3D optical (3DO) imaging can serve as an accurate and precise, low-cost, noninvasive surrogate to DXA imaging [\u003cspan additionalcitationids=\"CR15\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. 3DO measures the surface geometry of the human body using light in the optical spectrum as opposed to the penetrating radiation of DXA. It does not require the injection of an isotope contrast like MRI and can scan an entire adult body in under one minute. Scanning systems cost on the order of \u003cspan\u003e$\u003c/span\u003e10,000 and are programmed by the manufacturer to operate automatically without the need for certified technicians. The external 3D shape of a human body contains strong signals about its internal structure and composition that can be learned by machine learning algorithms. Recent advances in 3D scanning technology have made 3DO scanning of human bodies more accessible and widely distributed than ever [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. However, current work on learning body composition from 3DO scans relies on linear mathematical models, such as principal component analysis (PCA) and linear regression. These simplified linear assumptions impose potentially erroneous prior assumptions on the parameterization of both the 3D shape model and the mapping between shape parameters and body composition. Nonlinear methods that relax restrictive assumptions on the estimated functions may align better with the true relationship between shape and body composition and provide better prediction accuracy of target variables. An accurate model for estimating total and regional body composition metrics from 3DO scans could standardize body composition assessment by removing the costs and risks associated with clinical evaluation and irradiation. We propose to use deep, nonlinear methods to better estimate shape parameterization and body composition relative to prior work with linear baselines.\u003c/p\u003e \u003cp\u003eIn this work, we trained a deep 3D graph convolutional autoencoder on a diverse sample of full-body 3DO scans. A subset of these scans was captured with paired DXA scans for ground truth body composition training targets. We trained a regression model from extracted 3D graph convolutional features in the deep autoencoder network to DXA body composition variables using a nonlinear Gaussian process regression (GPR). We observed the effect of manipulating depths and dimensions of convolutional features on prediction accuracy and compared nonlinear estimates of 3D reconstruction error and body composition prediction precision and accuracy to results produced by linear methods from prior works. To our knowledge, this is the first application of a 3D graph convolutional autoencoder to body composition prediction. Our results indicate that nonlinear regression using a GPR with a squared dot product kernel provides greater accuracy and precision in body composition estimation on test data than previous linear methods.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cp\u003eThis work performs an ablation study between linear and nonlinear methods for shape modeling and body composition estimation of 3D human body shape. We assembled a composite dataset of 3DO human body shape from four independent sources and standardized them to a common topology. We trained a deep 3D autoencoder (3DAE) based on the work of Zhou et al. [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] on the composite dataset at multiple parameter sizes and evaluated its 3D reconstruction error on test data. 10 variables of body composition were studied consistent with prior work with reference values obtained with same-day DXA scans. The variables studied were:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eTotal body fat mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eTotal body lean mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eVisceral fat mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eArm lean mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eLeg lean mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003ePercent body fat\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eTrunk fat mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eTrunk lean mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eArm fat Mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eArm lean mass\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eWe trained nonlinear GPR models from 3DAE deep features at multiple scales to predict reference DXA values. We then trained a PCA model as a linear baseline using the same training data and parameter counts as the nonlinear model as an ablation study against prior work. From these linear PCA features we trained linear least squares regression and nonlinear GPR models to create linear-linear and hybrid linear-nonlinear models for DXA body composition prediction. This iterative modeling allowed us to quantify the impact of novel applications of nonlinear methods against linear baselines.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eExperimental Cohort\u003c/h2\u003e \u003cp\u003eFour data sets were used for this study: CAESAR [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], Shape Up! Adults (SUA) (NIH R01 DK109008) [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], Shape Up! Kids (SUK) (NIH R01 DK111698) [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e], and DFAUST [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows how this data was subdivided to train and test our models.\u003c/p\u003e \u003cp\u003eThe CAESAR 3DO scan dataset represented 2400 American and Canadian adults aged 18\u0026ndash;65 3D scanned in a neutral A-pose, with legs and arms held fully extended and abducted\u0026thinsp;~\u0026thinsp;30 degrees from the midline of the body. Participants were mostly unclothed except for form-fitting gray underwear and a swim cap to standardize hair appearance. Roughly half of the recruited cohort were female. Sex, height, and weight were the only demographic variables used to construct the shape model; no other demographic collected was used in this study. Subjects were scanned on a custom-built Cyberwar WB4 3D scanner.\u003c/p\u003e \u003cp\u003eThe SUA and SUK datasets (ClinicalTrials.gov ID NCT03706612 (SUK) and ID NCT03637855 (SUA)) were cross-sectional and stratified by age (SUK: 5\u0026ndash;8, 9\u0026ndash;12, 13\u0026ndash;17\u0026nbsp;year., SUA: 18\u0026ndash;39, 40\u0026ndash;59, \u0026ge;\u0026thinsp;60\u0026nbsp;year.), ethnicity (non-Hispanic white, non‐Hispanic black, Hispanic, Asian, and Native Hawaiian or other Pacific Islander (NHOPI)), sex, and BMI Z-score. Along with extensive demographics, quantitative measures included whole body DXA scans and 3DO scans. We acquired duplicate whole-body DXA scans of each participant on either a Hologic Horizon/A system (UCSF) or a Discovery/A system (PBRC and UHCC) (Hologic Inc., Marlborough, MA, USA). Participants were positioned and scanned according to guidelines specified by the respective system manufacturers. All scans were analyzed at UHCC by a single certified technologist using Hologic Apex version 5.6 with the National Health and Nutrition Examination Survey (NHANES) Body Composition Analysis calibration option disabled. DXA systems quality control was performed by monitoring the weekly values of the Hologic Whole Body Phantom. Two independent 3DO scans were acquired for each participant in up to three scanning devices: Fit3D Proscanner 4.x, Fit3D Inc, Redwood City, CA, USA, Styku S100 4.1, Styku LLC, Los Angeles, CA, USA, and Size Stream SS20, Size Stream, Cary, NC, USA. All 3DO scans were captured in a neutral A-pose that closely mirrored the pose of the CAESAR dataset.\u003c/p\u003e \u003cp\u003eThe DFAUST dataset was a 4D capture of human pose. A templated mesh was registered to over 41,000 snapshots from continuous sequence of 3D point clouds (3dMD LLC, Atlanta, GA) of 10 unique individuals performing dynamic movements captured at 60 frames per second. 4264 meshes were reserved for testing and were not used in this study. No clinical data such as body composition was reported.\u003c/p\u003e \u003cp\u003eThe DFAUST dataset was used to pretrain the 3DAE model. The pretraining step initialized the network weights to a state similar to the presentation in the original work of Zhou et al. [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] An ensemble dataset of CAESAR, SUA, and SUK was used as training data to finetune the model. During deep network training, a held-out data split, the evaluation set, was reserved to benchmark the performance of the model at the conclusion of each training epoch as determined by its geometric reconstruction loss. 20% of the CAESAR data was reserved for the finetuning evaluation steps (FT eval). To create a standardized benchmark between the methods, the same training and test split in Tian et al. [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] was preserved to investigate the performance of the new deep autoencoder and nonlinear GPR on the same test set. The test set was a random sample of 20% of the available Shape Up! Adults scans.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e3D scan count and unique individual representation for the pretrain, finetune, evaluation, and test sets. The pretraining step used only DFAUST data and included its own evaluation split consisting of 20% of the DFAUST scans. The finetune step was an iterative training step performed using only CAESAR, SUA, and SUK data starting with the network weights determined by the pretraining step. The evaluation set was used to determine the best performing model state to save without causing overfitting to the training data. The test set was kept the same as prior work to create controlled comparisons.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDataset\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003eTotal 3D scans / Total unique individuals\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003ePretraining\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003eFinetuning\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003eFT Eval\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003eTest\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDFAUST\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e36,956 / 10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCAESAR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2140 / 2014\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e462 / 462\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eShape Up! Adults\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1500 / 542\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e424 / 336\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eShape Up! Kids\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e646 / 200\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eTotal\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4286 / 2882\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTo enforce topological consistency between all 3DO meshes, all scans in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e were standardized to a constant topology containing 6890 vertices (referred to as the SMPL [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] topology) except for the DFAUST data, which is already in this format natively. We used the automated template fitting method of Tian et al. [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] and the nonrigid deformation of Allen et al [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] for this process. Implementation details are provided in the Supplementary Materials. A visual example of this process is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe ensemble 3DO data selected for this study collectively covered many of the limitations inherent to each individual dataset. Clinically sampled, multi-identity datasets like SUA and SUK are custom-made for studies modeling human shape variation and body composition estimation across a diverse, cross-sectional sample of the population. SUA and SUK contain 3D meshes and body composition reference values for adults and kids respectively but are low in participation due to the additional overhead and difficulty of collecting clinical data. CAESAR augments this dataset by doubling the number of single-pose 3DO scans available while almost quadrupling the number of unique individuals represented. However, since there is no clinical data associated with CAESAR, it can only be used to train the 3D shape model and not the mapping to body composition. Only CAESAR scans were held out for model evaluation during the finetuning step of 3DAE training to preserve as many SUA and SUK scans with paired DXA measurements as possible for body composition regression training and testing.\u003c/p\u003e \u003cp\u003eThe DFAUST data was different from the other data subsets as it contained very few unique individuals (10, 5 male and 5 female) but the largest number of unique 3DO scans (~\u0026thinsp;37,000). This dataset varied pose instead of individual identity to create shape diversity. Modeling shape variation due to posing is not an objective of this study and DFAUST was not used to jointly train the 3DAE shape model with the other datasets. However, DFAUST was very useful in initializing the 3DAE weights to a better-than-random starting state for training on the CAESAR-SUA-SUK ensemble dataset.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e3D Deep Autoencoder with Graph Convolutional Network\u003c/h2\u003e \u003cp\u003eRaw 3DO scans contain potentially hundreds of thousands of unorganized vertices and are not suitable inputs for regression algorithms without first processing then into standardized feature vectors for all dataset members. In this study, we use a 3D graph convolutional neural network adapted from Zhou et al. [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] to perform nonlinear dimensionality reduction and feature extraction on the 3D body shape space. This deep network is a 3D autoencoder that possesses many attributes inherent to deep convolutional neural networks (CNNs). A local kernel operator paired with layered feature pooling and unpooling, equivalent to multilevel filters in 2D image CNNs, gives this method local feature sensitivity while enabling the representation of nonlinear relationships between the latent encoding and the decoded shape. For a 3D autoencoder, the inputs are the 3D mesh vertex coordinates represented as 6890x3 sized tensors. The loss is a geometric mean absolute error (MAE) loss minimizing the reconstruction error against the original input coordinates.\u003c/p\u003e \u003cp\u003eUnlike image convolution on a regular square grid, the 3D mesh graph is irregular with varying connectivity. This requires some architectural modifications in the network to apply convolutional operations to human body scans. Our chosen implementation of a 3DAE assumes topological consistency of all mesh inputs. This simplifies the network design by allowing us to determine the spatial down sampling and upsampling operations once per mesh template in a preprocessing step. We defined both the convolutional kernel radius and the spatial kernel stride as two to be consistent with Zhou et al. [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] A visualization of the precomputed graph down sampling for each layer of the autoencoder is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. Additional implementation details are written in the Supplementary Materials and in Zhou et al.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eData paucity was one of the primary reasons for using linear algorithms rather than a deep network in prior work as 3DO data paired with ground truth DXA measurements are absent in the literature outside of the limited SUA and SUK collections. We initialized our model with the DFAUST dataset as augmentation data to mitigate the lack of data availability. We pretrained our 3DAE on DFAUST for 200 epochs, then trained on a composite of SUA, SUK, and CAESAR for an additional 400 epochs excluding DFAUST. We trained 3DAE models bottleneck channel depths of 7, 43, 90, and 601. With 7 nodes at the bottleneck layer, these channel depths defined total latent feature vector sizes of 49, 301, and 630, and 4284 as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. All models were trained with a kernel basis size of \u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;37. The learning rate was set to 1e-4. Training batch size was set to at 16. (See Zhou et al. for implementation details) All model variations had the same architecture and training data aside from the bottleneck layer.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eLearning a nonlinear transfer function with GPR\u003c/h2\u003e \u003cp\u003eFor body composition analysis, we performed nonlinear Gaussian process regression between the features extracted from the SUA dataset and their paired DXA measurements. Prior work on estimating DXA body composition from 3D body shape used least squares linear regression, which imposed a restrictive assumption of linearity on the relationship between body shape variables and body composition. GPR was previously used in Wang et al. [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] for visceral adipose tissue estimation from body circumferences and is a nonlinear, probabilistic generalization of linear regression that relaxes the imposition of linearity on the function between body shape features and body composition. GPR is more generalized than linear regression but not as unrestricted as a multi-layer perceptron network (MLP). The limited number of data observations in our dataset made GPR a more appropriate regression method than MLPs while still relaxing tight assumptions on the function shape.\u003c/p\u003e \u003cp\u003eWe trained a GPR model to learn a nonlinear mapping between the encoded latent vectors of the training data and the DXA measurements for 10 body composition measures. We chose a squared dot product kernel for GPR under the assumption that relationships between body shape features and body composition were nonlinear but monotonic in both the first and second derivatives. Our assumption of monotonicity between body shape and body composition is consistent with the observation that variables such as visceral fat and percent fat are positively correlated with anthropometric measurements such as waist circumference and body volume [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. We experimented with other kernels such as the radial basis function (RBF) and higher powers of dot product kernels and found they performed worse than a squared dot product kernel. We present results for the squared dot product kernel in this paper. We trained GPR models for male and female participants separately.\u003c/p\u003e \u003cp\u003eGPR derivation details can be referenced in the Appendix and in greater detail in [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] and [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. We implemented GPR using scikit-learn [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. We concatenated [height, weight, age] of each participant to the feature vector input to GPR in accordance with the procedures established in previous work. To comprehensively search the feature representation space across multiple scales for the best predictive inputs, we performed GPR on all intermediate feature layers of the 3DAE shape model to predict body composition targets. For each of the four levels of deep features shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, we performed GPR to body composition targets. We reported the prediction accuracies for the bottleneck layer containing 4284 (7x612) features and for the feature layer producing the lowest RMSE in GPR prediction.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eComprehensive ablation trials vs. linear methods via model permutations\u003c/h2\u003e \u003cp\u003ePCA is a common linear approach to dimensionality reduction for 3D human body shape due to the widespread adoption of methods such as SMPL [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] and was the shape modeling method used in prior work on 3DO body composition estimation. PCA is a deterministic linear operation with a globally optimal solution and produces feature vectors over a space of orthogonal components, making it well-behaved even for datasets containing just tens to hundreds of scans [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eTo test the comparative performance of new nonlinear models against linear baselines established in previous work, we trained a PCA model using the same 4286 training meshes of the 3DAE. DFAUST was not used for training PCA shape models. Although the test set membership of this study is the same as that of Tian et al (2022) [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], the training set in this work is greatly expanded. Recreating PCA models with consistent data membership allowed us to isolate the effects of deep 3DAE shape encoding and nonlinear GPR prediction relative to a baseline of PCA and ordinary least squares (OLS) on the same data. Due to the predefined downsampling of the 6890-vertex mesh topology shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, the bottleneck layer of the 3DAE must be a multiple of 7. We set the maximum bottleneck (latent code) size of our 3DAE to 4284 as it was the closest multiple of 7 to 4286, the maximum number of possible PCA components (corresponding to the number of meshes in the training set). While the 3DAE bottleneck layer dimension could be increased to an arbitrarily high number, resulting in lower reconstruction error, there would not be an equivalent linear PCA model to function as a comparative baseline.\u003c/p\u003e \u003cp\u003eTo further test our hypothesis regarding the better prediction accuracy of nonlinear GPR relative to OLS, we trained a GPR model using the same PCA basis from Tian et al. (2022) [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], labeled PCA-GPR M391/F457 which indicates 391 and 457 components for males and females respectively.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eStatistical Analysis\u003c/h2\u003e \u003cp\u003e We measured the geometric reconstruction accuracy of both linear and nonlinear shape models at different model sizes to assess the marginal contribution of nonlinear autoencoders to shape modeling accuracy relative to a linear PCA method used in prior work. We then iterated through different model permutations consisting of feature extraction with both linear and nonlinear shape models followed by both linear and nonlinear regression to DXA body composition measurements. Model permutations demonstrated the marginal contributions of GPR and 3DAE to the precision and accuracy of body composition prediction relative to the baselines established by linear algorithms used in prior work.\u003c/p\u003e \u003cp\u003eWe compared 3D geometric reconstruction error between the PCA baseline and 3DAE shape models at four model sizes as the per-vertex mean absolute error (MAE). We compared the accuracy of body composition estimation for four model permutations: the GPR baseline estimated using only demographic features [height, weight, age] with no shape information, the linear baseline consisting of ordinary least squares regression from PCA shape features (PCA-OLS), hybrid consisting of GPR using PCA shape features (PCA-GPR), and fully nonlinear GPR from deep shape features (3DAE-GPR). We quantified estimation error with root-mean-squared-error (RMSE) relative to reference DXA measurements and plotted the normalized RMSE of each predicted metric as a percentage of the PCA-OLS fully linear baseline. The coefficient of determination (R\u003csup\u003e2\u003c/sup\u003e) for agreement to DXA was assessed for the 3DAE-GPR model.\u003c/p\u003e \u003cp\u003eWe predicted body composition on test-retest data pairs to compare the precision of our model permutations to prior work and to DXA scanners using the coefficient of variation (%CV) of visceral fat and the repeat RMSE of percent fat. We compared the precision of 3DAE-GPR at two model sizes against the PCA-GPR and PCA-OLS permutations, trained and tested on the same training data and test-retest pairs. For 3DAE-GPR, we trained the GPR component on the bottleneck layer (301 and 4284 features for the small and large model sizes respectively) to enforce consistency with the parameter count of the PCA permutations.\u003c/p\u003e \u003cp\u003eWe compared the accuracy of our best model performance to a comprehensive list of prior work using percent fat and visceral fat prediction as the benchmark and showed that we can achieve state of the art results on 3DO body composition prediction using deep convolutional feature extraction and nonlinear regression.\u003c/p\u003e \u003cp\u003eWe conducted ablation studies to evaluate the effects of skipping model training steps or withholding training data on geometric reconstruction accuracy and body composition prediction accuracy to confirm that all training procedures and all subsets of the data described in the method positively contributed to the accuracy of our results. We recorded the reconstruction accuracy of the 3DAE using a random initialization without the DFAUST pretrained initialization. We then tested the inverse data withholding condition by training a 3DAE on DFAUST only with no further finetuning with our multi-identity ensemble dataset on SUA test data. We withheld CAESAR, SUK and both CAESAR and SUK data from the training sets to test if dissimilarities between data subsets may have reduced reconstruction or regression performance. We tested the geometric reconstruction accuracy and the body composition prediction accuracy for each exclusion scenario. Ablation trials were tested on a \u003cem\u003ed\u003c/em\u003e\u0026thinsp;=\u0026thinsp;301 sized bottleneck network due to its much lower training time.\u003c/p\u003e \u003c/div\u003e"},{"header":"RESULTS","content":"\u003cp\u003eThe Shape Up! Adults study population has been previously described [15] and summarized in Table S1 in the Supplementary Material. Only this subset of the total data was used for body composition regression training and testing.\u003c/p\u003e\n\u003cp\u003eGeometric reconstruction error for both 3DAE and PCA shape models of four increasing sizes are shown in Table 3. The dimensionality \u003cem\u003ed\u003c/em\u003e represents either the number of PCA coefficients used to parameterize shape (linear model) or the number of latent variables in the bottleneck layer of the 3DAE (nonlinear model) connecting the encoder module to the symmetric decoder. Reconstruction error was calculated as the geometric mean absolute error (MAE) between original and reconstructed vertex 3D positions. As expected, larger models were able to reconstruct the test data with lower error. Both linear and deep methods are comparable in terms of geometric reconstruction accuracy for the first three model sizes. PCA achieved lower geometric reconstruction error at the highest parameter count while 3DAE reconstruction error leveled off at just above 2mm MAE. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e2\u003c/strong\u003e\u003cstrong\u003e.\u003c/strong\u003e Per-vertex reconstruction error measured as mean absolute error (MAE) between input and decoded meshes for held-out test meshes from Shape Up! Adults (n=424). The dimensionality (d) represents the number of latent layer variables in the autoencoder bottleneck or the number of principal components used for a linear shape space reconstruction.\u003c/p\u003e\n\u003cdiv align=\"\"\u003e\n \u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eD\u003c/em\u003e=49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003ed\u003c/em\u003e=301\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003ed\u003c/em\u003e=630\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003ed\u003c/em\u003e=4284\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e3DAE MAE (mm) \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e5.16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2.57\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2.18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2.02\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003ePCA MAE (mm)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e5.26\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2.41\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e2.23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e1.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eFigure 3 depicts the body composition prediction results from \u003cem\u003ed\u003c/em\u003e=4284 sized models using different permutations of linear and nonlinear shape feature extraction and regression methods. Excluding the baseline column, each subsequent column represents a change of exactly one model parameter holding all others constant; i.e. OLS to GPR, PCA to 3DAE, 4824 total parameters to 400x64 parameters. All model permutations were trained and tested on the exact same mesh data.\u003c/p\u003e\n\u003cp\u003eThe baseline GPR model accuracy is the RMSE resulting from a regression using only known features [height, weight, age] without any conditioning on the 3DO shape features as was previously done in [15] and [29]. This RMSE was higher than any subsequent regression model using shape features as input. The reduction of prediction error when shape information is introduced demonstrated that 3DO shape is a useful signal for body composition prediction even when nonlinear regression algorithms are employed.\u003c/p\u003e\n\u003cp\u003ePCA-OLS is equivalent to the linear methods of prior work. This is a linear regression model for body composition prediction taking PCA features as inputs. The PCA model had 4284 parameters to stay exactly consistent with the dimensionality the 3DAE.\u003c/p\u003e\n\u003cp\u003ePCA-GPR represents a hybrid pipeline predicting body composition with nonlinear GPR from linear PCA features. Nonlinear regression from linear shape features achieved lower RMSE on every predicted metric relative to linear regression except arm lean on females (which was equal) and leg lean on females (which was higher by 0.02 kg, or 5%). These results indicated that GPR was a more accurate regression method than OLS when all other factors were held constant for most body composition targets.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e3DAE-GPR represents the fully nonlinear pipeline where body composition is predicted from GPR using 3DAE deep features as inputs. The results for both the bottleneck layer (7x612) and the layer with the lowest RMSE are shown. The third layer (dimensions 400x64) was the most accurate feature layer for males and the first layer (dimensions 6890x16) was the best for females. In males, 3DAE feature extraction lowered RMSE relative to the previous model permutations, but in females no level of feature extraction outperformed GPR on the original mesh coordinates. This result suggests the features extracted from female meshes were less informative relative to male features or were highly correlated regardless of the method and model size. GPR always improves accuracy relative to OLS, but 3DAE feature extraction only improved accuracy for males.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOLS regression to body composition with 3DAE features resulted in very low correlations with DXA due to the nonlinearity of the deep features and was not reported. We also tested concatenating all feature layers into a single multi-scale feature vector for GPR regression but did not achieve lower errors in doing so. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe rescaled the charts in Figure 3 by normalizing each column by the RMSE of the fully linear PCA_OLS model and plotted the values in Figure 4 for visual clarity. In males, all subsequent model permutations had RMSEs less than that of PCA_OLS (indicated by a normalized RMSE of 100%). For females, only leg lean increased in RMSE when moving from PCA_OLS to PCA_GPR. However, as previously observed most RMSEs increased in females when incrementing PCA to 3DAE. When comparing nonlinear methods to linear methods, GPR is more accurate for body composition prediction in most metrics, but 3DAE is not always a better feature extraction method for body composition prediction from shape relative to linear PCA in females.\u003c/p\u003e\n\u003cp\u003eTest-retest precision of percent fat and visceral fat estimation is shown in Table 3. For visceral fat mass precision, Coefficient of Variation (%CV) was calculated according to the definition in Gl\u0026uuml;er et al. [30] between two scans of the same individual in the test set taken on the same day. RMSE between trial 1 and trial 2 was shown for percent fat precision as the %CV of a percentage measurement is not used in convention.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eGPR was more precise on retests than OLS as shown by the PCA-OLS versus PCA-GPR trials, where the latter resulted in up to a 30% decrease in precision error. 3DAE also decreased precision error relative to PCA in both sexes, as illustrated by the 3DAE-GPR versus PCA-GPR trials. 3DAE coupled with nonlinear GPR had the highest precision. Compared to DXA, percent fat precision error for 3DAE-GPR was roughly twice as high. However, visceral fat precision was lower than DXA, indicating the 3DAE model paired with GPR prediction is much more reliable on retest accuracy than prior work. The larger \u003cem\u003ed\u003c/em\u003e=4284 3DAE model was comparable in precision to the smaller model. 3DAE was more precise than PCA, and GPR was more precise than OLS.\u003c/p\u003e\n\u003cp\u003eTable 3. Test-retest precision between repeat 3DO scan pairs of each participant in the SUA test set on the same scan device. 3DAE models were benchmarked with GPR trained from their bottleneck layers to standardize the comparison between model sizes. 3DAE was more precise than PCA, and GPR was more precise than OLS.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"633\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.037974683544302%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eVisceral fat % CV\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.60126582278481%\" valign=\"top\"\u003e\n \u003cp\u003e3DAE-GPR 301\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e3DAE-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-OLS 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003eTian et al. 2022 [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.39873417721519%\" valign=\"top\"\u003e\n \u003cp\u003eDXA (Criterion)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.037974683544302%\" valign=\"top\"\u003e\n \u003cp\u003eMales (n = 143)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.60126582278481%\" valign=\"top\"\u003e\n \u003cp\u003e4.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e5.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e5.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e7.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e8.8%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.39873417721519%\" valign=\"top\"\u003e\n \u003cp\u003e4.8%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"18.037974683544302%\" valign=\"top\"\u003e\n \u003cp\u003eFemales (n=199)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.60126582278481%\" valign=\"top\"\u003e\n \u003cp\u003e5.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e5.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e6.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e8.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.240506329113924%\" valign=\"top\"\u003e\n \u003cp\u003e12.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.39873417721519%\" valign=\"top\"\u003e\n \u003cp\u003e6.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"636\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.896389324960754%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003ePercent fat precision RMSE\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.675039246467819%\" valign=\"top\"\u003e\n \u003cp\u003e3DAE-GPR 301\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e3DAE-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-OLS 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003eTian et al. 2022 [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003eDXA\u003c/p\u003e\n \u003cp\u003e(Criterion)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.896389324960754%\" valign=\"top\"\u003e\n \u003cp\u003eMales (n = 143)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.675039246467819%\" valign=\"top\"\u003e\n \u003cp\u003e1.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e0.9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e1.1%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e1.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e1.9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e0.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.896389324960754%\" valign=\"top\"\u003e\n \u003cp\u003eFemales (n=199)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.675039246467819%\" valign=\"top\"\u003e\n \u003cp\u003e1.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e1.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e1.4%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e2.0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e2.9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.285714285714286%\" valign=\"top\"\u003e\n \u003cp\u003e0.5%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eComparisons of our 3DAE-GPR models against prior work are shown in Table 4 using the \u003cem\u003ed\u003c/em\u003e=4284 latent size model and Gaussian process regression with the best performing feature layer (third for males, first for females). Percent fat (PFAT) and visceral fat mass (VFAT) were selected as the comparative target variable as they achieved the lowest accuracy in previous works based on linear models.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eGPR using the exact same PCA features of Tian et al. (PCA-GPR M391/F457) lowered the RMSE from OLS for percent fat in both sexes and in female visceral fat but increased it slightly in male visceral fat by 8%. The models presented in [15] were built on a dataset containing an order of magnitude fewer members than what we presented in this work. These results suggest linear pipelines may be more competitive with nonlinear methods when training data quantity is more limited. We included RMSEs for our best performing, maximum size PCA model (PCA-GPR 4284) to demonstrate that increased parameter count does not cause overfitting and degrade accuracy relative to a sparser model.\u003c/p\u003e\n\u003cp\u003eOur fully nonlinear model, 3DAE-GPR, produced the lowest error on visceral fat and percent fat estimation compared to all prior work on 3DO body composition estimation (bolded). We note that compared to Tian et al (2022), our best 3DAE-GPR model achieved lower RMSE on all of the 10 body composition metrics measured in Figure 4, shown in Supplementary Table 2. However, predicted metrics other than percent fat and visceral fat already showed high correlation with DXA in prior work using linear methods. Thus, we focused the comparison to prior work in Table 4 on the metabolically significant and previously underperforming predictions of percent fat and visceral fat. The test set used in this work was held the same as [15]. However, the training dataset was greatly expanded in size and scope relative to prior works.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 4. Root-mean-squared errors (RMSE) for predicted percent fat (PFAT) and visceral fat (VFAT) of all current 3D-optical body composition prediction literature on Shape Up! Adults compared to the 3DAE-GPR prediction of the d=301 and d=4284 models using the most accurate feature layer identified in Figure 3. Best performing values are bolded.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003ePaper\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cem\u003eN\u003c/em\u003e test meshes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003ePFAT RMSE (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eVFAT RMSE (kg)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003eNg et al (2019) Anthro only\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 177\u003c/p\u003e\n \u003cp\u003eF: 230\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 4.03\u003c/p\u003e\n \u003cp\u003eF: 3.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.15\u003c/p\u003e\n \u003cp\u003eF: 0.14\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003eNg et al (2019) [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 177\u003c/p\u003e\n \u003cp\u003eF: 230\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 3.55\u003c/p\u003e\n \u003cp\u003eF: 3.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.14\u003c/p\u003e\n \u003cp\u003eF: 0.13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003eTian et al (2020) [28]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 31\u003c/p\u003e\n \u003cp\u003eF: 39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 3.90\u003c/p\u003e\n \u003cp\u003eF: 3.29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.15\u003c/p\u003e\n \u003cp\u003eF: 0.17\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003eWong et al (2021) [31]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 159\u003c/p\u003e\n \u003cp\u003eF: 202\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 2.73\u003c/p\u003e\n \u003cp\u003eF: 3.46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.13\u003c/p\u003e\n \u003cp\u003eF: 0.13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003eTian et al (2022) [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 182\u003c/p\u003e\n \u003cp\u003eF: 248\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 3.24\u003c/p\u003e\n \u003cp\u003eF: 4.22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.12\u003c/p\u003e\n \u003cp\u003eF: 0.14\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-GPR M391/F457 [15]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 182\u003c/p\u003e\n \u003cp\u003eF: 248\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 2.79\u003c/p\u003e\n \u003cp\u003eF: 3.09\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003eM: 0.13\u003c/p\u003e\n \u003cp\u003eF: 0.12\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003ePCA-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 181\u003c/p\u003e\n \u003cp\u003eF: 239\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003eM: 2.68\u003c/p\u003e\n \u003cp\u003eF: 2.85\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eM: 0.11\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eF: 0.11\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.92%\" valign=\"top\"\u003e\n \u003cp\u003e3DAE-GPR 4284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"21.44%\" valign=\"top\"\u003e\n \u003cp\u003eM: 181\u003c/p\u003e\n \u003cp\u003eF: 239\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.96%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eM: 2.50\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eF: 2.81\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.68%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eM: 0.11\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eF: 0.11\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eAblation results\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFigure 5 shows a mesh reconstruction using a \u003cem\u003ed\u003c/em\u003e=4284 model trained with 400 epochs on the finetuning ensemble training data from 1) a random initialization state and 2) from a pretrained initialization trained with 200 epochs using DFAUST data only. The model trained from a random initialization achieved 22.7mm MAE on test data reconstruction, more than 10x the error of the error show in Table 3. The pretraining steps using 40,000 DFAUST meshes was essential for creating an accurate shape model using a nonlinear 3DAE.\u003c/p\u003e\n\u003cp\u003eFigure 6 shows visualizations of geometric reconstruction error as heat maps for a male and female subject before and after fine-tuning the \u003cem\u003ed\u003c/em\u003e=4284 3DAE model with high resolution Shape Up! and CAESAR data. Without fine-tuning, the 3DAE model was equivalent to the work presented in Zhou et al. [18] trained exclusively on DFAUST data and achieved a 3D reconstruction error of 8.7 mm, as opposed to the 2.0 mm shown in\u0026nbsp;Table 2\u0026nbsp;for the finetuned model. This model generalized very poorly to unseen scans of many unique individuals such as in Shape Up!, as DFAUST only contained 10 unique individuals captured in thousands of different poses. A 3DAE model for clinical machine learning applications needs to generalize well to any individual scanned from the general population in a neutral pose. Our fine-tuned model represented the non-rigid, identity-dependent deformations of unique individuals much more accurately.\u003c/p\u003e\n\u003cp\u003eWithholding non-SUA data from 3DAE training did not improve 3D reconstruction error on SUA test meshes. In all three ablation trials, reconstruction accuracy on the same SUA test set (2.58mm, 2.71mm, 2.76mm for removing only SUK, only CAESAR, and both, respectively) was worse than the model trained on all data combined (reconstruction accuracy of 2.57) in Table 3. This ablation result validates the inclusion of a diverse dataset across multiple collection protocols.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eGPR models trained on the 3DAE model excluding SUK data showed 1% difference in RMSE in both directions. Not all target variables were uniformly lower in error when including versus excluding SUK. The variation could be attributed to noise and does not justify excluding SUK. Excluding CAESAR data from the training scans had similarly negligible effects on male prediction accuracy but increased female RMSEs without exception by up to 4%. Excluding both SUK and CAESAR produced results similar to the case where only CAESAR was excluded, but with even higher errors in females. Overall, we found that including all available 3D mesh data when training shape and regression models produced the lowest errors for 3D reconstruction and body composition prediction.\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eThis study showed three primary findings. First, GPR generally improves body composition prediction accuracy and precision relative to linear regression regardless of whether the feature domain is linear (PCA) or nonlinear (3DAE). By comprehensively exploring model permutations and changing exactly one variable at each iteration, we were able to demonstrate that nonlinear regression with GPR performed better than OLS for every body composition target except female leg lean. Second, 3DAE feature extraction improved body composition prediction for males. However, deep features were not more informative than the unencoded mesh geometry in females and did not outperform PCA in geometric reconstruction at high parameter counts. Third, both 3DAE and GPR achieved higher precision than their linear counterparts in both males and females when measured on percent fat and visceral fat mass.\u003c/p\u003e \u003cp\u003eA combination of an expanded and diversified training set with deep feature extraction and nonlinear GPR prediction allowed us to build an end-to-end model that outperformed all prior works on body composition prediction accuracy. The improved accuracy and precision demonstrated here builds on our prior work [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] and further establishes 3DO as a reliable and accessible clinical tool for the assessment of body composition. Such a tool has promising clinical implications for the management of chronic disease, where body composition has known associations with morbidity and mortality.\u003c/p\u003e \u003cp\u003eTo our knowledge this was the first application of deep nonlinear autoencoder networks to 3DO body composition estimation. We explicitly chose a spatial graph convolutional network for our autoencoder approach. Other autoencoder approaches, such as implicit surface encoding [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e], non-convolutional variational architectures [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e], and spectral graph convolution [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e] have been used to study body shape, but without associations to clinical outcomes. Future work may find that these other approaches have advantages to the work presented here.\u003c/p\u003e \u003cp\u003eWe hypothesized that lower geometric reconstruction error in a shape autoencoder model was correlated with higher body composition prediction accuracy during regression from the extracted features. However, although a linear PCA model of size \u003cem\u003ed\u003c/em\u003e\u0026thinsp;=\u0026thinsp;4284 produced the lowest shape reconstruction error, it did not outperform 3DAE on body composition prediction accuracy or precision. Geometric reconstruction accuracy may be affected by shape deformations irrelevant to body composition variation such as pose or face detail. PCA may be outperforming 3DAE in shape reconstruction at high parameter counts due to 3DAE potentially learning redundant and correlated features when the model size is large. Unlike 3DAE, PCA is guaranteed to learn uncorrelated, orthogonal features due to its mathematical construction. Future work should investigate the relationship between geometric reconstruction accuracy and body composition prediction accuracy by controlling for uncorrelated geometric variations and using different autoencoder architectures.\u003c/p\u003e \u003cp\u003ePCA\u0026rsquo;s high performance on shape reconstruction validate the methods of past work built on PCA models with linear regression [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] despite the restrictive linear assumptions of the algorithm. Prior work restricted the shape model and body composition prediction features to a sparse subset of the total number of PCA components to avoid overfitting during regression. Our extensive testing with different model sizes and permutations does not support the assumption that large parameter counts overfit on test data. PCA-GPR model using 4284 components achieved lower RMSEs on percent fat and visceral fat on both males and females relative to models trained with 391 or 457 features. Restricting shape and regression models to the first \u003cem\u003en\u003c/em\u003e components that describe 95% or 99% of the shape variance may not be justified in future work as it potentially handicaps prediction accuracies unnecessarily.\u003c/p\u003e \u003cp\u003eOur chosen autoencoder architecture does not explicitly disentangle pose deformation from identity-dependent deformations, such as in Wong et al. [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] or Jiang et al [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]. Factoring out slight pose variation across our dataset may improve the reconstruction accuracy and regression accuracy of our method. Increased sensitivity to small changes in body shape may allow our method to improve monitoring body composition change over time in the same individual [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e]. As our dataset was transformed to adopt the mesh topology of SMPL [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] while preserving the geometric surface detail of the original 3DO scans, it may be straightforward in future work to \u0026ldquo;unpose\u0026rdquo; every mesh in our 4286-member ensemble dataset to a neutral T-pose like that of Wong et al. using the skinning weights and joint position definitions of the SMPL template. The accuracy of this procedure will depend on how anatomically consistent the template correspondences are between our dataset and the meshes in Loper et al.\u003c/p\u003e \u003cp\u003eFuture work can explore the impact of different nonlinear regression methods on body composition prediction accuracy. GPR regularized the shape of the regression function to its kernel. A less restricted but more flexible regression method such as a multi-layer perceptron (MLP) may achieve better prediction error with proper regularization; however, deeper models run the risk of overfitting to the training data especially if the latent parameter count is large. We observed that GPR using a radial basis function kernel (RBF) already exhibits symptoms of overfitting, showing zero error on the training data but greatly increased error on the test data.\u003c/p\u003e \u003cp\u003eLike prior works using linear models, this work separately trains a shape feature extractor followed by a body composition regression from features. Future work may integrate the current two-step pipeline into a single end-to-end deep network. This can be achieved by connecting the 3DAE encoder layers directly to a neural network regression model that targets body composition as its output instead of self-reconstruction. This end-to-end model can be initialized to state reported in this work by importing our trained 3DAE weights into the encoder of the network and the weights of a separately trained regression model into the neural network regression layers. GPR may be recreated as a neural network in PyTorch with GPyTorch [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. This implementation could allow the combined network to achieve greater accuracy by starting from the state presented in this work and further optimizing all features to target body composition prediction with no intermediate objectives.\u003c/p\u003e \u003cp\u003eOur 3D deep shape model was trained on the largest collection of high quality, multi-identity 3DO scan data currently available, spanning CAESAR, SUA, SUK, and DFAUST over more than a 20-year period. However, with 2,900 unique individuals, this dataset is still orders of magnitude smaller than those used in analogous networks for deep 2D image learning. This restriction on dataset size and population variation sampling density can create gaps in our model that exhibit low reconstruction and prediction accuracy where training data was not available or under sampled, especially at the extremes of body shapes as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e. Ongoing data collection of high resolution 3DO scans will improve the reconstruction accuracy of our method. Improvement of the nonlinear regression model will require additional paired DXA scans.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe CAESAR data used in this work was built from the original 3D scan data collected by Robinette et al. [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] and not on a derived reconstruction such as the MPI CAESAR dataset produced by Pishchulin et al [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e]. The MPI dataset is a template standardization of CAESAR using a limited number of PCA features for reconstruction. The resulting shapes are linearly projected compressions of the original 3D scan data and do not preserve original high-resolution detail well, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e8\u003c/span\u003e. Including these shapes into our training database would bias our deep nonlinear model towards an approximation of a linear solution. Our remeshing of the CAESAR scans into the SMPL format includes a nonrigid surface-to-surface deformation that produced templated meshes consistent with the original scan geometry and did not constrain the training data to projections onto a linear subspace. Future work studying human shape variation as a function of varying identity rather than pose may build upon our results by utilizing the higher fidelity standardized templates of our dataset rather than training on potentially degraded 3D shape geometry.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"CONCLUSION","content":"\u003cp\u003eA comprehensive comparison of nonlinear methods for shape feature extraction and body composition regression against previous linear algorithms showed that nonlinear GPR improved body composition prediction accuracy and precision relative to linear regression for 10 metrics of body composition in males and for 9 in females. Nonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Relative to linear PCA feature computation, feature extraction with a deep 3D autoencoder provided marginal improvements to prediction accuracy for males but did not supersede the performance of GPR on raw mesh vertex positions for females. Deep shape features produced 6\u0026ndash;8% reduction in prediction error over linear PCA features for males. However, deep 3DAE features reduced precision error between 4\u0026ndash;14% relative to linear features with all other variables held equal. Our best performing nonlinear pipeline using 3DAE-GPR outperformed prior works on body composition prediction accuracy for all metrics. Precision error of our method is within 1-2x that of DXA, the gold standard for compartmental body composition measurement. Agreement with DXA as measured by R\u003csup\u003e2\u003c/sup\u003e values were greater than or equal to 0.86 for all predicted metrics. These findings improve the clinical utility of 3DO as accurate and accessible tool for the assessment of body composition. Future work in this space should explore the effects of disentangling pose variations from 3DO scans along with different deep network architectures such as variational latent encodings, end-to-end deep models, and predicting longitudinal change over time in individual subjects.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e3\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eDimensional Optical (3DO)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e60\u003c/div\u003e \u003cdiv class=\"Description\"\u003e\u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e001 vertex mesh template (60k)\u003c/div\u003e \u003cdiv class=\"Description\"\u003e\u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003ebody mass index (BMI)\u003c/div\u003e \u003cdiv class=\"Description\"\u003e\u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003edual X\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eray absorptiometry (DXA)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eprincipal component analysis (PCA)\u003c/div\u003e \u003cdiv class=\"Description\"\u003e\u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAuthor Contributions:\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIsaac Y. Tian: Methodology, Software, Formal Analysis, Investigation, Writing \u0026ndash; Original\u003c/p\u003e\n\u003cp\u003eJason Liu: Validation, Data Curation, Writing \u0026ndash; R\u0026amp;E\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMichael C. Wong: Methodology, Investigation, Writing \u0026ndash; R\u0026amp;E\u003c/p\u003e\n\u003cp\u003eNisa N. Kelly: Investigation, Writing \u0026ndash; R\u0026amp;E, Data Curation, Project administration\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Yong E. Liu: Investigation, Writing \u0026ndash; R\u0026amp;E, Data Curation\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Andrea K. Garber: Investigation, Writing \u0026ndash; R\u0026amp;E, Funding Acquisition\u003c/p\u003e\n\u003cp\u003eSteven B. Heymsfield: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing \u0026ndash; R\u0026amp;E\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Brian Curless: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing \u0026ndash; R\u0026amp;E\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;John A. Shepherd: Conceptualization, Methodology, Resources, Supervision, Funding Acquisition, Writing \u0026ndash; R\u0026amp;E\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding Disclosure:\u003c/strong\u003e AKG: NICHD #R01HD082166, National Institutes of Health NORC Center Grants (P30DK072476, Pennington/Louisiana and P30DK040561, Harvard); JAS: National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (R01DK109008 and R01DK111698).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflict of Interest Statement:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eJAS is a scientific advisor for Styku LLC and owns stock options. BC is employed by Google LLC and owns stock options. SBH is on the medical advisory boards of Tanita Corp. and Medifast Corp. and is a former employee of Merck \u0026amp; Co. SBH is an Amazon Scholar and owns stock options. Styku LLC and Fit3D Inc. contributed scanning equipment in support of NIH studies.\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData \u0026amp; Code Availability:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and/or analyzed during the current study are not publicly available due to the inclusion of protected health information (PHI) but are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003eThe underlying code for this study [and training/validation datasets] is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eWilson JP, Kanaya AM, Fan B, Shepherd JA. Ratio of trunk to leg volume as a new body shape metric for diabetes and mortality. PLoS ONE 2013;8. doi:10.1371/journal.pone.0068716.\u003c/li\u003e\n \u003cli\u003eKootaka Y, Kamiya K, Hamazaki N, et al. The GLIM criteria for defining malnutrition can predict physical function and prognosis in patients with cardiovascular disease. Clin Nutr. 2021;40(1):146-152.\u003c/li\u003e\n \u003cli\u003eAu PC, Li HL, Lee GK, et al. Sarcopenia and mortality in cancer: A meta-analysis. Osteoporos Sarcopenia. 2021;7(Suppl 1):S28-S33.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eGoodpaster BH, Krishnaswami S, Harris TB, Katsiaras A, Kritchevsky SB, Simonsick EM, et al. Obesity, regional body fat distribution, and the metabolic syndrome in older men and women. Archives of Internal Medicine 2005;165:777. doi:10.1001/archinte.165.7.777.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eZhang C, Rexrode KM, van Dam RM, Li TY, Hu FB. Abdominal obesity and the risk of all-cause, cardiovascular, and cancer mortality. Circulation 2008;117:1658\u0026ndash;67. doi:10.1161/circulationaha.107.739714.\u003c/li\u003e\n \u003cli\u003eHui WS, Liu Z, Ho SC. Metabolic syndrome and all-cause mortality: A meta-analysis of prospective cohort studies. European Journal of Epidemiology 2010;25:375\u0026ndash;84. doi:10.1007/s10654-010-9459-z.\u003c/li\u003e\n \u003cli\u003eDulloo AG, Jacquet J, Solinas G, Montani J-P, Schutz Y. Body composition phenotypes in pathways to obesity and the metabolic syndrome. International Journal of Obesity 2010;34. doi:10.1038/ijo.2010.234.\u003c/li\u003e\n \u003cli\u003eWilson JP, Kanaya AM, Fan B, Shepherd JA. Ratio of trunk to leg volume as a new body shape metric for diabetes and mortality. PLoS ONE 2013;8. doi:10.1371/journal.pone.0068716.\u003c/li\u003e\n \u003cli\u003eGalindo Martin CA, Aportela Vazquez VA, Becerril Hernandez F, et al. The GLIM criteria for adult malnutrition and its relation with adverse outcomes, a prospective observational study. Clin Nutr ESPEN. 2020;38:67-73.\u003c/li\u003e\n \u003cli\u003eSanchez-Rodriguez D, Locquet M, Bruyere O, et al. Prediction of 5-year mortality risk by malnutrition according to the GLIM format using seven pragmatic approaches to define the criterion of loss of muscle mass. Clin Nutr. 2021;40(4):2188-2199.\u003c/li\u003e\n \u003cli\u003eMarco E, Sanchez-Rodriguez D, Davalos-Yerovi VN, et al. Malnutrition according to ESPEN consensus predicts hospitalizations and long-term mortality in rehabilitation patients with stable chronic obstructive pulmonary disease. Clin Nutr. 2019;38(5):2180-2186.\u003c/li\u003e\n \u003cli\u003eRondel A, Langius JAE, de van der Schueren MAE, Kruizenga HM. The new ESPEN diagnostic criteria for malnutrition predict overall survival in hospitalised patients. Clin Nutr. 2018;37(1):163-168\u003c/li\u003e\n \u003cli\u003eFosb\u0026oslash;l M\u0026Oslash;, Zerahn B. Contemporary methods of body composition measurement. Clinical Physiology and Functional Imaging 2014;35:81\u0026ndash;97. doi:10.1111/cpf.12152.\u003c/li\u003e\n \u003cli\u003eNg BK, Sommer MJ, Wong MC, Pagano I, Nie Y, Fan B, et al. Detailed 3-dimensional body shape features predict body composition, blood metabolites, and functional strength: The shape up! studies. The American Journal of Clinical Nutrition 2019;110:1316\u0026ndash;26. doi:10.1093/ajcn/nqz218.\u003c/li\u003e\n \u003cli\u003eTian IY, Wong MC, Kennedy S, Kelly NN, Liu YE, Garber AK, et al. A device‐agnostic shape model for automated body composition estimates from 3D optical scans. Medical Physics 2022;49:6395\u0026ndash;409. doi:10.1002/mp.15843.\u003c/li\u003e\n \u003cli\u003eTinsley GM, Moore ML, Benavides ML, Dellinger JR, Adamson BT. 3-dimensional optical scanning for body composition assessment: A 4-component model comparison of four commercially available scanners. Clinical Nutrition 2020;39:3160\u0026ndash;7. doi:10.1016/j.clnu.2020.02.008.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eWenninger S, Achenbach J, Bartl A, Latoschik ME, Botsch M. Realistic virtual humans from smartphone videos. 26th ACM Symposium on Virtual Reality Software and Technology 2020. doi:10.1145/3385956.3418940.\u003c/li\u003e\n \u003cli\u003eY. Zhou, C. Wu, Z. Li, C. Cao, Y. Ye, J. Saragih, H. Li, and Y. Sheikh, \u0026ldquo;Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels,\u0026rdquo; in Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS\u0026apos;20), 2020, pp. 9251\u0026ndash;9262.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eRobinette KM, Blackwell S, Daanen H, Boehmer M, Fleming S. Civilian American and European Surface Anthropometry Resource (CAESAR), final report. volume 1. summary 2002. doi:10.21236/ada406704.\u003c/li\u003e\n \u003cli\u003eWong MC, Ng BK, Kennedy SF, Hwaung P, Liu EY, Kelly NN, et al. Children and adolescents\u0026rsquo; anthropometrics body composition from 3‐D optical surface scans. Obesity 2019;27:1738\u0026ndash;49. doi:10.1002/oby.22637.\u003c/li\u003e\n \u003cli\u003eBogo F, Romero J, Pons-Moll G, Black MJ. Dynamic FAUST: Registering human bodies in Motion. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017. doi:10.1109/cvpr.2017.591.\u003c/li\u003e\n \u003cli\u003eM. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, \u0026ldquo;SMPL,\u0026rdquo; ACM Transactions on Graphics, vol. 34, no. 6, pp. 1\u0026ndash;16, 2015.\u003c/li\u003e\n \u003cli\u003eAllen B, Curless B, Popović Z. The space of human body shapes. ACM Transactions on Graphics 2003;22:587\u0026ndash;94. doi:10.1145/882262.882311.\u003c/li\u003e\n \u003cli\u003eHahner S, Garcke J. Mesh convolutional Autoencoder for semi-regular meshes of different sizes. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2022. doi:10.1109/wacv51458.2022.00240.\u003c/li\u003e\n \u003cli\u003eWang Q, Lu Y, Zhang X, Hahn JK. A novel hybrid model for visceral adipose tissue prediction using shape descriptors. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2019. doi:10.1109/embc.2019.8857092.\u003c/li\u003e\n \u003cli\u003eSchulz E, Speekenbrink M, Krause A. A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions. Journal of Mathematical Psychology. 2018;85:1-16. doi:https://doi.org/10.1016/j.jmp.2018.03.001\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eGuo S. Implement A Gaussian Process From Scratch. Medium. Published January 26, 2021. Accessed May 10, 2023. https://towardsdatascience.com/implement-a-gaussian-process-from-scratch-2a074a470bce\u003c/li\u003e\n \u003cli\u003ePedregosa F, Varoquaux G, Gramfort A, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research. 2011;12(85):2825-2830.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eTian IY, Ng BK, Wong MC, Kennedy S, Hwaung P, Kelly N, et al. Predicting 3D body shape and body composition from conventional 2D photography. Medical Physics 2020;47:6232\u0026ndash;45. doi:10.1002/mp.14492.\u003c/li\u003e\n \u003cli\u003eGl\u0026uuml;er C-C, Blake G, Lu Y, Blunt1 BA, Jergas1 M, Genant1 HK. Accurate assessment of precision errors: How to measure the reproducibility of bone densitometry techniques. Osteoporosis International 1995;5:262\u0026ndash;70. doi:10.1007/bf01774016.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eWong MC, Ng BK, Tian I, Sobhiyeh S, Pagano I, Dechenaud M, et al. A pose‐independent method for accurate and precise body composition from 3D optical scans. Obesity 2021;29:1835\u0026ndash;47. doi:10.1002/oby.23256.\u003c/li\u003e\n \u003cli\u003eT. Alldieck, H. Xu, and C. Sminchisescu, \u0026ldquo;ImGHUM: Implicit generative models of 3D human shape and articulated pose,\u0026rdquo; 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021.\u003c/li\u003e\n \u003cli\u003eB. Jiang, J. Zhang, J. Cai, and J. Zheng, \u0026ldquo;Disentangled human body embedding based on deep hierarchical neural network,\u0026rdquo; IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 8, pp. 2560\u0026ndash;2575, 2020.\u003c/li\u003e\n \u003cli\u003eC. Lemeunier, F. Denis, G. Lavou\u0026eacute;, and F. Dupont, \u0026ldquo;Representation learning of 3D meshes using an Autoencoder in the spectral domain,\u0026rdquo; Computers \u0026amp;amp; Graphics, vol. 107, pp. 131\u0026ndash;143, 2022.\u003c/li\u003e\n \u003cli\u003eWong MC, Bennett JP, Leong LT, Tian IY, Liu YE, Kelly NN, et al. Monitoring body composition change for intervention studies with advancing 3D optical imaging technology in comparison to dual-energy X-ray absorptiometry. The American Journal of Clinical Nutrition,Volume 117, Issue 4, 2023, Pages 802-813, ISSN 0002-9165, https://doi.org/10.1016/j.ajcnut.2023.02.006.\u003c/li\u003e\n \u003cli\u003eGardner JR, Pleiss G, Weinberger KQ, Bindel D, Wilson A. GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. Neural Information Processing Systems. 2018;31:7576-7586.\u003c/li\u003e\n \u003cli\u003ePishchulin L, Wuhrer S, Helten T, Theobalt C, Schiele B. Building statistical shape spaces for 3D human modeling. Pattern Recognition 2017;67:276\u0026ndash;86. doi:10.1016/j.patcog.2017.02.018.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"3D Scanning, Body Composition, Machine Learning, Autoencoder, Gaussian Process","lastPublishedDoi":"10.21203/rs.3.rs-3935042/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3935042/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTotal and regional body composition are strongly correlated with metabolic syndrome and have been estimated non-invasively from 3D optical scans using linear parameterizations of body shape and linear regression models. Prior works produced accurate and precise predictions on many, but not all, body composition targets relative to the reference dual X-Ray absorptiometry (DXA) measurement.\u003cstrong\u003e \u003c/strong\u003eHere, we report the effects of replacing linear models with nonlinear parameterization and regression models on the precision and accuracy of body composition estimation in a novel application of deep 3D convolutional graph networks to human body composition modeling.\u003c/p\u003e\n\u003cp\u003eWe assembled an ensemble dataset of 4286 topologically standardized 3D optical scans from four different human body shape databases, DFAUST, CAESAR, Shape Up! Adults, and Shape Up! Kids and trained a parameterized shape model using a graph convolutional 3D autoencoder (3DAE) in lieu of linear PCA. We trained a nonlinear Gaussian process regression (GPR) on the 3DAE parameter space to predict body composition via correlations to paired DXA reference measurements from the Shape Up! scan subset. We tested our model on a set of 424 randomly withheld test meshes and compared the effects of nonlinear computation against prior linear models.\u003c/p\u003e\n\u003cp\u003eNonlinear GPR produced up to 20% reduction in prediction error and up to 30% increase in precision over linear regression for both sexes in 10 tested body composition variables. Deep shape features produced 6-8% reduction in prediction error over linear PCA features for males only and a 4-14% reduction in precision error for both sexes. Our best performing nonlinear model predicting body composition from deep features outperformed prior work using linear methods on all tested body composition prediction metrics in both precision and accuracy. All coefficients of determination (R\u003csup\u003e2\u003c/sup\u003e) for all predicted variables were above 0.86.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eWe show that GPR is a more precise and accurate method for modeling body composition mappings from body shape features than linear regression. Deep 3D features learned by a graph convolutional autoencoder only improved male body composition accuracy but improved precision in both sexes. Our work achieved lower estimation RMSEs than all previous work on 10 metrics of body composition.\u003c/p\u003e","manuscriptTitle":"3D Convolutional Deep Learning for Nonlinear Estimation of Body Composition from Whole-Body Morphology","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-13 19:42:14","doi":"10.21203/rs.3.rs-3935042/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"revise","date":"2024-11-07T05:48:00+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-11-06T22:01:32+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-10-31T17:12:47+00:00","index":3,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-10-28T18:01:27+00:00","index":2,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2024-08-23T17:46:40+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2024-08-14T12:58:48+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewersInvited","content":"","date":"2024-02-10T07:32:45+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-02-08T01:58:43+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-02-07T10:59:06+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Digital Medicine","date":"2024-02-06T22:17:10+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"41b8dc7d-19ce-45eb-8d13-f98fd65fb202","owner":[],"postedDate":"February 13th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":28693185,"name":"Health sciences/Health care/Medical imaging/Three-dimensional imaging"},{"id":28693186,"name":"Health sciences/Health care/Weight management"}],"tags":[],"updatedAt":"2025-02-03T08:06:29+00:00","versionOfRecord":{"articleIdentity":"rs-3935042","link":"https://doi.org/10.1038/s41746-025-01469-6","journal":{"identity":"npj-digital-medicine","isVorOnly":false,"title":"npj Digital Medicine"},"publishedOn":"2025-02-02 05:00:00","publishedOnDateReadable":"February 2nd, 2025"},"versionCreatedAt":"2024-02-13 19:42:14","video":"","vorDoi":"10.1038/s41746-025-01469-6","vorDoiUrl":"https://doi.org/10.1038/s41746-025-01469-6","workflowStages":[]},"version":"v1","identity":"rs-3935042","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3935042","identity":"rs-3935042","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}