Questionable value of absolute mean gray value for clinical practice

I read with great interest the recent article by Guerriero et al.1. The authors of this study stated three objectives: (1) to explore ultrasound tissue characterization of deep infiltrating endometriosis (DIE) by using the mean gray value (MGV); (2) to evaluate whether MGV findings are dependent on the location of the nodule; (3) to assess the intra- and interobserver concordance of MGV quantification obtained using three-dimensional (3D) ultrasound. The researchers analyzed pre-stored 3D datasets (a single dataset per participant), assessing the MGV of 1-cm3 spherical samples placed in the interior of DIE lesions located in retrocervical and rectosigmoid regions. Additionally, two different observers estimated MGV in a subset of 24 volumes: each observer assessed each volume on two separate occasions, 1 week apart, to assess intra- and interobserver reproducibility. The authors observed that MGV values from spherical samples of the rectosigmoid region were significantly higher than were those from retrocervical locations. They also observed intraclass correlation coefficient (ICC) values > 0.95 when assessing the intra- and interobserver reliability, with limits of agreement of approximately ± 5 and ± 8, respectively. They concluded that absolute MGV seems to be able to identify different ultrasound tissue characteristics on the basis of location of nodules of deep endometriosis and that the reproducibility of MGV is extremely good. They also compared MGV with Hounsfield units, stating that, ‘From a conceptual point of view this [MGV] is quite similar to the Hounsfield unit (HU) values used in CT to analyze tissue properties and composition’. I have some concerns regarding this article. The most important is that the authors did not inform the readers that MGV is highly dependent on machine settings and attenuation, as occurs with the 3D power Doppler (3D-PD) indices (vascularization, flow and vascularization flow indices)2. Even though machine settings are liable to standardization, attenuation still represents a challenge, because it is not conceivable to ensure the same attenuation across examinations as it depends on both depth and tissue characteristics. Therefore, as for the 3D-PD indices3-5, it seems important to try to create a point of self-standardization: for example, by evaluating the ratio between the MGV of the region of interest and that of an adjacent known tissue. I performed the following small study to illustrate the effect of machine settings (particularly gain and tissue harmonic imaging (THI)) and attenuation on MGV. A gelatin phantom was scanned using a Voluson E8 ultrasound machine (GE Medical Systems, Zipf, Austria) equipped with a RIC5-9-D transvaginal probe. The gelatin phantom comprised a 75 × 75 × 75-mm cubic acrylic structure filled with a gelatin-agar mixture, consisting of a 2.73% solution of agar and 5.47% gelatin supplemented with glass beads (2.0%) to act as acoustic scattering sources, resulting in an attenuation coefficient of 0.5 dB/MHz/cm (similar to that observed in liver and average soft tissue). The probe was held in place by a mechanical stand and the phantom was supported on a stable work surface6. Between the transducer and the phantom, one of two attenuation blocks 1 cm in depth were inserted. These gelatin blocks had similar composition except for the concentration of glass beads: one had 2.0% glass beads, resulting in a coefficient of attenuation of 0.5 dB/MHz/cm (regular attenuation block); the other had 5.0% glass beads, resulting in a coefficient of attenuation of 1.0 dB/MHz/cm (high attenuation block). I acquired 10 static 3D datasets with both the regular and the high attenuation blocks, using gain = −10.0, 0.0 and +10.0, and with THI ‘on’ and ‘off’, i.e. 120 static 3D datasets in total. MGV was assessed from 2.0-cm3 spherical samples placed in the same position in the phantom, separated from the probe by the regular or high attenuation blocks (Figure 1). The results clearly showed that MGV is highly dependent on both machine settings and attenuation: MGV values were lower when using lower gain, when the high attenuation gelatin block was inserted and when using THI (Figure 2). Therefore, MGV should apparently not be compared with HU values, as the latter are only dependent on the tissue being examined and each tissue has a specific value, for example lung = −700 HU, fat = −84 HU, water = 0 HU, muscle = 0 HU and bone = 700–3000 HU7. Another concern about this article regards the reproducibility study. For this evaluation, Guerriero et al. used a single dataset, i.e. a single set of voxels, and so were able to assess only the variability involved in choosing the region of interest. This did not really reflect the variability that would be observed in clinical practice, since they did not account for some inevitable sources of variability during acquisition, such as the pressure applied by the person scanning, organ position and bowel movements8. In order to properly evaluate the intra- and interobserver reliability/agreement of ultrasound methods, completely independent scans should be performed9, 10. Consequently, their results are misleading as they bring a false overconfidence to the method. Examining the same dataset several times is likely to provide overestimated reliability/agreement, which can be misleading. For example, a similar problem occurred when studying the 3D-PD indices from spherical samples of the placenta2, 11: an initial study observed very high intra- and interobserver ICCs (> 0.95) when observers examined the same stored dataset and concluded that the method was highly reliable12; however, a study that tried to evaluate the true variability, by examining completely independent scans (acquired a few moments apart from each other) observed that the resulting ICCs were below the minimum standard (ICC < 0.70)13 needed even for research purposes. Regarding the Bland–Altman plots, Guerriero et al. believe that they ‘demonstrate very good agreement between MGV measurements performed by the observers’. However, this statement does not seem to be representative of the plots: in fact, the limits of agreement (interval expected to include 95% of the differences) were approximately ± 5 and ± 8 for intra- and interobserver differences, respectively. These findings should not be translated as being very good agreement because they show that the expected difference between two MGV measurements performed in the same lesion might be as high as 8 (even when only reassessing the same stable dataset), which is higher than the mean difference observed between rectosigmoid and retrocervical lesions (approximately 6). I therefore disagree with the authors' conclusion that their study shows that ‘MGV seems to be able to allow identification of different ultrasound tissue characteristics on the basis of location of the nodules of deep endometriosis and that the reproducibility of these measurements is extremely good’. In my opinion, using MGV in clinical practice for any purpose would be premature at the moment. Future studies should evaluate whether it is possible to use some form of internal self-standardization to reduce the influence on MGV of machine settings and attenuation. W. P. Martins Department of Obstetrics and Gynecology, Medical School of Ribeirao Preto, University of Sao Paulo (FMRP-USP), Av. Bandeirantes, 3900 – 8 andar - HCRP- Campus Universitario, Ribeirao Preto, Sao Paulo, Brazil; Ultrasonography and Retraining Medical School of Ribeirao Preto (EURP), Ribeirao Preto, Sao Paulo, Brazil (e-mail: [email protected])