Grading Sonographic Severity of Adenomyosis: A Pilot Study Assessing Feasibility and Interobserver Reliability

The baseline characteristics of the first 5 patients used for testing feasibility were not collected. The baseline characteristics and outpatient clinic sonographic assessment of the 30 women are listed in Table  2 . The main reason for the ultrasound examination was heavy menstrual bleeding or pelvic pain (chronic or dysmenorrhea). During the outpatient clinic visit, mainly diffuse adenomyosis was reported, especially at the anterior wall. Hyperechogenic islands, lines and buds, and irregular junctional zone were the most reported sonographic features of adenomyosis. Baseline Characteristics and Outpatient Clinic Sonographic Assessment NA, not assessed. Multiple answers in 1 patient are possible. Table  3 demonstrates the feasibility scores as discussed with the investigators. Manual contouring by VOCAL and XI VOCAL appeared to be an inadequate technique because of the diffuse character of adenomyosis. Counting affected slices in MSV with a set interslice thickness was very time consuming because the assessment had to be performed over 50 images, and a ratio was difficult to create due to the presence of blank images when the rotation angle was wider than the uterine contour. Additionally, it appeared to be difficult to interpret the images and assess the direct MUSA features in “VOCAL volume,” “XI VOCAL volume,” and “MPR estimation.” The 2D grayscale videos varied in quality (sweep was too fast or did not include the complete uterus), and manner (from left to right [sagittal], or from cervix to fundus [transverse], and with or without power Doppler). Feasibility Scores of the Different Methods Technique, time, and interpretation were scored on a 1–3 Likert scale (1 = “disagree,” 2 = “disagree nor agree,” 3 = “strongly agree”) and were weighed equally and cumulated to determine the overall feasibility. A cumulative feasibility score 6 was set as moderate to good overall feasibility. “XI VOCAL counting,” “MPR estimation,” and “2D‐clip estimation” had a total feasibility score >6 and were therefore selected for further analysis (see Table  3 ). The feasible methods, “XI VOCAL counting,” “MPR estimation,” and “2D‐clip estimation” were tested for interobserver reliability. “XI VOCAL counting” was performed in 10 and 20 transversal slices. The interobserver reliability of counting affected slices out of a total of 20 slices was better than the reliability using only 10 slices. The reliability in counting affected slices of 10 slices was moderate (ICC 0.625, 95% CI: 0.431–0.782). The reliability was best between 2 observers (JAFH and LMT). The interobserver reliability when counting the affected slices in a total of 20 slices was good (ICC 0.773, 95% CI: 0.630–0.875). The overall reliability after categorization between the 3 observers was good ( κ Fleiss  = 0.630, 95% CI: 0.471–0.789). “MPR estimation” was based on the 3 orthogonal planes and the rendered 3D image in MPR mode after seeing the video clip (n = 20), or only the MPR mode images when video clip was not available (n = 10). This resulted in a poor overall reliability ( κ Fleiss  = 0.341, 95% CI: 0.19–0.49). The lowest reliability was in category moderate ( κ Fleiss  = 0.139, 95% CI: −0.068 to 0.345), good reliability was seen in the severe category ( κ Fleiss  = 0.708, 95% CI: 0.501–0.915). Assessing the severity using 2D video clips by 3 observers in 20 patients and grading them as mild (50% affected) had an overall good reliability ( κ Fleiss  = 0.647, 95% CI: 0.468–0.827), with reliability increased toward the severe cases ( κ Fleiss  = 0.754, 95% CI: 0.444–0.977). Overall, of these methods, “XI VOCAL counting” (20) and “2D‐clip estimation” had the best interobserver reliability (Table  4 ). Interobserver Reliability of the Feasible Methods For interobserver reliability, the following values were used: kappa‐coefficient: ≤0.20 = slight reliability; 0.21–0.40 = poor reliability; 0.41–0.60 = moderate reliability; 0.61–0.80 = good reliability; and 0.81–1.00 = excellent reliability. 20 Intraclass correlation (ICC) value of less than 0.5 is set as poor reliability, between 0.5 and 0.75 as moderate, between 0.75 and 0.9 as good, and a value greater than 0.90 is excellent. The feasible and reproducible methods, “XI VOCAL counting,” “MPR estimation,” and “2D‐clip estimation,” were compared with real‐time MUSA estimation and the number of present MUSA features. The median “XI VOCAL counting” number was higher per real‐time estimated severity category (mild 9.7 [interquartile range (IQR) 5.0–11.8]; moderate 10.7 [IQR 4.7–12.3]; severe: 17.0 [IQR 9.3–17.7]). This resulted in a positive correlation as assessed with Spearman's rank order ( r s (27) = 0.607, 95% CI: 0.279–0.809). Categorizing “XI VOCAL counting” based on the affected slices: mild 0–6 slices, moderate 6–14 slices, severe 14–20 slices resulted in a moderate interest reliability with real‐time estimation ( κ w  = 0.520, CI 0.275–0.765). “XI VOCAL counting” categories and the number of MUSA features also showed a higher median number of features present per category (mild: 4.0 [IQR 2.5–5.5], moderate: 5.0 [IQR 3.25–5.75], and severe: 6.0 [4.5–8.0]). Spearman's correlation coefficient was conducted between the number of MUSA features and the “XI VOCAL counting” scores ( r s (27) = .529, 95% CI: 0.176–0.761). A total of 55% of the categories scores with “MPR estimation” and real‐time estimation were scored similarly, resulting in a moderate weighted reliability ( κ w  = 0.468, 95% CI: 0.226–0.711). The comparison with number of features shows increasing median number of features per severity category (mild: 4.0 [IQR 3.0–5.0], moderate: 5.0 [IQR 3.5–6.0], and severe: 6.0 [IQR 5.0–8.0]). A positive correlation was found between the severity categories and the number of features ( r s (27) = 0.523, 95% CI: 0.169–0.757). The categories scored with “2D‐clip estimation” and with real‐time estimation were 57.9% comparable and weighted reliability was moderate ( κ w  = 0.443, P  = .11, 95% CI: 0.124–0.762). The number of MUSA features per category scored with “2D clip estimation” were almost equally distributed across the different severity categories (mild: 4.0 [3.0–5.2], moderate: 4.5 [3.3–5.8], and severe: 5.0 [4.5–6.5]). The correlation of the methods was r s (27): 0.347, 95% CI: −0.141 to 7.00.

In this paper, we describe feasible, offline, semi‐quantified methods to grade the sonographic severity of adenomyosis. “XI VOCAL counting” scored best in terms of interobserver reliability and had the best correlation with the subjective MUSA severity categories and the number of MUSA features. This method appears to be the most objective and promising for future use. Since XI VOCAL uses standardized slices, this technique allows for a systematic evaluation of the entire uterus. By counting the affected slices, consistent and uniform reporting can also be achieved. Furthermore, there is potential for “XI VOCAL counting” to be integrated in ultrasound units as built‐in software making it readily available as a point‐of‐care test. It might be helpful to investigate the effect of uterine volume or interslice distance on the “XI VOCAL counting” score. This method might also be further enhanced by designing a deep learning tool or algorithm, which automatically identifies adenomyosis based on segmentation of volume sets. Prior studies have stressed the importance of a uniform classification and grading system of adenomyosis. 22 , 23 , 24 In 2018, Lazzeri et al designed an sonographic mapping system to assess the severity and extent of adenomyosis. 25 Based on the differentiation (focal or diffuse) and location of adenomyosis, 5 subtypes were distinguished and graded from 1 to 4 for each subtype, resulting in a total score for the extent of adenomyosis. Similarly, their method showed substantial to almost perfect interobserver reliability. However, in this study, severity assessment partly relied on the measurement of junctional zone thickness. It is worth noting that Harmsen et al recently elucidated the inconsistencies observed in junctional zone assessment across various diagnostic modalities, and thickness and visibility are influenced by factors such as menstrual cycle phase, peristalsis, and medication. Consequently, Harmsen et al strongly emphasized that adenomyosis should not be diagnosed with ultrasonography by measuring junctional zone thickness. 26 Other studies focused on the number of sonographic features suggestive of adenomyosis to classify degree of severity. 11 , 12 However, they did not consider the uterine volume affected with adenomyosis which may have resulted in misclassification. For example, a high number of features in a small portion of the uterus may have been classified as severe adenomyosis. In contrast, a low number of features extensively distributed throughout the uterus may have been classified as mild. This paradox was similarly observed in our study on “2D‐clip estimation” whereby an equal number of MUSA features was observed in all degrees of severity. Additionally, all features were assigned an equal weight, suggesting equal relevance. In “XI VOCAL counting,” we evaluated 3D uterine volumes and counted the number of slices with direct MUSA features. Therefore, while the clinical relevance of individual MUSA features has not been established, we focused on the, most likely clinically relevant, direct features and the affected volume of the uterus. In some histopathological classification systems, the extent of adenomyosis has been taken into account grading it as mild, moderate or severe, depending on the depth of myometrial invasion or number of foci per low‐power field. 22 These classification systems were made by inspecting a limited number hysterectomy specimen slices, and adenomyosis prevalence appeared to increase by increasing the number of examined slices. 27 This limitation is difficult to overcome and is expected to be similar in “XI VOCAL counting.” A retrospective cohort study in infertile patients with adenomyosis who underwent IVF used uterine volume to measure the severity of adenomyosis. 28 They found a lower cumulative live birth rate in larger uterine volumes, suggesting volume is important in grading severity. A greater uterine volume, not caused by myoma or multiparity, could be a reflection of myometrial hypertrophy due to advanced adenomyosis, 29 and “globular uterus” as an indirect MUSA feature of adenomyosis. 9 With other grading methods, 11 , 12 , 25 including “XI VOCAL counting,” the overall uterine volume was not considered. Uterine volume should be studied in future research correlating sonographic severity of adenomyosis with symptoms (eg, integrating the interslice distance of “XI VOCAL counting”). Although we included a random sample of 3D volumes and 2D clips without excluding suboptimal images, both “XI VOCAL counting” and “2D‐clip estimation” demonstrated good interobserver reliability. An interesting finding was the improved interobserver reliability for “MPR estimation” after investigators watched the 2D grayscale video clip. The observers reported that the video improved assessment of adenomyosis localization, deviations in uterine anatomy (myoma, niche, etc.), and overall severity. In addition, interobserver reliability improved with increasing severity of adenomyosis, likely due to coexisting indirect features. In our study, all examinations were made in a standardized way by inexperienced and experienced sonographers, with standardized consistent ultrasound machine settings. Another strength is that all 3 independent observers were blinded to the patients' clinical symptoms and outpatient clinic sonographic assessment. Furthermore, the evaluation of collected volumes was performed systematically using a predefined format. Several limitations need to be addressed. Since most women in the study visited the outpatient clinic with heavy menstrual bleeding and/or pain, there may have been a disproportionate number of severe cases of adenomyosis. It is therefore possible, that “XI VOCAL Counting” may not perform as well in cases with earlier stages or less severe adenomyosis. To reduce this risk of selection bias, we set our selection criteria on having at least one direct MUSA feature, which resulted in 30% of the patients being estimated as having mild adenomyosis. However, it is uncertain whether this actually reduced bias. Additionally, the external reliability of “XI VOCAL counting” may have been overestimated. Variables that may have impacted reliability include use of the same clips and volume sets (possibly resulting in less physiological variability due to same probe pressure and spontaneous myometrial peristalsis) and the observers were experts in the field of adenomyosis sonography who all trained at the same medical center. 30 Finally, it is important to highlight that no reference standards exist and that we compared our methods with the currently used methods, which are not validated. Therefore, our analyses regarding these comparisons should be interpreted with caution. It only shows trends and did not evaluate diagnostic performance. Additionally, this was a small pilot study a larger prospective cohort study is required to confirm our results and evaluate for diagnostic performance and clinical relevance. Considerations for implementation included the added costs of training and necessary equipment (3D probe, 5D viewer™). Furthermore, while “XI VOCAL counting” is offline software that allows for expert consultation, integration in the ultrasound machine will be necessary for point‐of‐care application in the outpatient clinic setting.

Adenomyosis is a prevalent benign uterine disease that may significantly affect patient quality of life and reproductive potential. However, data on the relationship between adenomyosis and symptoms or subfertility is scarce and contradictory. Differentiating between severity categories of adenomyosis may prove important. To our knowledge, this is the first pilot study evaluating semi‐quantifiable methods to assess sonographic severity of adenomyosis. “XI VOCAL counting” appears to be most objective and promising for future use. This method allows for the analysis of 3D‐volume datasets to assess the total affected myometrium. “XI VOCAL counting” has the potential to more adequately diagnose patients with adenomyosis and could become available as a point‐of‐care test in clinical practice. In this study, “XI VOCAL counting” appeared to be feasible and demonstrated good interobserver reliability. Future research regarding “XI VOCAL counting” may be of interest to assist in the development of more individualized prognostic targets and eventually lead to personalized therapies.

We performed a cross‐sectional pilot study in the gynecological outpatient clinic of the Amsterdam University Medical Center. The ethical board approved the study (2021.0035) and it is prospectively registered ( NCT06117410 ). From January 2021 until June 2022, we recruited 35 consecutive premenopausal women with sonographic signs of adenomyosis visiting our clinic for various gynecological reasons (see Table  1 ), after informed consent was obtained. Adenomyosis was diagnosed when at least one direct MUSA feature was present at 2D TVUS. Exclusion criteria were the use of any hormonal medication (eg, oral contraceptives, levonorgestrel‐releasing intrauterine device, and Gonadotropin‐releasing hormone, dominant myoma(s) (defined as diameter greater than 5 cm, or more than 3 myomas), suspicion of uterine or cervical malignancy and pregnancy. Description of the Six Offline Sonographic Methods and Their Usage for Assessing Severity of Adenomyosis Six different gynecologists, with a special interest in benign gynecology and ultrasound, examined the patients at the outpatient clinic. This outpatient visit included a structured medical and obstetric history, medication and gynecological symptoms, and a physical and sonographic examination. The transvaginal sonographic examination was performed with either a Samsung WS80A ultrasound machine using the V5‐9 intracavity transducer or with a HeraI10 ultrasound machine using the EV2‐10A intracavity transducer (all Samsung, Seoul, South Korea). The settings of the HeraI10 in grayscale were as follows: Gain 86, Dynamic Range 50, Frequency 34–46 Hz. The WS80A settings were as follows: gain 44–71, dynamic range 100–123, and frequency either 4.7 or 5.6 MHz. In 3D TVUS, a 90° angle in Z ‐direction was used. Transvaginal sonographic 2D‐ and 3D examination was performed in a standardized manner. The following sonographic findings were reported: uterine size (length, height, width), the presence of intracavitary abnormalities, myoma(s), signs of endometriosis, and assessment of adenomyosis as described by MUSA (Supplemental Figures  1 and 2 ). 9 , 10 Patient data, 2D images, video clips, and 3D volumes were pseudonymized and stored. This data was consecutively evaluated with potential offline sonographic methods, and tested for feasibility to assess severity, interobserver reliability, and compared with current subjective methods as shown in the flowchart (Figure  1 ). The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions. Study flowchart. Experienced gynecological sonographers (JAFH, RADL, TVdB, and LMT) proposed potential methods for evaluating the collected video clips and 3D volumes based on literature and their experience. The stored 3D volumes and video clips were evaluated offline, using Samsung Medison 5D Viewer™. This software program enables offline analysis of ultrasound datasets with different advanced tools. Offline assessment is used to describe the blinded assessment of the transvaginal acquired uterine sonographic images, video clips and volumes by the observers, in comparison to real‐time subjective evaluation of severity. Measuring volumes of structures or organs has been extensively researched and multiple offline sonographic tools and methods have been reported. The virtual organ computer‐aided analysis (VOCAL) and eXtended Imaging VOCAL (XI VOCAL) methods are found accurate in measuring myoma 14 and fetal volume, 15 respectively. Furthermore, multiplanar and 3D rendering (MPR) software is used in antral follicle count 16 and evaluating the junctional zone in the diagnosis of adenomyosis. 6 Finally, multislice view (MSV) is used in the diagnosis of fetal anomalies 17 and obstetric levator ani muscle injuries. 18 , 19 These methods were adjusted for their use in the assessment of adenomyosis, resulting in 5 new methods to estimate the severity of adenomyosis. The currently existing real‐time MUSA method of estimating the extent of adenomyosis was also added to the offline evaluation of grayscale video clip. These 6 methods were used for the feasibility assessment to estimate severity of adenomyosis. A detailed explanation of the methods and their usage in this study is found in Table  1 and Figures  2 and 3 . Schematic presentation of methods. ( A ) “VOCAL volume,” severity presented in percentage affected myometrium (%); ( B ) “XI VOCAL volume,” severity presented in percentage (%); ( C ) “MSV counting,” severity presented in ratio affected slices of total slices with a set interslice distance; ( D ) “XI VOCAL counting,” severity presented in ratio affected slices of 10 or 20 slices; ( E ) “MPR estimation,” severity presented in category based on percentage affected myometrium; ( F ) “2D‐clip estimation” severity presented in category based on percentage affected myometrium. ( A – E ) Methods 1–5 visualize volumes evaluated with 5D Medical Viewer™; ( F ) Method 6 displays a grayscale 2D‐clip. Image created with Biorender. Realistic representation of the methods for evaluating severity. All images are made transvaginally. Numbers 1–6 represent the 6 methods. 1: “VOCAL volume”; 2: “XI VOCAL volume”; 3: “MSV counting”; 4: “XI VOCAL counting”; 5: “MPR estimation”; 6: “2D‐clip estimation”; 1–5: Methods 1–5 visualize 3D volumes evaluated with 5D Medical Viewer™; 6: Method 6 displays a grayscale 2D‐clip. Image created with Biorender. A feasibility assessment was performed on the first 5 patients by 1 sonographer (LMT) and the pros and cons regarding feasibility of usage were discussed with the other investigators (JAFH, TVdB, RADL) to select the methods for further evaluation. Feasibility of the severity evaluation was scored in terms of technique, time, and interpretation. The technique of the proposed method was feasible when it was easy to use and intuitive. The method was assessed time‐efficient if the method enabled the interpreter to evaluate the volume in less than 5 minutes per patient. The interpretation was accurate if you could use direct features of adenomyosis in the assessment. Technique, time, and interpretation were scored on a 1–3 Likert scale (1 = “disagree,” 2 = “disagree nor agree,” 3 = “strongly agree”) and were weighed equally. These statements were analyzed to determine the overall feasibility of the method, a total score >6 was set as moderate to good overall feasibility. The feasible methods (“XI VOCAL counting,” “MPR‐estimation,” and “2D‐clip estimation”) were consecutively tested in 30 patients for interobserver reliability. Three independent examiners evaluated the collected pseudonymized 2D videos and 3D‐volume datasets. The 3 examiners (JAFH, RADL, and LMT) were experienced in adenomyosis ultrasound and 3D volumes (20, 10, and 3 years, respectively). The examiners were blinded to the clinical data and severity assessment made by the gynecologist during the outpatient clinic visit. Finally, we compared the 3 feasible methods (“XI VOCAL counting,” “MPR‐estimation,” and “2D‐clip estimation”) with 2 currently used methods: 1) subjective real‐time classification; and 2) number of MUSA features present. The subjective classification was categorized as mild, moderate, or severe, based on the estimated percentage of affected myometrium (50%, respectively). Both methods were reported during the outpatient clinic by the gynecologist. All analyses were carried out using IBM SPSS Statistics Software, version 28. No power calculation was done for the analysis in this paper, as this is a pilot study. The interobserver reliability was assessed for the feasible methods. A Fleiss kappa ( κ Fleiss ) was used in categorical data, with a 95% confidence interval (CI). General rules for the interpretation of the kappa‐coefficient were used: ≤ 0.20 = very poor, 0.21–0.40 = poor, 0.41–0.60 = moderate, 0.61–0.80 = good, and 0.81–1.00 = very good. 20 Intraclass correlation coefficient (ICC) was used in continuous data. This was measured with the 2‐way mixed‐effects model, and based on single rater and consistency definition. An ICC value of less than 0.5 was considered poor reliability, between 0.5 and 0.75 moderate, between 0.75 and 0.9 good, and a value greater than 0.90 was excellent. 21 To test the correlation between the new methods and the old methods a Spearman's rank‐order correlation ( r s ) was calculated. For “XI VOCAL counting,” new severity scores were categorized as mild, moderate, or severe, based on their range and the best interobserver reliability. Intertest reliability with the subjective MUSA mild–moderate–severe categories was tested with Cohen's weighted kappa ( κ w ).

Supplemental Figure 1 Reporting guideline for sonographic features of adenomyosis. Figure from Van den Bosch et al 2019. 9 Supplemental Figure 2 Schematic representation of direct and indirect Morphological Uterus Sonographic Assessment (MUSA) features of adenomyosis. Figure from Harmsen et al 2022. 10