Comparison of HSCORE assessment of endometrial beta3 integrin subunit expression with digital HSCORE using computerized image analysis (ImageJ).

One hundred slides, 50 positive cases (HSCORE > 0.7) and 50 negative cases (HSCORE ≤ 0.7), previously scored by an expert were used for these comparisons. A pilot study with 33 slides revealed that correlation between the whole tissue sample and 3 major representative areas of the same slide had a correlation coefficient of 97.9% (95% CI 95.7–98.9), which was highly significant (p < 0.0001), (data not shown). In addition, the average area of the sample analyzed in these 33 slides was 221 mm 2 . In this study the average of number of pictures analyzed per sample was 3. The intraobserver variability of naive researcher had a bias (the difference between the means) of 0.008 and a standard deviation (SD) of 0.3 (95% limits of agreement −0.7 to 0.7) using HSCORE. However the intervariability between naive and expert researcher yielded a bias of 0.9 and a SD = 0.8 (95% limits of agreement −0.7 to 2.5), as shown in Figures 2A and 2B . The inexperienced researcher assigned a higher HSCORE (all 6 negative slides were scored as positive, 1.05 or higher) compared to the expert. The correlation between D-HSCORE and HSCORE was highly significant (p < 0.0001) and had an r s = 0.86 (95% CI 0.79–0.90), as shown in Figure 2C . The ROC analysis yielded an area under the curve of 98.8% (95% CI 97–1), with a p < 0.0001 ( Figure 2D ). A cut-off of 37.2 has a sensitivity of 0.96 and a specificity of 0.98, yielding positive and negative likelihood ratios of 48 and 0.04, respectively. Comparison between findings of the expert using HSCORE and the naive using D-HSCORE are very similar and are presented in Figures 2E and 2F . In 50 cases that were called “negative” based on the cutoff of ≤ 0.7, 49 were determined to be negative by the naive reader using ImageJ. Likewise, in the 50 cases that were called “positive” based on the same cut-off, 48 were determined to be positive using ImageJ. Therefore, using the 37.2 cut-off in ImageJ, only 3 samples were discordant from the HSCORE. Intravariability and intervariability using ImageJ was 0% (95% CI 0–0.03) as expected since the same slides and ROI files were used (data not shown).

Immunohistochemical staining is a commonly used method for assessment of protein expression in situ but is highly subjective and prone to variation based on both experience and training and possibly due to unconscious subjectivity. This study was carried out to compare the semiquantitative HSCORE assessment with a truly quantitative assessment of staining results using ImageJ: D-HSCORE. The HSCORE has been reported to have a low intraobserver variation. 22 However, this result is based on correlation analysis instead of the Bland-Altman analysis, which is more appropriate. 30 Recent evaluation of HSCORE assessment of the β3 integrin subunit in repeat biopsies on the same woman demonstrated that HSCORE results tended to be reproducible with regards to being positive or negative. 31 This low intraobserver variability can be seen in Figure 2A : the naive researcher gave almost identical scores in both assessments. Nevertheless, compared to the experienced reader, the inexperienced researcher, using the traditional HSCORE methods, was unable to correctly assign samples 50% of the time due to an overestimate of background staining. It is noteworthy that the naive researcher gave to all samples an HSCORE ≥ 1.05, which is above the previously defined cut-off for a negative score. 10 The interobserver 95% CI limits of agreement reflect this wide variation, from −0.7 to 2.5. Thus, the naive researcher may be −0.7 below or 2.5 above the experienced researcher. This variation is more evident when the comparison of the mean values in both groups is examined. The naive researcher perceived much significantly higher median levels (1.7 [1.1–2.05]) as compared to the experienced researcher (0 [0–0.5]) (Mann-Whitney test p = 0.004, figures are given as (median [range]). As part of the comparison between traditional HSCORE and D-HSCORE, we were able to show that ImageJ on digitalized images from previously stained and scored sections gave highly comparable results to the experienced researcher. The correlation coefficient between HSCORE and D-HSCORE, r s = 0.86, is considered strong according to the literature. 29 In contrast to a low interobserver correlation when a naive reader used the HSCORE methods, the interobserver agreement was perfect using the D-HSCORE methodology. To our knowledge this is the first published report regarding the use of ImageJ to analyze endometrial β3 integrin subunit immunohistochemical results. Given its ease of use and reduced subjectivity, the cut-off presented here for positive and negative β3 integrin subunit staining will likely increase the consistency of results obtained among researchers. A cut-off of ≤ 0.7 was developed specifically for the use of HSCORE in predicting the presence of endometriosis 6 with a high specificity (91%) but lower sensitivity (38%). This reflects the fact that not all women with endometriosis lack integrin expression. As a result, this topic has become controversial since the usefulness of this biomarker for identification of endometriosis and infertility has not been replicated by all researchers. 6 As a bio-marker of endometrial receptivity, however, β3 has recently been shown to predict IVF failure and therefore may detect those women with endometriosis who have implantation deficits. 31 Methods to minimize variation and subjectivity will therefore be important going forward, including the use of image analysis software such as ImageJ. We report that the equivalent cut-off of < 0.7 in HSCORE is 37.2 using D-HSCORE, though further studies are needed to apply this cut-off to real-life situations such as endometriosis or fertility outcomes. One limitation of this methodology is that it has 10 steps to get the final analysis, while the traditional HSCORE assessment has just 1. Each slide must be digitalized and each gland should be manually identified. One possible use for this software might be for training purposes or as a quality control technique for immunohistochemistry results. This is an issue that needs further research. A separate limitation of this study is our choice of a gold standard. We used slides previously scored with traditional HSCORE by one individual instead of a separate technique such as western blot or real-time polymerase chain reaction from extracted glands. Ongoing studies using such techniques are currently being studied by our group. In conclusion, this study found that D-HSCORE performed by an inexperienced researcher was comparable to the HSCORE performed by an expert. Further, due to overestimation of DAB staining, none of the negative samples were correctly identified by the naive reader in this blinded comparison. However, that same individual, using ImageJ, was able to establish a high correlation with HSCORE to the experienced researcher. The methodology presented using ImageJ and color deconvolution may be used by anyone and is a more reliable and less subjective method for quantifying β3 integrin subunit in endometrial glands. These methods can be applied to any immunohistochemical result that uses DAB, without the need of expert training in the use of the subjective HSCORE methods.

One hundred sections of endometrium obtained from the laboratory archives of the Reproductive Endocrinology and Infertility unit from Greenville Hospital System, Greenville, South Carolina, were used for study analysis. Endometrial biopsies were obtained after informed consent. Each slide had been immunostained for the β3 integrin subunit as previously described 4 and evaluated by HSCORE by a single expert observer (B.A.L.) in a blinded fashion according to the formula proposed by Budwit-Novotny and colleagues. 22 Slides were selected to include 50 positive and 50 negative results, from a larger group of over 400 available samples. All endometrial biopsies were performed during the midluteal phase of the menstrual cycle 7–10 days after the luteinizing hormone surge using commercially available urinary luteining hormone surge prediction kits; no other criteria were used in the selection process. One of the researchers (B.A.L.) had previously assigned the HSCORE to the slides. The same slides were then subjected to β3 integrin subunit expression in a blinded fashion using the traditional HSCORE and the digital HSCORE (D-HSCORE) by a naive researcher (D.G.F., a 3rd year medical student). This researcher had previously received instructions on HSCORE for 2 hours by the experienced reader (B.A.L.). To assess the intraobserver variability of HSCORE, 12 slides (6 positives and 6 negatives) were reanalyzed by the naive researcher 1 week apart. Intervariability was performed comparing the HSCORE readings between the naive and the expert researcher. Each slide was digitalized using an Olympus BX51 microscope (Olympus Optical Co., Tokyo, Japan) connected to a digital color camera (Q-Color 5, Olympus). If necessary, multiple digital pictures were taken until the digitalization of the whole specimen was achieved. Images were obtained with a UPLFLN 20× objective (numerical aperture, 0.5; field number, 26.5) at a size of 2,560 × 1,920 pixels (resolution, 1 mm = 2,950 pixels) under standard lighting conditions and saved as a .tif image with 16 bits. Each slide was coded for ImageJ analysis. Slides were blindly analyzed with image analysis software (ImageJ 1.43j, National Institutes of Health, Bethesda, Maryland, U.S.A.). Initial settings of the software were applied to measure area (mm 2 ), median and mean intensity. Global scale of the image analysis was set as 2,950 pixels = 1 mm, in a pixel ratio of 1. The following 10 steps were used for image analysis. (1) An empty area from the slide was marked and analyzed with a color histogram for the RGB channels. The RGB values should be similar and close to 255. (2) If they were not similar, the image was adjusted by subtracting the background using the command “Process → subtract background” from the file menu. (3) Selection of the area (region of interest [ROI]), in this case, endometrial glands, was performed using the “Brush” tool, adjusting the size of the brush according to the thickness of the endometrial gland (e.g., 35 pixels) as shown in Figure 1A (scale bar = 100 μm). (4) The ROI file was then saved and (5) stored in a file (ROI Manager) in the computer hard disk. (6) The digitalized area was submitted to the plug-in “color deconvolution” using the built-in vector HDAB, where the staining of hematoxylin and diaminobenzidine (DAB) was separated into 3 different panels with hematoxylin ( Figure 1B ), with DAB only image ( Figure 1C ) and background. (7) Panel 2 was selected and (8) the ROI file was overlaid on the image ( Figure 1D ). (9) From this image the software calculated the area in mm 2 , the mean and the median intensity of DAB, ranging from 0 (black) to 255 (total white). (10) The final DAB intensity was calculated according to the formula f = 255 − i , where f = final DAB intensity, i = mean DAB intensity obtained from the software; i ranges from 0 (zero = deep brown, highest expression), to 255 (total white). When multiple pictures were taken from the same slide, the mean D-HSCORE was calculated on each. A maximal of 3 pictures from the main representative areas was used; e.g., slide #1 yielded 3 pictures. Each picture had their subtotal f = 54.2; 47.47; 50.13, in this case the final f = 50.63. All endometrial biopsies were obtained under an approved study consent submitted through the IRB at Greenville Hospital System. The use of these sections was approved by the ethics committee of Hospital de Clínicas de Porto Alegre under the number 11-0651. Sample size was calculated according to the literature 28 using the following parameters: an alpha error = 0.01, power = 0.95, and an estimated correlation coefficient ( r ) of 0.8. These figures yielded a sample size of at least 18 cases. The strength of the correlation coefficient was considered according to the literature 29 ; an r of 0 = no correlation, +0.2 = weak, +0.5 = moderate, +0.8 = strong, and 1 = perfect. Statistical analysis was performed using Spearman’s rank for correlation, Mann-Whitney test to compare the expression of β3 integrin subunit (final DAB intensity) in glandular endometrium, and a ROC curve to identify the cut-off between positive and negative in D-HSCORE. Intervariability and intravariability of methods using HSCORE was assessed with the Bland-Altman method. To assess the intraobserver and interobserver variability of D-HSCORE, analyses were performed using the same digital pictures and saved ROI files described above. D-HSCORE intraobserver variability was performed by one of the researchers (R.F.S.), and interobserver variability was performed between two researchers (D.G.F. and R.F.S.). GraphPad Prism version 5 for Macintosh (GraphPad Software, Inc., San Diego, California, U.S.A.) was used for statistical analysis.