{"paper_id":"152c50b7-8ed3-4651-a8b5-afd64006172d","body_text":"Keywords\ncutaneous leishmaniasis\nALL Metrics\n-\nViews\nDownloads\nHow to cite this article\nMohammed AB, Mohammed FS, Zewdu FT et al. An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.13869.1) NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.\nExport Citation\nSciwheel\nEndNote\nRef. Manager\nBibtex\nProCite\nSente\nSelect a format first\n▬\n✚\nResearch Article\n[version 1; peer review: 1 approved with reservations, 1 not approved]\nAmel Beshir Mohammed1,2, Fewzia Shikur Mohammed1,2, Feleke Tilahun Zewdu3,4, [...] Shimelis Doni Nigusse\nhttps://orcid.org/0000-0003-0455-2518\n1,2, Saba Lambert https://orcid.org/0000-0003-4376-5495\n1,5, Michael Marks https://orcid.org/0000-0002-7585-4743\n5-7, Stephen L. Walker5,7, Endalamaw Gadisa https://orcid.org/0000-0002-5179-6278\n8, SHARP CollaborationAmel Beshir Mohammed1,2, Fewzia Shikur Mohammed1,2, [...] Feleke Tilahun Zewdu3,4, Shimelis Doni Nigusse\nhttps://orcid.org/0000-0003-0455-2518\n1,2, Saba Lambert https://orcid.org/0000-0003-4376-5495\n1,5, Michael Marks https://orcid.org/0000-0002-7585-4743\n5-7, Stephen L. Walker5,7, Endalamaw Gadisa https://orcid.org/0000-0002-5179-6278\n8, SHARP Collaboration PUBLISHED 18 Feb 2025\nAuthor details Author details\n1 ALERT Hospital, Addis Ababa, Ethiopia\n2 Department of Dermatovenereology, College of Health Sciences, Addis Ababa University, Addis Ababa, Addis Ababa, Ethiopia\n3 Boru Meda Hospital, Boru Meda, Ethiopia\n4 Department of Epidemiology and Biostatistics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia\n5 Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK\n6 Division of Infection and Immunity, University College London, London, UK\n7 Hospital for Tropical Diseases, University College London Hospitals NHS Foundation Trust, London, UK\n8 Department of Malaria and Neglected Tropical Disease Research, Armauer Hansen Research Institute, Addis Ababa, Ethiopia\n2 Department of Dermatovenereology, College of Health Sciences, Addis Ababa University, Addis Ababa, Addis Ababa, Ethiopia\n3 Boru Meda Hospital, Boru Meda, Ethiopia\n4 Department of Epidemiology and Biostatistics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia\n5 Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK\n6 Division of Infection and Immunity, University College London, London, UK\n7 Hospital for Tropical Diseases, University College London Hospitals NHS Foundation Trust, London, UK\n8 Department of Malaria and Neglected Tropical Disease Research, Armauer Hansen Research Institute, Addis Ababa, Ethiopia\nAmel Beshir Mohammed\nRoles: Methodology, Writing – Review & Editing\nRoles: Methodology, Writing – Review & Editing\nFewzia Shikur Mohammed\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nFeleke Tilahun Zewdu\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nShimelis Doni Nigusse\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nSaba Lambert\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Writing – Review & Editing\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Writing – Review & Editing\nMichael Marks\nRoles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing\nRoles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing\nStephen L. Walker\nRoles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation\nRoles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation\nEndalamaw Gadisa\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Writing – Review & Editing\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Writing – Review & Editing\nOPEN PEER REVIEW\nREVIEWER STATUS\nCutaneous leishmaniasis (CL) remains a major public health challenge, especially in endemic regions like Ethiopia, where an estimated 40,000 new cases occur annually. Effective treatment evaluation for CL relies on consistent clinical assessments, yet variability in lesion descriptions can complicate reliable outcome measures.\nWe conducted an inter-reliability study of clinicians’ evaluations of CL lesion morphology and size at ALERT Hospital, Addis Ababa. Twelve clinicians independently examined 12 patients with parasitologically confirmed CL, each clinician assessing lesion morphology, size, and severity.\nWe found high consistency in reporting major morphological categories (e.g., plaques) but significant variability in secondary features like dyspigmentation and scale, as well as mucosal involvement. Lesion size measurements showed limited variability, suggesting its reliability as a potential measure for future clinical trials. Disparities in severity assessments highlight the need for a standardized scoring system in CL.\nOur findings underscore the importance of training for consistent, high-quality clinical evaluations of CL and suggests that lesion size could be a reproducible outcome measure in treatment efficacy trials.\nCutaneous leishmaniasis is a significant public health issue in Ethiopia, with around 40,000 new cases each year. The disease can cause a wide variety of different types of skin lesions which can complicate diagnosis and assessment. Conducting reliable treatment evaluation requires consistent assessment of cutaneous leishmaniasis lesions, especially in clinical trials.\nWe conducted a prospective study at ALERT Hospital in Addis Ababa to assess how consistently 12 clinicians, who were all experienced in managing patients with cutaneous leishmaniasis, evaluated the appearance, size and severity of cutaneous leishmaniasis lesions in 12 patients.\nWe found that whilst clinicians were broadly consistent in identifying the major features such as the type of lesions they showed more significant variability in describing secondary features and severity. Lesion size measurements were more consistent, making them a promising tool for future clinical trials. The findings highlight the need for better training and standardized scoring systems to improve the reliability of clinical evaluations in cutaneous leishmaniasis.\ncutaneous leishmaniasis\nCorresponding Author(s)\nMichael Marks (Michael.Marks@lshtm.ac.uk)\nGrant information: This project is funded by the National Institute for Health and Care Research (NIHR) under its [‘Research and Innovation for Global Health Transformation programme (Grant Reference Number NIHR200125)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nCopyright: © 2025 Mohammed AB et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Mohammed AB, Mohammed FS, Zewdu FT et al. An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.13869.1) First published: 18 Feb 2025, 5:12 (https://doi.org/10.3310/nihropenres.13869.1) Latest published: 14 May 2025, 5:12 (https://doi.org/10.3310/nihropenres.13869.2) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nThere is a newer version\nof this article available.\nof this article available.\nCutaneous leishmaniasis (CL) is the most common form of leishmaniasis. The disease causes skin lesions on exposed parts of the body at the site of an infected sandfly bite. With or without treatment CL lesions can lead to life-long scars and result in serious disability or stigma. It is estimated that 600 000 to 1 million new cases occur worldwide annually with 40,000 new cases of CL in Ethiopia1. Three Leishmania species have been reported in Ethiopia, but the vast majority of cases are thought to be due to L aethiopica.\nL. aethiopica is associated with a variety of clinical phenotypes. The most frequent presentation is localized CL (LCL), but infection is also associated with mucocutaneous leishmaniasis (MCL), and diffuse CL (DCL). LCL usually starts as a papule that gradually enlarges forming nodules or plaques. Ulceration may occur with L. aethiopica. MCL can occur either via the bite of the sand fly at the border of mucosal surfaces or through extension from an adjacent cutaneous lesion. It most commonly involves the nasal mucosa but the lips and oral mucosa can also be involved. DCL is a chronic and progressive disorder which starts with papular or nodular lesions which become larger and more numerous2.\nDespite the large number of affected individuals, studies of the clinical manifestations of L. aethiopica, including the morphology of the lesions, progression of the disease and response to therapy, are limited. A study in North-west Ethiopia reported the most common features to be induration, erythema, ulceration, and crusting.\nThe WHO strategic framework for integrated control and management of “skin-related” NTDs highlighted anti-microbial therapy for CL as a research gap and this is particularly the case for treatment of L. aethiopica infection3,4. Robust outcome measures are required for treatment efficacy studies. Outcome measures in CL in studies are based on clinical evaluation of individual lesions including re-epithelization in ulcerated lesions and flattening in non-ulcerated ones and lesion measurement. In 2018, a proposal for harmonized clinical trial methodologies for cutaneous leishmaniasis was published to develop a consensus and standardize clinical evaluation as a measure of disease response and cure in CL5.\nGiven the reliance on clinical observations of lesion morphology and severity it is therefore essential to evaluate whether the clinical evaluation of individual physicians is comparable. We therefore conducted a study to assess the inter-rater reliability of clinicians’ assessment of lesion morphology and size in individuals affected by CL in Ethiopia.\nNeither patients nor the public were directly involved in the design or analysis of the study.\nThe study was conducted nested within a larger cohort study of CL in Ethiopia. As part of this larger study, we conducted an inter-observer exercise at ALERT comprehensive specialized Hospital, Addis Ababa6.\nExaminers were selected from the clinical staff of ALERT Hospital and members of the Skin Health Africa Research Programme (SHARP) team. We collected information on their previous training in skin disease and experience of managing CL.\nClinicians at ALERT Hospital identified 12 individuals with a parasitologically confirmed diagnosis of CL who were invited to participate in the study. A standardized proforma was used to capture routine clinical data about each affected individual. For individuals with more than one clinical lesion of CL, the treating clinician indicated the “index” lesion to be assessed in the inter-observer assessment exercise. This “index” lesion was the lesion identified by the affected individual as the one of greatest concern prior to the start of the exercise.\nA computer generated latin-square was used to randomize an order of evaluation for each observer and affected individual. Each observer was assigned a unique letter and each affected individual a unique number. To measure inter-observer agreement each observer independently examined all 12 individuals with CL once for a period of 5 minutes.\nExaminers recorded on a standard data collection sheet their assessment of each “index” lesion. Standardized definitions of CL developed for our cohort study (Box 1) were used6. For each lesion examiners were asked to record morphology, size and the presence or absence of clinical features including induration, ulceration, crust, hyper/hypopigmentation, scarring and mucosal involvement. Examiners measured the largest diameter of the lesion using a disposable tape measure. Examiners were asked to provide an assessment of the clinical phenotype of CL and to classify the disease severity as mild or moderate or severe. All data were double entered into an electronic study database for analysis.\nCL is diagnosed in a person with skin and/or mucosal lesions with evidence of Leishmania infection in the affected tissues characterised by the presence of Leishmania amastigotes on smear microscopy or growth of Leishmania promastigotes in culture or the detection of Leishmania DNA by polymerase chain reaction.\nLocalized CL: A confirmed case of leishmaniasis, with no mucosal involvement, characterized by ten or fewer cutaneous papules and/or nodules and/or plaques with or without ulceration involving one body site\nMulti-regional localized CL: A confirmed case of leishmaniasis, with no mucosal involvement, characterized by ten or fewer cutaneous papules and/or nodules and/or plaques with or without ulceration involving two or more body sites\nMucocutaneous leishmaniasis: A confirmed case of leishmaniasis characterized by ten or fewer papules and/or nodules and/or plaques with or without ulceration involving skin and an adjacent mucosal surface\nMucosal leishmaniasis: A confirmed case of leishmaniasis characterized by papules and/or nodules and/or plaques with or without ulceration involving exclusively a mucosal surface\nDiffuse CL: A confirmed case of leishmaniasis characterized by eleven or more papules and/or nodules and/or plaques with or without mucosal involvement.\nBody sites are classified as:\n1. Head and neck\n2. Torso - anterior (including genitalia)\n3. Torso - posterior (including buttocks)\n4. Right upper limb\n5. Left upper limb\n6. Right lower limb\n7. Left lower limb\nAs there is no published data on the degree of agreement in assessing cases of Cutaneous Leishmaniasis we used a convenience sample size based on ensuring we had a range of both examiners and patients with clinical differing clinical phenotypes. The primary planned analysis was descriptive. We report the proportion of affected individuals judged to have each feature by the examiners. For lesion size we describe variability by reporting the median and IQR lesion size reported for each index lesion. Findings are reported in line with the STROBE guidelines. Analysis was conducted in R version 4.2.1.\nTwelve affected individuals agreed to participate in the study. Seven were male and five female. The median age was 25 years. Based on the treating clinician’s assessment six had been diagnosed with LCL, four with MCL and two with mucosal CL. All twelve individuals were currently receiving inpatient treatment with intramuscular sodium stibogluconate.\nTwelve examiners took part in the study. Seven of the clinicians were male and 5 were female. The median age of examiners was 39. The majority (n = 10, 83%) were clinicians with specialist postgraduate training in dermatology and six (50%) were accredited dermatologists. (Table 1).\n| Median Age (years) | 39 |\n| Male | 7 |\n| Years experience | 8.3 (SD 6.8) |\n| Clinical specialty | |\n| Dermatologist | 6 |\n| Dermatology Trainee | 3 |\n| Infectious Diseases | 1 |\n| MSc in Dermatology | 1 |\n| GP with Dermatology postgraduate diploma | 1 |\nSome clinical features appeared to be more consistently reported by examiners than others (Table 2). For lesion morphology all examiners agreed on the presence of plaques in one individual, whilst 10 or more examiners agreed on the presence of plaques in five. In the remaining individuals with CL either eight or nine examiners assessed the lesion to be a plaque. All examiners agreed there were no nodular lesions in eight individuals. For the remaining four affected individuals one to four examiners felt there were nodular lesions. There was greater variability in reporting of the presence or absence of papules. For five individuals all examiners agreed papules were absent whereas for seven individuals between one to five examiners assessed they were present.\nInvolvement of the mucosa was reported by all examiners for three individuals and by 11 examiners in two others. For three individuals with CL all examiners agreed there was no mucosal involvement and for a further two individuals only one or two examiners assessed the mucosa to be affected. For two individuals there was more marked disagreement with seven and nine examiners reporting involvement and five and three examiners reporting no mucosal involvement.\nFor lesion features all examiners agreed on the presence of erythema in three individuals and at least 10 examiners agreed it was present in a further four. For the remaining individuals with CL there was marked variability with between three to eight examiners reporting erythema. For five individuals all examiners reported the presence of induration and for a further four at least 11 examiners felt induration was present. For the remaining three affected individuals nine examiners felt induration was present and three felt it was absent.\nFor six individuals with CL all examiners agreed that there was no evidence of ulceration. For the remaining six patients the number of examiners reporting ulceration varied from one to nine. For two individuals all 12 examiners agreed that there was evidence of crust and for two individuals all examiners agreed crust was absent. For the remaining eight individuals with CL the number of examiners reporting crust varied from one to eleven.\nThere were no individuals with CL whom examiners were in agreement on the presence or absence of scale but there were six where at least 10 examiners agreed scale was present and three where at least 10 examiners agreed it was absent.\nFinally, all examiners agreed there was no evidence of scarring in four individuals. For the remaining eight between one and eleven examiners deemed scarring to be present.\nEstimates of the reported maximum lesion diameter were broadly consistent across examiners but graphical visualization with a modified Bland-Altman plot suggests variation increased with the size of the lesion (Figure 1)7.\nThe majority of observers assessed that two third of patients had moderate to severe disease with the remaining affected individuals assessed as having mild-moderate disease. There were only two individuals with CL in whom there was a 2-grade discrepancy in the assessment of severity but there were no individuals in whom all examiners assigned the same severity score and only one in whom there was more than 80% agreement. Disagreement about mucosal involvement also meant that there were only four (33%) individuals with CL where all 12 examiners and the treating clinicians agreed on the clinical phenotype (Table 3).\nIn the first ever study evaluating the inter-observer reliability of clinical examination to assess lesion morphology in CL we demonstrated considerable variation in the reporting of several key clinical features. Reporting of overall lesion morphology as plaque, nodule or papule was generally highly consistent but the presence or absence of secondary clinical characteristics such as dyspigmentation or the presence of scale were more variable. Lesion size is proposed as a key characteristic for assessing treatment outcomes in future trials5. Reassuringly, we showed that there was limited variation in the assessed size of lesions when measured by multiple independent assessors, although there seemed to be evidence that variability was highest for larger CL lesions. Overall, these data suggest that lesion size assessment is likely to be a reliable measure in both observational and interventional prospective studies.\nWe found assessment of mucosal involvement to be variable. As this has important implications for treatment and characterization for clinical trials it may be advisable in some cases to obtain an assessment by an ear, nose and throat specialist to allow accurate characterization. There is not yet a standardized definition of severity for CL6. In the absence of such a system we found little consensus amongst clinicians when assessing individuals with CL which poses challenges in reliable reporting of their characteristics in both observational and interventional studies. Development of a validated severity score should be considered to help standardize practice.\nInter-observer variability is well recognized in clinical assessment particularly when assessing subjective features of disease. Similar to our findings, previous studies in other NTDs have shown that the degree of reliability may vary for different clinical features within a given disease. For example, diagnosis of scabies by healthcare workers has been shown to vary markedly depending on the severity of the infestation8,9. Similarly for leprosy studies have shown marked differences in inter-observer reliability in determining the presence or absence of nerve thickening or in the reliability of different methods of assessing sensory loss within skin lesions10.\nThe new WHO roadmap for NTDs focuses on strengthening responses to skin-NTDs3. Achieving this will require considerable investment in training of healthcare workers in the diagnosis and management of a range of skin diseases including CL. Given the degree of variability between expert examiners in this study our data suggest such efforts will require considerable effort to ensure high-quality, reliable and reproducible clinical diagnosis and assessment of clinical healing parameters.\nOur study has limitations. Firstly, all affected individuals were inpatients receiving care for CL at a referral centre. This is likely to have resulted in selection of individuals with more severe disease than might be seen elsewhere. Similarly, all individuals were receiving treatment and therefore their lesions might differ from untreated ones. Whilst this might affect lesion morphology, we believe it is unlikely to have affected our assessment of inter-examiner reliability. The generalizability of our findings to regions of the world where other Leishmania species are endemic is unclear, particularly in settings where ulcerated lesions may be more common.\nAssessment of inter-examiner reliability for multiple examiners is challenging. Conventionally kappa-scores are calculated to compare the performance of an examiner against a reference-standard. Multi-examiner weighted kappa scores have been developed to extend this approach to situations broadly analogous to our study design. However, both conventional and multi-examiner kappa scores have weaknesses11. Disagreements limited to a small number of individuals being examined can have an outsize impact on kappa scores even when there is near universal consensus on the majority individuals. We therefore opted to only present descriptive statistics. Finally, our study focuses on the assessment of individuals with known CL by experts. Further studies are needed to assess the performance of other cadres of healthcare workers in the initial evaluation of individuals with suspected CL.\nTrials of novel and improved therapeutics for CL are urgently needed. To provide comprehensive data on efficacy such trials will need to combine reliable and reproducible parasitological, clinical and patient reported outcome measures. Measurement of lesion size is likely to be a critical component of such measures, and our data are reassuring that this can be reliably measured by multiple independent assessors. More work is needed to identify and improve assessment of other clinical features in order to develop robust trial outcome measures for CL.\nEthical approval was obtained from AHRI/ALERT Ethics Review Committee (Ref: PO/23/21), the National Research Ethics Review Committee (Ref: 7/2-506/m259/35), and the London School of Hygiene and Tropical Medicine (Ref: 26421) in October 2021. Written individual informed consent was obtained from all participants (examiners and affected individuals) in the study. The study complied with the Declaration of Helsinki.\nLSHTM Data Compass: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia.\nhttps://doi.org/10.17037/DATA.0000449112\nThe project contains the following underlying data:\nLSHTM Data Compass - An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia.\nhttps://doi.org/10.17037/DATA.0000449112\nThe project contains the following supplementary data:\nCL_Interobserver_CRF – An ODK data collection form as used in the study\nCL_interobserver Codebook – A codebook describing the data coding\nInter-observer adult consent – a Copy of the patient information sheet and consent form\nData is available under CC by 4.0 license.\nWe wish to thank the individuals and communities for their participation in the work of the Skin Health Africa Research Programme.\nFaculty Opinions recommendedReferences\n- 1. Alvar J, Vélez ID, Bern C, et al.: Leishmaniasis worldwide and global estimates of its incidence. PLoS One. 2012; 7(5): 35671. PubMed Abstract | Publisher Full Text | Free Full Text\n- 2. Bryceson AD: Diffuse Cutaneous Leishmaniasis in Ethiopia. I. the clinical and histological features of the disease. Trans R Soc Trop Med Hyg. 1969; 63(6): 708–37. PubMed Abstract | Publisher Full Text\n- 3. Ending the neglect to attain the sustainable development goals: a strategic framework for integrated control and management of skin-related neglected tropical diseases. (accessed June 25, 2024). Reference Source\n- 4. van Griensven J, Gadisa E, Aseffa A, et al.: Treatment of Cutaneous Leishmaniasis caused by Leishmania aethiopica: a systematic review. PLoS Negl Trop Dis. 2016; 10(3): e0004495. PubMed Abstract | Publisher Full Text | Free Full Text\n- 5. Olliaro P, Grogl M, Boni M, et al.: Harmonized clinical trial methodologies for localized Cutaneous Leishmaniasis and potential for extensive network with capacities for clinical evaluation. PLoS Negl Trop Dis. 2018; 12(1): e0006141. PubMed Abstract | Publisher Full Text | Free Full Text\n- 6. Mohammed AB, Mohammed FS, Zewdu FT, et al.: Protocol for a prospective observational cohort study of Cutaneous Leishmaniasis in Ethiopia [version 1; peer review: 2 approved with reservations]. NIHR Open Res. 2023; 3: 49. PubMed Abstract | Publisher Full Text | Free Full Text\n- 7. Christensen HS, Borgbjerg J, Børty L, et al.: On Jones et al.’s method for extending Bland-Altman plots to limits of agreement with the mean for multiple observers. BMC Med Res Methodol. 2020; 20(1): 304.Diffuse cutaneous leishmaniasis PubMed Abstract | Publisher Full Text | Free Full Text\n- 8. Walker SL, Collinson S, Timothy J, et al.: A community-based validation of the International Alliance for the Control of Scabies Consensus Criteria by expert and non-expert examiners in Liberia. PLoS Negl Trop Dis. 2020; 14(10): e0008717. PubMed Abstract | Publisher Full Text | Free Full Text\n- 9. Osti MH, Sokana O, Lake S, et al.: The body distribution of scabies skin lesions. JEADV Clinical Practice. 2022; 1(2): 111–121. Publisher Full Text\n- 10. Ponnighaus JM, Fine PE, Bliss L: Certainty levels in the diagnosis of leprosy. Int J Lepr Other Mycobact Dis. 1987; 55(3): 454–62. PubMed Abstract\n- 11. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977; 33(1): 159–74. PubMed Abstract | Publisher Full Text\n- 12. Marks M: Data for: \"An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in Cutaneous Leishmaniasis caused by Leishmania aethiopica in Ethiopia\". [Data Collection]. London School of Hygiene & Tropical Medicine, London, United Kingdom, 2025. http://www.doi.org/10.17037/DATA.00004491\nAuthor details Author details\n1 ALERT Hospital, Addis Ababa, Ethiopia\n2 Department of Dermatovenereology, College of Health Sciences, Addis Ababa University, Addis Ababa, Addis Ababa, Ethiopia\n3 Boru Meda Hospital, Boru Meda, Ethiopia\n4 Department of Epidemiology and Biostatistics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia\n5 Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK\n6 Division of Infection and Immunity, University College London, London, UK\n7 Hospital for Tropical Diseases, University College London Hospitals NHS Foundation Trust, London, UK\n8 Department of Malaria and Neglected Tropical Disease Research, Armauer Hansen Research Institute, Addis Ababa, Ethiopia\n2 Department of Dermatovenereology, College of Health Sciences, Addis Ababa University, Addis Ababa, Addis Ababa, Ethiopia\n3 Boru Meda Hospital, Boru Meda, Ethiopia\n4 Department of Epidemiology and Biostatistics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia\n5 Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, London, UK\n6 Division of Infection and Immunity, University College London, London, UK\n7 Hospital for Tropical Diseases, University College London Hospitals NHS Foundation Trust, London, UK\n8 Department of Malaria and Neglected Tropical Disease Research, Armauer Hansen Research Institute, Addis Ababa, Ethiopia\nAmel Beshir Mohammed\nRoles: Methodology, Writing – Review & Editing\nRoles: Methodology, Writing – Review & Editing\nFewzia Shikur Mohammed\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nFeleke Tilahun Zewdu\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nShimelis Doni Nigusse\nRoles: Investigation, Methodology, Writing – Review & Editing\nRoles: Investigation, Methodology, Writing – Review & Editing\nSaba Lambert\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Writing – Review & Editing\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Writing – Review & Editing\nMichael Marks\nRoles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing\nRoles: Conceptualization, Data Curation, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing\nStephen L. Walker\nRoles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation\nRoles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Writing – Original Draft Preparation\nEndalamaw Gadisa\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Writing – Review & Editing\nRoles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Writing – Review & Editing\nCompeting interests\nNo competing interests were disclosed.\nGrant information\nThis project is funded by the National Institute for Health and Care Research (NIHR) under its [‘Research and Innovation for Global Health Transformation programme (Grant Reference Number NIHR200125)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nThe funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.\nArticle Versions (2)\nCopyright\n© 2025 Mohammed AB et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.\nmetrics\nVIEWS\n$counts.viewCount\ndownloads\nCitations\nCITE\nhow to cite this article\nMohammed AB, Mohammed FS, Zewdu FT et al. An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.13869.1)\nNOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.\ntrack\nreceive updates on this article\nTrack an article to receive email alerts on any updates to this article.\nCurrent Reviewer Status: ?\nKey to Reviewer Statuses VIEW HIDE\nApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested\nApproved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.\nNot approvedFundamental flaws in the paper seriously undermine the findings and conclusions\nVersion 1\nVERSION 1\nPUBLISHED 18 Feb 2025 Views\n0\nHow to cite this report:\nGómez MA. Reviewer Report For: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.15071.r35115) The direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-35115\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-35115\nNOTE: it is important to ensure the information in square brackets after the title is included in this citation.\nReviewer Report 12 Apr 2025\nMaria Adelaida Gómez, Centro Internacional de Entrenamiento e Investigaciones Medicas, Cali, Colombia\nNot Approved\nVIEWS 0\nMajor comments:\nI confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above. Close\n- It is not clear at what time over the course of disease and treatment were the patients evaluated. In the discussion section one could infer patients were undergoing treatment, but it was not stated\nMajor comments:\nMinor comments\n- It is not clear at what time over the course of disease and treatment were the patients evaluated. In the discussion section one could infer patients were undergoing treatment, but it was not stated at what moment over the course of treatment were the patients evaluated. This leads me to question the relationship between the need (assessment of clinical cure) and the evaluation time point. Why was the study conducted on a single time point, instead of evaluating the disease progression over the course of treatment, for example?\n- For a study like this, images of lesions illustrating the characteristics that were more consistently identified vs. those that were not are necessary. This could also serve as a tool for reference to other clinicians experiencing similar difficulties in clinical assessment of CL (especially non-ulcerated lesions).\n- In the discussion the authors mention “There is not yet a standardized definition of severity for CL”. WHO and PAHO have generated guidelines that allow characterization of complicated vs. uncomplicated CL, based on number of lesions, lesion size, anatomical site of lesions, among others. This classification is the basis for selection of local vs. systemic treatments. This can be used as a baseline for definition of severity.\nMinor comments\n- I suggest caution with the statement of lesions developing at the site of sandfly bite, as this is not always the case (“The disease causes skin lesions on exposed parts of the body at the site of an infected sandfly bite”)\n- First paragraph of results section “mucosal CL”. In the box the classifications are either mucocutaneous or mucosal leishmaniasis. In this case, what are the authors referring to? MCL or CL? Also table 2 has “Pure mucosal”, please keep consistency in the use of terms.\n- What instrument was used for measurement of lesion size? This is relevant as the instrument used may introduce bias in the variability, or lack of, in this measurement among observers.\n-\nIs the work clearly and accurately presented and does it cite the current literature?\nYes\n-\nIs the study design appropriate and is the work technically sound?\nPartly\n-\nAre sufficient details of methods and analysis provided to allow replication by others?\nPartly\n-\nIf applicable, is the statistical analysis and its interpretation appropriate?\nPartly\n-\nAre all the source data underlying the results available to ensure full reproducibility?\nNo\n-\nAre the conclusions drawn adequately supported by the results?\nYes\nReferences\n1. Mohammed A, Mohammed F, Zewdu F, Nigusse S, et al.: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia. NIHR Open Research. 2025; 5. Publisher Full TextCompeting Interests: No competing interests were disclosed.\nReviewer Expertise: Human Cutaneous Leishmaniasis\nCITE\nHOW TO CITE THIS REPORT Gómez MA. Reviewer Report For: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.15071.r35115)\nThe direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-35115\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-35115\nNOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.\nViews\n0\nHow to cite this report:\nFuller LC and Wilder-Smith a. Reviewer Report For: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.15071.r34785) The direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-34785\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-34785\nNOTE: it is important to ensure the information in square brackets after the title is included in this citation.\nReviewer Report 26 Feb 2025\nannika Wilder-Smith, Faculty of Medicine, University of Zurich, Zurich, Switzerland\nApproved with Reservations\nVIEWS 0\nThis manuscript reports a study nested within a larger piece of work that is referenced. It describes the variability in clinical findings between 12 clinicians in assessing 12 known cutaneous leishmaniasis cases. The setting is a specialist hospital in Ethiopia ... Continue reading We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above. Close\nThis manuscript reports a study nested within a larger piece of work that is referenced. It describes the variability in clinical findings between 12 clinicians in assessing 12 known cutaneous leishmaniasis cases. The setting is a specialist hospital in Ethiopia with acknowledged experience in managing leishmaniasis and hosting the country's flagship dermatology training program. The overarching aim of the report is to highlight the need for\n1. standardised reporting structure for cutaneous leishmaniasis to support future studies developing or assessing new therapeutic approaches.\n2. Development of a standardised severity score and\n3. Development of standardised training for clinical assessors involved in future studies to ensure consistency.\nWhilst the training and exposure of the majority of the clinicians will be \"relevant\" it is not standardised and it might be worth emphasising in the discussion this fact and including a recommendation for ensuring any clinical assessors in cutaneous leishmaniasis studies have undertaken specific training on lesion analysis. It might be useful to state if any such training exists (eg on the Open WHO platform NTD channel - obviously under migration to WHO academy at the moment), is this fit for purpose ie generalisable across the globe, or is there an educational gap?\nI assume there is no scope for inclusion of clinical photographs to illustrate the cases and phenotype characteristics.\nSpecifically small comments:\nIntroduction: para 3 you mention a study in North west Ethiopia reporting common features - can you reference this?\nMethods:\nyou mention a standardised proforma - is it worth showing this?\nResults\nIs it worth stating that although the clinicians included were experienced you are not able to specifically comment on cutaneous leishmaniasis training or else report on how this is covered in the local Ethiopian Dermatology Curriculum?\nDiscussion\nWhilst noting the phenotypes and clinical features being measured, might you consider highlighting those most likely to be of clinical relevance in measuring treatment impact?\nMucosal involvement: whilst you might want to involve ENT in assisting the development of training tools to guide clinical assessors as to whether mucosal involvement is present or absent, it should be fairly straightforward to clarify with appropriate training.\nThe comments are minor and I hope helpful.\n1. standardised reporting structure for cutaneous leishmaniasis to support future studies developing or assessing new therapeutic approaches.\n2. Development of a standardised severity score and\n3. Development of standardised training for clinical assessors involved in future studies to ensure consistency.\nWhilst the training and exposure of the majority of the clinicians will be \"relevant\" it is not standardised and it might be worth emphasising in the discussion this fact and including a recommendation for ensuring any clinical assessors in cutaneous leishmaniasis studies have undertaken specific training on lesion analysis. It might be useful to state if any such training exists (eg on the Open WHO platform NTD channel - obviously under migration to WHO academy at the moment), is this fit for purpose ie generalisable across the globe, or is there an educational gap?\nI assume there is no scope for inclusion of clinical photographs to illustrate the cases and phenotype characteristics.\nSpecifically small comments:\nIntroduction: para 3 you mention a study in North west Ethiopia reporting common features - can you reference this?\nMethods:\nyou mention a standardised proforma - is it worth showing this?\nResults\nIs it worth stating that although the clinicians included were experienced you are not able to specifically comment on cutaneous leishmaniasis training or else report on how this is covered in the local Ethiopian Dermatology Curriculum?\nDiscussion\nWhilst noting the phenotypes and clinical features being measured, might you consider highlighting those most likely to be of clinical relevance in measuring treatment impact?\nMucosal involvement: whilst you might want to involve ENT in assisting the development of training tools to guide clinical assessors as to whether mucosal involvement is present or absent, it should be fairly straightforward to clarify with appropriate training.\nThe comments are minor and I hope helpful.\n-\nIs the work clearly and accurately presented and does it cite the current literature?\nYes\n-\nIs the study design appropriate and is the work technically sound?\nYes\n-\nAre sufficient details of methods and analysis provided to allow replication by others?\nPartly\n-\nIf applicable, is the statistical analysis and its interpretation appropriate?\nYes\n-\nAre all the source data underlying the results available to ensure full reproducibility?\nPartly\n-\nAre the conclusions drawn adequately supported by the results?\nYes\nCompeting Interests: No competing interests were disclosed.\nReviewer Expertise: I am a dermatologist with research experience in skin neglected tropical diseases.\nCITE\nHOW TO CITE THIS REPORT Fuller LC and Wilder-Smith a. Reviewer Report For: An assessment of interobserver agreement on lesion size, morphology and clinical phenotype in cutaneous leishmaniasis caused by Leishmania aethiopica in Ethiopia [version 1; peer review: 1 approved with reservations, 1 not approved]. NIHR Open Res 2025, 5:12 (https://doi.org/10.3310/nihropenres.15071.r34785)\nThe direct URL for this report is:\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-34785\nhttps://openresearch.nihr.ac.uk/articles/5-12/v1#referee-response-34785\nNOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.\nAlongside their report, reviewers assign a status to the article:\n- Approved\n- Approved with reservations\n- Not approved\n| Invited Reviewers | ||\n|---|---|---|\n| 1 | 2 | |\n| Version 2 (revision) 14 May 25 | read | read |\n| Version 1 18 Feb 25 | read | read |\nSign up for content alerts\nYou are now signed up to receive this alert\nAlongside their report, reviewers assign a status to the article:\nApproved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested\nApproved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.\nNot approved - fundamental flaws in the paper seriously undermine the findings and conclusions\nProvide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:\nExamples of 'Non-Financial Competing Interests'\n- Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper.\n- You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors.\n- You are a close professional associate of any of the authors (e.g. scientific mentor, recent student).\n- You work at the same institute as any of the authors.\n- You hope/expect to benefit (e.g. favour or employment) as a result of your submission.\n- You are an Editor for the journal in which the article is published.\nExamples of 'Financial Competing Interests'\n- You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements.\n- You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors.\n- You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on.\nSign up for content alerts and receive a weekly or monthly email with all newly published articles\nRegister with NIHR Open Research\nAlready registered? Sign in\nclose\nError\nIf you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.\nWe'll keep you updated on any major new updates to NIHR Open Research\nSign In\nIf you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.\nEmail us for further assistance.\nThe email address should be the one you originally registered with F1000.\nEmail address not valid, please try again\nYou registered with F1000 via Google, so we cannot reset your password.\nTo sign in, please click here.\nIf you still need help with your Google account password, please click here.\nYou registered with F1000 via Facebook, so we cannot reset your password.\nTo sign in, please click here.\nIf you still need help with your Facebook account password, please click here.\nCode not correct, please try again\nServer error, please try again.\nIf your email address is registered with us, we will email you instructions to reset your password.\nIf you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.\nPlease wait...","source_license":"CC-BY-4.0","license_restricted":false}