{"paper_id":"0fea137f-48ff-4c1f-b00f-6a895b08c611","body_text":"1 \nUsing Glittr.org to find, compare and 1 \nre-use online training materials 2 \nUsing Glittr.org to find, compare and re-use online training materials 3 \n 4 \nGeert van Geest 1,2*, Yann Haefliger 1, Monique Zahn-Zabal 1, Patricia M. Palagi 1 5 \n 6 \n1Swiss Institute of Bioinformatics, Quartier Sorge - Bâtiment Amphipôle, 1015 Lausanne, 7 \nSwitzerland. 8 \n2Interfaculty Bioinformatics Unit, University of Bern, Baltzerstrasse 6, 3012, Bern, Switzerland. 9 \n*Corresponding author. E-mail: geert.vangeest@sib.swiss 10 \n  11 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 2 \nAbstract 12 \nA wealth of excellent training materials on computational life sciences is scattered around the 13 \ninternet, but they can be hard to find. Many materials reside in public git repositories that are 14 \nhosted on platforms such as GitHub and GitLab. Glittr.org aggregates and indexes such git 15 \nrepositories enabling users to find training materials that would otherwise be hard to identify. 16 \nWith the application, users can search and compare training materials based on topic and 17 \nauthor, but also on engagement metrics like stargazers (bookmarks) and recency (days since 18 \nlast commit). Glittr.org currently contains 568 entries, which are assigned to six different 19 \ncategories within the field of computational life sciences. By analysing the database, we reveal 20 \ninsights in the availability of materials per topic, collaboration patterns of developers, and 21 \nlicensing practices. This knowledge helps to understand in which areas open training materials 22 \nare lacking, the importance of git for collaboration on training materials and how licensing can 23 \nbe improved to enhance sharing and reuse. Taken together, we show that Glittr.org contains a 24 \nwealth of connected and openly available metadata. Therefore, it enhances adherence to the 25 \nFAIR (Findable, Accessible, Interoperable, Reusable) principles, which benefits the entire life 26 \nsciences community. 27 \nIntroduction 28 \nThe life sciences domain is constantly growing and evolving, requiring professionals to be 29 \ntrained to stay up to date with the most relevant skills and knowledge. To facilitate this, various 30 \norganizations deliver courses on a large range of topics in many different formats. Many course 31 \nproviders dedicate themselves to actively share their materials with the public, resulting in a 32 \ngrowing wealth of public training materials available across the web. This greatly enhances the 33 \npossibility to re-use these training materials, which is invaluable for both learners and lesson 34 \ndevelopers; learners can use the material on their own time to obtain required skills, while 35 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 3 \nlesson developers can use the existing material to build their courses further upon existing 36 \nones.   37 \nTo help developers make their shared material easier to find and re-use, the FAIR principles [1–38 \n5] provide guidelines on organizing their materials and metadata. The way training material 39 \nmetadata are represented depends on the platform where they are hosted. Currently, training 40 \nmaterials are hosted in diverse locations, like organizational websites, cloud storage, learning 41 \nmanagement systems, and Git platforms (e.g., GitHub, GitLab). Therefore, metadata 42 \nrepresentation often does not conform to standards, making it difficult to parse and utilize it 43 \neffectively. This issue can be relieved by aggregation and indexation, which encourages 44 \nmetadata standardization and therefore enhances findability, re-usability, and integration of 45 \nmetadata over multiple platforms.   46 \nRegistries that currently aggregate and index training materials in the life sciences include the 47 \nOpen Educational Resources (OER) Commons [6], the training portal of the Global Organisation 48 \nfor Bioinformatics Learning, Education and Training (GOBLET) [7], and ELIXIR’s online training 49 \nregistry Training eSupport System (TeSS) [8]. These registries can accommodate any digital 50 \nobject related to education that are accessible through an URL, making them potentially 51 \ncomprehensive. However, this flexibility also comes with disadvantages. Since not every 52 \nwebpage contains standardized metadata, it requires investments from the training material 53 \ndeveloper to manually enter metadata at the registry or to automate metadata provision, for 54 \nexample by implementing a standardized metadata markup like Bioschemas [9,10].  55 \nUsing public git repositories for developing and hosting training materials is gaining popularity, 56 \nespecially for materials related to (bio)informatics and data science. It is generally 57 \nrecommended for reproducibility [11] and adopted by many institutions, communities, and 58 \nindividuals (e.g. the Carpentries [12], Galaxy training network [13], the National Bioinformatics 59 \nInfrastructure Sweden (NBIS) and the SIB Swiss Institute of Bioinformatics. Having training 60 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 4 \nmaterials in public Git repositories has major advantages. First, using Git encourages 61 \ncommunity-driven development. Git is version-control software, and as such, can be used to 62 \ntrack changes in files and associate these changes to individuals with distinct levels of 63 \npermissions. This makes it ideal for co-development and re-use of materials. Using git drives 64 \ncommunity engagement by supporting direct feedback, suggestions, and corrections. 65 \nSecondly, the infrastructure around the most popular platform for public git repositories, 66 \nGitHub (https://github.com)  and GitLab (https://about.gitlab.com), are standardized and 67 \ncommonly used. Many people have the know-how to use version control, so the threshold to 68 \nuse or learn how to use git for training material is low – especially in communities related to the 69 \ninformatics field. Thirdly, public Git repositories offer standardized retrieval of a wealth of 70 \nmetadata. Authorship, activity logs, and popularity metrics are all recorded and made 71 \naccessible through a single point of entry. 72 \nDespite many advantages, it remains difficult to find training materials on public git 73 \nrepositories. Although both GitHub and GitLab have sophisticated search functionality, it 74 \nremains difficult to identify and compare repositories with training materials on a topic of 75 \ninterest. The findability of repositories with training materials depends on the effort of the 76 \nauthor to describe the repository, for example in the repository description, the name, the 77 \nassociated website, and repository tags. Since not all authors annotate their repositories 78 \nextensively and the way developers annotate repositories is not standardized, many 79 \nrepositories with excellent training materials remain hard to find.  80 \nThis article introduces Glittr.org, a novel web-based resource designed to enhance the 81 \ndiscovery and comparison of reusable bioinformatics training materials hosted on public Git 82 \nrepositories. Addressing the needs of both students and trainers, Glittr.org contains intuitive 83 \nsearch functionalities based on pre-defined topics and repository metadata. This approach not 84 \nonly facilitates the users’ search based on specific topics but also the refinement of their 85 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 5 \nsearch based on factors like the training material's popularity and recency. Moreover, Glittr.org 86 \nsupports the adherence to the FAIR principles by making all repository metadata readily 87 \naccessible through REST API endpoints, fostering findability and reusability of bioinformatics 88 \ntraining materials within the wider research community. 89 \nMaterials and methods 90 \nData collection and curation 91 \nGlittr.org is a manually curated list of bioinformatics training materials from anywhere in the 92 \nworld and in any language. The list items are public GitHub or GitLab repositories. The teaching 93 \naim of the material should be a general topic in bioinformatics, not tool usage – meaning that 94 \ninclusion of tutorials on using single software are discouraged. The reason for this is to ensure 95 \nGlittr.org provides an overview of metadata on training materials, and not on metadata related 96 \nto software. The quality of the training materials is not evaluated. Repositories that are 97 \npotentially eligible to be part of Glittr.org are identified from existing collections, internet 98 \nsearches, specific searches on GitHub, (social media) promotions and user contributions 99 \nthrough a web form.  100 \nRepository metadata are retrieved from GitHub and GitLab. Among those are the repository 101 \nname, URL, repository description, author and contributor names, their profile and web site, 102 \nthe number of stargazers (bookmarks), the date of the last push, and the license. These are all 103 \nprogrammatically accessible from the REST APIs of GitHub and GitLab. 104 \nTo enable categorization and improve search functionality in Glittr.org, the training content in 105 \nthe repositories are described with tags. These tags are based on existing ontologies as much 106 \nas possible. Using ontologies greatly enhances standardization and therefore knowledge 107 \nsharing. Training material in bioinformatics can cover a very wide range of topics, and using a 108 \nsingle ontology to describe them can be problematic. Glittr.org therefore uses multiple 109 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 6 \nontologies, aiming to give an as rich as possible topic description for each repository. Currently, 110 \nthe following ontologies are used: EDAM [14], Data Science Education Ontology 111 \n(http://www.pagestudy.org/DSEO/), Experimental Factor Ontology [15] and FAIR Data Train 112 \nOntology (https://bioportal.bioontology.org/ontologies/FDT-O). If a term is considered relevant, 113 \nbut not in any of these ontologies, a term will be used from any other ontology on BioPortal [16], 114 \na software on bio.tools [17]  or an item at Wikidata [18]. By using multiple ontologies, we have 115 \nthe flexibility to annotate the wide range of training material, but still have good level of 116 \nstandardization in relation to the relevant field.  117 \nEach repository is associated with one or more tags by a curator, where the order of tags 118 \nmatters, meaning that the tag that describes the content best is first (the ‘main tag’) and is then 119 \nfollowed by other relevant tags all with equal importance. For example, for a single cell 120 \ntranscriptomics course in R, the tag order could be ‘Single-cell sequencing’, ‘R’, 121 \n‘transcriptomics’ and ‘RNA-seq’. The first and therefore main tag would be ‘Single-cell 122 \nsequencing’, which is used to categorize the material, while the others have equal importance 123 \nand are used to further describe the material. Each tag falls within a broader category, which 124 \nare maintained within Glittr.org.  Currently, these are: (1) Scripting and languages, (2) 125 \nComputational methods and pipelines, (3) Omics analysis, (4) Reproducibility and data 126 \nmanagement, (5) Statistics and machine learning, and (6) Others. The main tag of the above-127 \nmentioned single cell course is part of the category ‘Omics analysis’ and so the course material 128 \nfalls within this category.  129 \nWeb application 130 \nThe Glittr.org web application is created using the Laravel (https://laravel.com/) framework and 131 \nis written in PHP. Repository metadata is actively updated twice daily using Laravel jobs. This 132 \napproach enables handling of potential issues by implementing delayed retries if the 133 \nGitHub/GitLab APIs are temporarily unavailable. Each metadata update for a single repository 134 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 7 \nconstitutes a dedicated Laravel job, with the overall management of all repository metadata 135 \nupdates performed with the Laravel Haystack package (https://docs.laravel-haystack.dev/). 136 \nThis package ensures job validation and sends email notifications in case of errors.  137 \nAll the reactivity supporting list functionalities such as filtering, pagination, and sorting is based on 138 \nthe Livewire (https://livewire.laravel.com/) framework. The JSON API endpoints for repositories and 139 \nBioschemas are generated with the Laravel-query-builder (https://spatie.be/docs/laravel-query-140 \nbuilder/v5/introduction) package developed by Spatie (https://spatie.be/open-source). This ensures 141 \nthe query parameters and output results follow the JSON API specification as closely as possible. The 142 \nbackend to manage data (topics and categories, repositories, submissions) is created with the 143 \npackage Jetstream (https://jetstream.laravel.com/introduction.html).  144 \nAll source code is accessible at https://github.com/sib-swiss/glittr and the website is available 145 \nthrough https://glittr.org. The application is a SIB Training resource and is hosted at servers 146 \nfrom SIB. As an institute with a long history of working with database life cycles, we are 147 \ncommitted to sustaining glittr.org for the time it is serving the community by aligning to the 148 \nTRUST principles [19]. 149 \nA snapshot of the front-end website is found in Fig 1. The user can easily access both the 150 \nrepository with course material and the course web site (when applicable). In addition, a link to 151 \npublic information about the author or organization that created the material is provided to 152 \nretrieve more author information. The topics (tags) covered by the material allows users to 153 \nquickly find course material relevant to their search. The number of stargazers estimates the 154 \npopularity of the repository, while the number of days since the last push provides an indication 155 \nof the recency of the material in the repository. The license which applies to the repository 156 \ncontents indicates to trainers wishing to re-use the material what conditions apply. 157 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 8 \n 158 \nFig 1. Snapshot of Glittr.org. Here, two tags are selected ‘Transcriptomics’ and ‘R’, showing the (current) 159 \ntop 2 of 25 repositories with both tags. By default, results are sorted based on number of ‘stargazers’, i.e. 160 \nbookmarks.  161 \nThe application currently has three REST API endpoints in which data stored by Glittr.org can be 162 \nprogrammatically accessed: api/repositories, api/tags and api/bioschemas. The 163 \napi/repositories endpoint contains all repositories including their metadata as presented on the 164 \nwebsite. The repositories/tags endpoint contains tag metadata organized by their 165 \ncorresponding category. Tag metadata consists of the ontology, ontology class and description. 166 \nThe api/bioschemas endpoint contains repository and tag metadata according to the 167 \nBioschemas [9] TrainingMaterial profile [10] (https://bioschemas.org/profiles/TrainingMaterial). 168 \nTraining material characteristics 169 \nTo give an overview and statistics of all the metadata stored on Glittr.org, we used the R httr2 170 \npackage to access the Glittr.org REST API (see above). Public information on authors, and 171 \nnumber of contributors was extracted from the GitHub REST API. Location of authors and 172 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 9 \norganization was extracted from the ‘location’ field within the users API. This string was 173 \ntranslated in a country with the R package geocode, which has wrappers for the Google 174 \nGeocoding API. The source code for acquiring the statistics can be found at 175 \nhttps://github.com/sib-swiss/glittr-stats and the report can be approached through https://sib-176 \nswiss.github.io/glittr-stats/ (updated weekly).  177 \nResults and Discussion 178 \nThere are currently 568 repositories listed on Glittr.org. By combining data from our REST APIs 179 \nand those of GitHub and GitLab we can give a detailed insight in these repositories. In the first 180 \npart of this section, we show what kind of repositories with training material are currently 181 \navailable, for which topics training materials exist, what licenses are used, and by whom the 182 \nrepositories are created. In the second part we present several use-cases on how different 183 \ntypes of users can interact with Glittr.org to show how trainers, learners and developers can 184 \nuse different functionalities of glittr.org.  185 \nTraining material topics 186 \nRepositories on glittr.org are categorized in six categories and tagged with 55 different topics. 187 \nAll are connected to bioinformatics, but many are not specific to bioinformatics: for example, 188 \ncomputer languages are relevant to informatics in general, and statistics to all fields related to 189 \nscience. Therefore, using glittr.org is relevant to a broader public than just persons working in 190 \nthe field of bioinformatics. In this section we give a brief overview of popular categories and 191 \ntopics.  192 \nBy far most repositories currently on Glittr.org have ‘Scripting and languages’ as main category 193 \n– 315 in total (Fig 2A). Within this category, R [20] is the most frequent tag with 230 repositories, 194 \nfollowed by Python with 81 repositories (Fig 2C). This suggests that the R language is currently 195 \nthe most popular language to teach and therefore probably to learn in bioinformatics, with 196 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 10 \nPython coming in second. The popularity of R might not be the only reason; the ease with which 197 \none can host static websites within the R framework, with e.g. bookdown [21] or Quarto 198 \n(https://quarto.org/), could also explain this.  199 \n 200 \nFig 2. Key statistics by category. Overview of number of repositories by category (A), distribution of 201 \nnumber of contributors per repository by category (B), number of repositories per tag, square root 202 \ntransformed (C), per author or organization (D) and per license (E). Colours are according to category as 203 \ndepicted in (A). In C) only the tags with more than 10 repositories are displayed and in D) only the 204 \nauthors/organizations with five or more repositories. At time of writing Glittr.org contained repositories 205 \nOthers\nStatistics and machine learning\nReproducibility and data management\nComputational methods and pipelines\nOmics analysis\nScripting and languages\n0 100 200 300\nNumber of repositories\nCategoriesA\nOthers\nStatistics and machine learning\nReproducibility and data management\nComputational methods and pipelines\nOmics analysis\nScripting and languages\n0 100 200 300 400 500\nNumber of contributors\nContributorsB\nTotal number of tags:  55\nChIP−seq\nJulia\nNextflow\nDocker\nMetagenomics\nShiny\nWorkflows\nContainerization\nVersion control\nVariant analysis\nData visualization\nFAIR data\nGeneral\nReproducibility\nData management\nSingle−cell sequencing\nUnix/Linux\nGenomics\nMachine learning\nData science\nNext generation sequencing\nStatistics\nRNA−seq\nTranscriptomics\nPython\nR\n0 50 100 150 200\nNumber of repositories\nTags with > 10 repositoriesC\nAuthors with < 5 repos:  242\ncambiotraining\ncarpentries−lab\nhadley\nbiocorecrg\nJuliaAcademy\nsemacu\nswcarpentry\nRockefellerUniversity\nlearnbyexample\nrstudio−conf−2022\nbioinformatics−ca\nGTPB\nbioinformaticsdotca\nbioinformatics−core−shared−training\ndatacarpentry\nhbctraining\nucdavis−bioinformatics−training\nNBISweden\nposit−conf−2023\nsib−swiss\ncarpentries−incubator\n0 10 20 30 40\nNumber of repositories\nAuthor or organizationD\nunlicense\nwtfpl\nagpl−3.0\nartistic−2.0\nbsd−3−clause\napache−2.0\ncc0−1.0\ncc−by−4.0\ngpl−3.0\ncc−by−sa−4.0\nmit\nnone\nother\n0 50 100 150 200\nNumber of repositories\nLicense typeE\nRepos with undefined country:  244\nArgentina\nChina\nFinland\nIndia\nLuxembourg\nNorway\nPoland\nUkraine\nBulgaria\nIreland\nItaly\nDenmark\nSpain\nBelgium\nFrance\nNetherlands\nPortugal\nGermany\nAustralia\nUnited Kingdom\nSweden\nCanada\nSwitzerland\nUnited States\n0 50 100\nNumber of repositories\nCountryF\n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 11 \nauthored by 242 authors/organizations with less than five repositories. For A), D) and E) the plots are 206 \ncoloured according to the category of the main (i.e. first) tag.  207 \nThe category ‘Omics analysis’ is specific to the field of bioinformatics, and many repositories 208 \nfall within this category. ‘Transcriptomics’ is the most frequent tag (74 repositories), followed by 209 \n‘RNA-seq’, suggesting an important place for transcriptomics-related courses in current 210 \nbioinformatics curricula. Transcriptomics analysis skills are popular to acquire, likely because 211 \nit is one of the most used applications of next generation sequencing, it is applied in many fields 212 \nin biology [22] and each experiment requires its own analysis choices. Its demand is well served 213 \nby the vast amount of material available on transcriptomics.   214 \nThere are 34 repositories on ‘Data management’, representing its growing importance within 215 \nthe scientific community regarding trends in open and FAIR data. Using public git repositories to 216 \ndevelop training materials on data management related topics, is likely to be of particular 217 \ninterest because using git supports transparency and community-based development. Efforts 218 \nthat support open and reproducible science like the ELIXIR Research Data Management (RDM) 219 \ncommunity, European Open Science Cloud (EOSC) and other research infrastructures likely 220 \ninfluence this trend towards using git repositories for training materials both on data 221 \nmanagement and other topics.  222 \nThe number of repositories and their diversity show that there is a wealth of public repositories 223 \nwith bioinformatics training material. For many topics, there are high-quality materials in 224 \ndifferent formats, giving lesson developers and learners many possibilities to re-use existing 225 \ntraining materials. Glittr.org organizes this wealth of materials, making them better findable and 226 \ntherefore available for re-use.  227 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 12 \nCollaboration 228 \nAn important advantage of using version control for training materials is that it supports 229 \ncollaboration and building further on existing material. This is also represented in the number of 230 \ncollaborators per repository that are on Glittr.org. Most repositories (78.8%) have more than 231 \none contributor, meaning the repository is developed by more people than the original author. 232 \nMany repositories have only a few contributors, between 1 and 5 (59.9%). However, quite a 233 \nlarge fraction (25.6%) has more than 10 contributors, showing that these repositories gain from 234 \na strong community involvement and therefore increased (re-)usage and quality.  235 \nRepositories in the category ‘Reproducibility and data management’ and ‘Computational 236 \nmethods and pipelines’ have the largest fraction of repositories developed by a large 237 \ncommunity (> 50 contributors; Fig 2B). Repositories with a large contributor base within these 238 \ncategories are e.g. elixir-europe/rdmkit - which is a community-created website with RDM best 239 \npractices [23], galaxyproject/training-material – the Galaxy training network repository with all 240 \ntraining materials related to Galaxy [13] - or are forked from a repository with a large community 241 \nbase (e.g. a template from the carpentries [12]). This shows that large and important initiatives 242 \nrely on git for the development of training materials and that these thrive by cooperation. 243 \nAuthors and organizations 244 \nThe origin of repositories on Glittr.org is very diverse; currently, the 568 repositories originate 245 \nfrom 263 different GitHub/GitLab authors or organizations. The organization with most 246 \nrepositories is the carpentries-incubator [12], followed by two national institutes, NBIS and SIB 247 \n(Fig 2C). Obviously, the distribution of repository category varies per organization/author. 248 \nBioinformatics institutes, like bioinformatics.ca, Instituto Gulbenkian de Ciência (GTPB), CRUK 249 \nCI (bioinformatics-core-shared-training), Harvard bioinformatics core (hbctraining), UC Davis 250 \nBioinformatics Core (ucdavis-bioinformatics-training), SIB and NBIS, are more focused on 251 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 13 \ncomputational biology and have more courses in the category ‘Omics analysis’. Other popular 252 \nnamespaces like carpentries incubator [12], the ’23 posit conference (posit-conf-2023) and 253 \nlearn by example (learnbyexample) are focused on computer skills and therefore have mostly 254 \nrepositories on ‘Scripting and languages’ and ‘Computational methods and pipelines’ (Fig 2C).  255 \nIn addition to the number of authors, the number of countries the authors affiliate themselves 256 \nwith is also diverse. Current repositories on Glittr.org originate from 24 different countries. Most 257 \nrepositories with the main author or organization having a country defined in its profile originate 258 \nfrom the United States, followed by Switzerland and Canada (Fig 2F). This shows that most 259 \ntraining materials originate from the Western and particularly English-speaking parts of the 260 \nworld. English is currently the ‘lingua franca’ of the academic world [24], and most training 261 \nmaterials in bioinformatics reflect that. Connections between the academic ecosystem and 262 \ntraining materials not written in English can be literally lost in translation, and therefore hard to 263 \nidentify. However, there are many communities that demand non-English bioinformatics 264 \ntraining material, and we are open to serving these communities by listing them on Glittr.org. 265 \nOrganization of the curation of repositories in other languages is more challenging because it 266 \nrelies on reading and understanding the content, but we encourage initiatives in this direction.  267 \nLicensing 268 \nAlthough licensing is an important way to enable and encourage re-use, licensing of public 269 \ntraining materials on GitHub and GitLab is not very standardized. Fig 2B gives an overview of 270 \nlicenses per repository that could be recognized as such. For many repositories (172; 30.4%), 271 \nthere is no license detected, and for 211 (37.3%) repositories the license could not be 272 \nautomatically recognized. The most popular recognized license is the MIT license, a software 273 \nlicense. The copyright licenses based on Creative Commons (CC) are also quite popular, 75 274 \nrepositories have a license recognized as such. Since training materials generally consists of 275 \ntext, presentations, and exercises, and not software, it is usually more appropriate to use a 276 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 14 \ncopyright license instead of a software license [4,11]. It therefore seems that many developers 277 \nof training materials might want to (re-)consider their license.   278 \nUse cases  279 \nGlittr.org has been developed for use by bioinformatics students, trainers, lesson developers 280 \nand database developers. To showcase different applications of the resource, we present five 281 \ndifferent use cases that we have envisaged for Glittr.org. The first four use cases illustrate the 282 \nease of use, as only a single click is necessary to obtain an answer, while the last one shows 283 \nhow to retrieve information programmatically. 284 \nA student looking for training materials 285 \nOur first use case is that of a student looking for training materials, for instance someone 286 \nwishing to learn the programming language Julia. This high-level, general-purpose dynamic 287 \nprogramming language used for numerical analysis and computational science is usually 288 \noffered in university and higher order education curricula. Using the ‘Filter by topic’ option to 289 \nselect Julia under the ‘Scripting and Languages’ section, only repositories which have been 290 \nmanually assigned the topic are retrieved. The top result is the most popular as, by default, the 291 \nresults are displayed in order of decreasing number of stargazers. It is also possible to sort by 292 \nincreasing number of days since the last push to find material which has been recently 293 \nupdated. The link to the website takes the student directly to the page showing the different 294 \noptions to get started with Julia. Glittr.org thus allows students to quickly access the most 295 \npopular, public course material on the subject. 296 \nA trainer looking for material for re-use 297 \nThis use case is that of a trainer who will be teaching a course on Snakemake and is looking for 298 \nmaterials that can be re-used. Snakemake is a workflow management system to create 299 \nreproducible and scalable data analyses. Selecting Snakemake in the Computational methods 300 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 15 \nand pipelines section of the ‘Filter by topic’ option retrieves several repositories with materials 301 \nto teach Snakemake. By looking at the license column, the trainer can easily spot that there are 302 \ntwo repositories that contain material that have a re-use permissive license, one under the 303 \nCreative Commons Attribution-ShareAlike 4.0 International license (CC BY-SA 4.0) and another 304 \nunder the GNU General Public License version 3 license (pl-3.0). For other repositories in the 305 \nlist, the license could not be automatically identified, and therefore the license has been 306 \nindicated as ‘other’. For these repositories some further investigation is needed before they can 307 \nbe re-used. The trainer has quickly found re-usable material on the topic of interest using the 308 \nGlittr.org web site.  309 \nA trainer sharing their own materials 310 \nThe third use case is that of a trainer who has developed a course on a hot topic in 311 \nbioinformatics on GitHub or GitLab and wishes to share the material. Contributions to the 312 \nGlittr.org web site from the training community are welcome and encouraged. The Contribute 313 \nbutton on the top right of the home page (Fig 1) links to a simple form. By completing the 314 \nsubmission form, the trainer's repository will be added (or updated) to the collection provided it 315 \nmeets the inclusion criteria (public GitHub or GitLab repositories containing bioinformatics 316 \ntraining material). The trainer’s material is now findable by being included in the Glittr.org web 317 \nsite. Obviously, this does not prevent the trainer from posting the material in other repositories 318 \nwhich is, in fact, recommended by the FAIR principles (the more the material is registered, 319 \nmore chances are that they are findable). 320 \nA database developer wanting to programmatically access Glittr.org 321 \nA final use case is that of a developer who must programmatically access one or more subsets 322 \nof training materials repositories on Glittr.org. This can easily be done using the API endpoints 323 \n(https://github.com/sib-swiss/glittr#api) and applying suitable filters. For example, the data for 324 \nthe graphs in Fig 2 are all extracted from the Glittr.org API with the R packages httr2 [25]. The 325 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 16 \nBioschemas endpoint is a special API endpoint in Glittr.org. It provides access to the metadata 326 \nof all the repositories in Glittr.org in Bioschemas TrainingMaterial markup. This allows portals 327 \nthat can ingest Bioschemas markup to re-use Glittr.org data. For example, ELIXIR TeSS [8], 328 \ncurrently ingests this metadata (https://tess.elixir-europe.org/content_providers/glittr-org).  329 \nConclusion 330 \nWhen we created Glittr.org, our aim was to create a simple-to-use, web-based listing of 331 \nbioinformatics training materials available on GitHub/GitLab because as training providers we 332 \nsaw the need for such a tool. Within one year, more than 500 repositories with bioinformatics 333 \ntraining materials with a diverse set of topics have been included in Glittr.org. By focussing only 334 \non git repositories, Glittr.org can automatically display relevant information like popularity, 335 \nlicense and recency that would otherwise not be possible. The resource has the potential to 336 \nhelp in training material selection, decision-making by both students and trainers and training 337 \nmaterial database integration, as illustrated by the five use cases presented. All the information 338 \nis available programmatically via the API and Bioschemas markup thereby enabling other 339 \ntraining registries to freely update their existing entries or add entries not yet present in their 340 \ndatabase. This will further increase the coverage of these registries and promote the findability, 341 \naccessibility, interoperability, and re-usability (FAIRness) of training materials in the life 342 \nsciences.  343 \nReferences 344 \n1.  Filiposka S, Green D, Mishev A, Kjorveziroski V, Corleto A, Napolitano E, et al. Draft 345 \nMethodology for FAIR-by-Design Learning Materials. 2023 [cited 11 Apr 2024]. Available: 346 \nhttps://zenodo.org/records/7875541 347 \n2.  Haase R, Tischer C, Bankhead P, Miura K, Cimini B. A Call for FAIR and Open-Access 348 \nTraining Materials to advance Bioimage Analysis. Open Science Framework; 2024 Mar. 349 \ndoi:10.31219/osf.io/2zgmc 350 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 17 \n3.  Alexandra Holinski, Alexia Cardona, Alice Matimba, Allegra Via, Anastasios Anastasios 351 \nPapaioannou, Anna Swan, et al. ELIXIR fair training handbook. 2024 [cited 22 Mar 2024]. 352 \ndoi:10.5281/ZENODO.10839551 353 \n4.  Garcia L, Batut B, Burke ML, Kuzak M, Psomopoulos F, Arcila R, et al. Ten simple rules for 354 \nmaking training materials FAIR. PLoS Computational Biology. 2020;16: 1–9. 355 \ndoi:10.1371/journal.pcbi.1007854 356 \n5.  Wilkinson MD, Dumontier M, Aalbersberg IjJ, Appleton G, Axton M, Baak A, et al. The FAIR 357 \nGuiding Principles for scientific data management and stewardship. Sci Data. 2016;3: 358 \n160018. doi:10.1038/sdata.2016.18 359 \n6.  Sparks SD. Open Educational Resources (OER): Overview and Definition. Education Week. 360 \n12 Apr 2017. Available: https://www.edweek.org/teaching-learning/open-educational-361 \nresources-oer-overview-and-definition/2017/04. Accessed 5 Feb 2024. 362 \n7.  Corpas M, Jimenez RC, Bongcam-Rudloff E, Budd A, Brazas MD, Fernandes PL, et al. The 363 \nGOBLET training portal: a global repository of bioinformatics training materials, courses 364 \nand trainers. Bioinformatics. 2015;31: 140–142. doi:10.1093/bioinformatics/btu601 365 \n8.  Beard N, Bacall F, Nenadic A, Thurston M, Goble CA, Sansone S-A, et al. TeSS: a platform 366 \nfor discovering life-science training opportunities. Bioinformatics. 2020;36: 3290–3291. 367 \ndoi:10.1093/bioinformatics/btaa047 368 \n9.  Gray AJG, Goble C, Jimenez RC. Bioschemas: From Potato Salad to Protein Annotation. 369 \nISWC 2017 Posters & Demonstrations and Industry Tracks: Proceedings of the ISWC 2017 370 \nPosters & Demonstrations and Industry Tracks co-located with 16th International 371 \nSemantic Web Conference (ISWC 2017). RWTH Aachen University; 2017. Available: 372 \nhttps://research.manchester.ac.uk/en/publications/bioschemas-from-potato-salad-to-373 \nprotein-annotation 374 \n10.  Castro LJ, Palagi PM, Beard N, Attwood TK, Brazas MD. Bioschemas training profiles: A set 375 \nof specifications for standardizing training information to facilitate the discovery of 376 \ntraining programs and resources. PLOS Computational Biology. 2023;19: e1011120. 377 \ndoi:10.1371/journal.pcbi.1011120 378 \n11.  Dogucu M, Çetinkaya-Rundel M. Tools and Recommendations for Reproducible Teaching. 379 \nJournal of Statistics and Data Science Education. 2022;30: 251–260. 380 \ndoi:10.1080/26939169.2022.2138645 381 \n12.  Wilson G. Software Carpentry: lessons learned. F1000Research; 2016. 382 \ndoi:10.12688/f1000research.3-62.v2 383 \n13.  Hiltemann S, Rasche H, Gladman S, Hotz H-R, Larivière D, Blankenberg D, et al. Galaxy 384 \nTraining: A powerful framework for teaching! PLOS Computational Biology. 2023;19: 385 \ne1010752. doi:10.1371/journal.pcbi.1010752 386 \n14.  Ison J, Kalaš M, Jonassen I, Bolser D, Uludag M, McWilliam H, et al. EDAM: an ontology of 387 \nbioinformatics operations, types of data and identifiers, topics and formats. 388 \nBioinformatics. 2013;29: 1325–1332. doi:10.1093/bioinformatics/btt113 389 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint \n\n 18 \n15.  Malone J, Holloway E, Adamusiak T, Kapushesky M, Zheng J, Kolesnikov N, et al. Modeling 390 \nsample variables with an Experimental Factor Ontology. Bioinformatics. 2010;26: 1112–391 \n1118. doi:10.1093/bioinformatics/btq099 392 \n16.  Noy NF, Shah NH, Whetzel PL, Dai B, Dorf M, Griffith N, et al. BioPortal: ontologies and 393 \nintegrated data resources at the click of a mouse. Nucleic Acids Res. 2009;37: W170–394 \nW173. doi:10.1093/nar/gkp440 395 \n17.  Ison J, Rapacki K, Ménager H, Kalaš M, Rydza E, Chmura P, et al. Tools and data services 396 \nregistry: a community effort to document bioinformatics resources. Nucleic Acids Res. 397 \n2016;44: D38–D47. doi:10.1093/nar/gkv1116 398 \n18.  Wikidata. [cited 9 Apr 2024]. Available: https://www.wikidata.org/ 399 \n19.  Lin D, Crabtree J, Dillo I, Downs RR, Edmunds R, Giaretta D, et al. The TRUST Principles for 400 \ndigital repositories. Sci Data. 2020;7: 144. doi:10.1038/s41597-020-0486-7 401 \n20.  R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R 402 \nFoundation for Statistical Computing; 2023. Available: https://www.R-project.org/ 403 \n21.  Xie Y. bookdown: Authoring Books and Technical Documents with R Markdown. Boca 404 \nRaton, Florida: Chapman and Hall/CRC; 2016. Available: 405 \nhttps://bookdown.org/yihui/bookdown 406 \n22.  Stark R, Grzelak M, Hadfield J. RNA sequencing: the teenage years. Nat Rev Genet. 407 \n2019;20: 631–656. doi:10.1038/s41576-019-0150-2 408 \n23.  ELIXIR. Research Data Management Kit. A deliverable from the EU-funded ELIXIR-409 \nCONVERGE project (grant agreement 871075). Available: https://rdmkit.elixir-europe.org 410 \n24.  Jenkins J. English as a Lingua Franca in the International University: The Politics of 411 \nAcademic English Language Policy. Routledge; 2013.  412 \n25.  Wickham H. httr2: Perform HTTP Requests and Process the Responses. 2023. Available: 413 \nhttps://httr2.r-lib.org 414 \n 415 \n.CC-BY 4.0 International licensemade available under a \n(which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is \nThe copyright holder for this preprintthis version posted August 22, 2024. ; https://doi.org/10.1101/2024.08.20.608021doi: bioRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}