Getting to YES: The Evolution of the University of Pittsburgh Medical Center Hillman Cancer Center Youth Enjoy Science (YES) Academy.

The Hillman Academy started in 2009 as a fledgling program started by a small core of faculty who volunteered their time and resources to train and mentor six students in cancer research. It has grown into a joint initiative that spans over a dozen departments at the University of Pittsburgh (Pitt), UPMC, and the HCC, which are, respectively, a Research I University ranked fifth in NIH funding, a nationally ranked academic healthcare system, and an NCI-designated comprehensive cancer center. For more than a decade, the Hillman Academy has provided immersive, real-world, research experiences to students in the Pittsburgh Public School (PPS) System and beyond. A major focus of these efforts has been to forge relationships with our inner-city schools, rural communities, and advocates around the region to connect with youth from URM/DA backgrounds and equip and inspire them to enter STEM fields at the undergraduate level, persist in these pursuits, and impact these fields in which disparities lead to negative outcomes for the groups these students represent ( Hacker K, Brink, L, Jones, L, 2017 ; Research, 2015; Singh & Jemal, 2017 ). The Hillman Academy began in 2009 as an 8-week summer research program with 6 high school students. Over time, through strong faculty and community support, as well as additional funding by the Doris Duke Charitable Foundation and NCI supplements to its P30 Continuing Umbrella of Research Experiences (CURE) program, the Hillman Academy became an integrated program centered on, but not limited to, authentic and mentored research experiences that engage multiple groups of students: i) 24+ URM/DA high school students who are funded to participate in a 2-year-long NCI YES program (described in detail below) that includes 2 summer research experiences and 2 academic-year professional development series, ii) 10–15 URM/DA high school students and 5 alumni undergraduates who are funded the Doris Duke Charitable Foundation to participate in summer research, and iii) non-URM/DA volunteer participants that also participate in the summer research program but are not compensated. By participating in many of the same group activities, the students benefit from interacting with peers from diverse racial, ethnic, academic, and geographical backgrounds that provide a college-like experience and large local learning environment. Over the past 13 years, the Hillman Academy grew considerably ( Figure 1 ) and has trained and mentored 239 URM/DA students through 294 paid internship opportunities (some students participate for multiple years), including 60 alone in 2021 despite a forced hybrid format due to COVID-19 restrictions. YES funding has had a significant impact on the proportion of URM/DA students in the Hillman Academy. Prior to YES funding, we could provide financial support to between 20–30 students, which was ~40–50% of the number of students in the program ( Figure 1 2010–2019 ). YES funding began in the fall of 2019, and in the 2 years since that time, the proportion of URM/DA students was 100% in the pandemic limited 2020 program and ~90% in 2021, which included over 50 URM/DA students ( Figure 1 2020–2021 – red box). We believe the growth in the number of participants, as well as applicants, over time speaks to how the program is embraced by faculty and the community. The sustainability and growth of the program is largely attributed to YES funding. YES funding of the Hillman Academy 1) doubled the number of URM/DA students we were able to financially support; 2) strengthened partnerships with Community-Based Programs by resourcing a community outreach program that makes hands-on STEM offerings to a broader set of students; 3) provided funded effort for faculty leads in 6 different departments, who recruit mentors and develop specific didactic material for their department; 4) provided resources for the logistics necessary to host a large number of URM/DA students such as housing, bus fares, shuttle service, laptops, etc.; and 5) provided the materials and reagents necessary for the large number of faculty hosts to engage students in authentic research. While the Hillman Academy as a whole provides the same integrated summer research experience for all groups of participants, students in the YES program participate in an additional year of summer research and two years of an academic-year enrichment program (described below) that builds off of the eight-week summer research experience. The goals of the Hillman Academy are outlined below. The Hillman Academy has set several educational and outreach goals for the benefit of our students. These specific goals, outlined here, are the guiding pillars of our program. Increase the diversity of students who will be prepared to pursue biomedical cancer research, healthcare, or other STEM-related careers by providing authentic and mentored research experiences to URM/DA high school students. Familiarize students with STEM careers options, particularly those related to cancer care and research, and equip them with the necessary skills to pursue them. Provide didactic training in cancer biology and related fields in six interconnected disciplines. Enhance scientific writing and speaking skills. Reach out to and activate other URM/DA students by hosting active learning workshops and discussions for our regional high schools, churches, and community partners. All Hillman Academy students participate in a full-time summer research internship of seven–eight weeks in which they engage in cutting-edge cancer research directed by multiple levels of mentorship, which includes postdoctoral fellows, graduate and undergraduate students, and research personnel in addition to faculty members as primary mentors. Students explore a real scientific question, systematically gather data, make conclusions based on their interpretation of their data, and present their work at Pitt and beyond. Our program provides interactive training in key scientific skills, such as how to find, read, and discuss scientific literature; conduct research responsibly and ethically; keep a laboratory notebook; design experiments; use basic statistical approaches to analyze data; write clearly; and present research effectively. We also strive to exclude practices that discourage URM/DA students such as extensive, abstract, and non-relatable courses taught from textbooks with difficult exams ( TovenLindsey et al., 2015 ). With the help of their mentors, Hillman Academy students lead a journal club presentation to their peers on a paper relevant to their research question and also engage one another through routine research roundtables at which students pose and justify their research question to a group of peers and provide updates on their progress over their two-year tenure. The Hillman Academy also hosts a variety of guest lecturers—with an emphasis of scientists from similar backgrounds as the students—to expose the students to a broad array of scientists, STEM careers, and paths to success. The end of each Hillman Academy summer program culminates in a symposium of oral presentations by all the students to their peers, mentors, laboratory colleagues, family members, and teachers. Additionally, they present a poster on their work in the atrium of the HCC during a competitive poster session, which is open to all the members of Pitt, UPMC, HCC, and the local community. For 2020 and 2021 only, the symposium was hosted via Zoom because of pandemic restrictions. The recordings of the student presentations are available on our YouTube channel ( https://www.youtube.com/channel/UCC_n1kISicItwgeo64LcDSg/videos ). While all Hillman Academy students participate in the summer research experience, students accepted into the YES program participate in an additional summer research experience as well as 2 academic-year professional development programs. The goal of the academic-year YES curriculum is to complement and build upon the summer research experience. Throughout the year, we meet once per month as a group and as often as necessary with individual students to both equip them with a host of essential scientific communication and professional development skills and work with them to translate their research into competitive scientific papers, abstracts, and scholarship applications. All students apply to a minimum of two science fairs or conferences with the guidance of their mentors and program directors. In summary, the Hillman Academy including the YES-portion of the program provides students with immersive experiences using documented approaches that increase the likelihood of URM/DA students pursuing and persisting in the sciences. In other words, they learn how to be a scientist by being a scientist in an immersive and supportive environment. The HCC is in the heart of Pittsburgh and less than five miles from all nine PPS high schools. In addition to this urban center, the area that the HCC serves also includes rural Pennsylvania and the upper portions of Appalachia, an impoverished region with very large health disparities and a high percentage of DA students. The total enrollment of PPS is over 24,000 students, with approximately 63% of students from Black, Hispanic, or American Indian populations, and 62% from low-income families. Only 70% of PPS students graduate, and as a group, PPS students perform considerably worse than the state average on standardize testing ( Council of the Great City Schools, 2016 ). Further, the communities these students call home suffer from health disparities that directly increase their risk of cancer morbidity and mortality ( Hacker K, Brink, L, Jones, L, 2017 ). Nearly 50% of all black women do not receive annual mammograms, and a significantly higher percentage of black adults versus white adults, 30% versus 17%, respectively, reported that they are current smokers. Black adults are also 9% more likely to be overweight or obese than white adults, and 22% more black adults are worried about feeding their families compared with white adults. Finally, those with lower education were 5 times more likely to rate their health as fair or poor compared with the more affluent ( Hacker K, Brink, L, Jones, L, 2017 ). Likewise, due to the closing of steel mills and coal mines, which were the major industries for decades outside of Pittsburgh, communities in the Monongahela Valley and surrounding communities (within 60 miles of HCC) have become older and poorer with a high incidence of drug use disorder. For example, in Fayette County and Monessen City, nearly 40% of the 60,000 households make less than 2 times the poverty limit, which is twice that of state and national averages ( Program in Urban and Regional Analysis, 2015 ). The educational opportunities for these students are minimal. Thus, the HCC is located at the intersection of two major populations of students underrepresented in biomedical science – students of color and students from disadvantaged backgrounds. Our approach to recruit a diverse group of URM/DA students into the Hillman Academy includes i) direct communication with program alumni, local students, teachers, counselors, administrators, principals, superintendents, intermediate units, and others in the PPS System, ii) outreach at schools, community centers, churches, and throughout our surrounding area, and iii) networking with our local and regional partners that are focused on educational equity with an extensive reach into diverse communities and schools ( Lori Delale-O’Connor et al., 2021 ). This multifaceted approach has led to a diverse group of students from most populations that are underrepresented in the biomedical workforce ( Figure 2A and B ). The NIH defines URM/DA students as US citizens or permanent residents who identify as 1) Black or African American, Hispanic or Latino/a, American Indian or Alaska Native, Native Hawaiian, or Pacific Islander, 2) individuals with disabilities as described in the Americans with Disabilities Act of 1990, or 3) those from disadvantaged background ( U.S. Department of Health and Human Services, 2019 ). To be supported by our NIH or DDCF awards, the students must self-report to us at least one way they are underrepresented. Through our 13 years, approximately 57% of our students qualified based on race/ethnicity, with 36% qualifying based on income levels, and 7% based on disability ( Figure 2A ). The proportions of Black students and students with disabilities are significantly enriched in the NCI YES student portion of our program that encompasses only 2020 and 2021— 49% Black ( Figure 2B ) and 16% students with disabilities—in YES vs 28% Black and 7% disabilities in the total URM/DA alumni population. This is a result of increased funding, community partnerships, and intentional recruiting practices engaging these populations of students. Our recruitment efforts are primarily in Pittsburgh and its surrounding areas and rely on relationships deliberately forged over the past 6 years. We organize 15–20 events and talks at local schools, community events, and for community partners per year. Since we placed more of an emphasis on recruiting through community partnerships in 2016, we saw a steady annual increase in the number of URM/DA students applying and the number has tripled over the past 5 years ( Figure 2C ). For several of our partners, we started a guaranteed admissions process after the beginning of YES funding in 2019. As an example, we guarantee admission to PPS students that are highly recommended by trusted teachers and school administrators, who are integral partners of our program. These identified students will still complete an application, and their applications will be flagged in our system to notify the admissions teams that they have already been accepted into the program. In addition, we reach out and give talks directly to students in many of their classrooms, to all students taking AP classes at PPS, and to students in the Pittsburgh Public Schools Black Student Union. We also have guaranteed admission to a subset of students from community partners including the Fund for Advancement of Minorities through Education (FAME) and the Homeless Children’s Education Fund (HCEF). We host events on campus for students and meet in groups or with individual families in-person or remotely to discuss opportunities through the Hillman Academy’s YES program. The Jack Kent Cooke Foundation, which has the goal of advancing the education of exceptionally promising students who have financial need, also partners with us to recruit guaranteed admit students each year. Our recruitment strategy also engages our many partners to amplify the opening of the application to potentially interested students. Remake Learning—the Pittsburgh Regional STEM Ecosystem, the Allegheny County Intermediate Unit, the Pittsburgh Promise, the Pennsylvania Biology Teachers Network, and others send information to their contacts directly and through social media. Finally, the Western Pennsylvania School for the Deaf, a non-profit, tuition-free school founded in 1869, serves Deaf and Hard-of-Hearing students from all of Pennsylvania and is less than five miles from the HCC. We directly recruit from this school and send flyers and videos we produced using American Sign Language to all Deaf High Schools throughout the country. The main focus of all Hillman Academy is to provide students with a full-time, eight-week, authentic research experience. Each student has an individual research project completed under their own mentor team typically composed of a faculty member and a graduate or medical student or postdoctoral fellow. In collaboration with the Pitt Center for Mentoring, we have held mentor training sessions for faculty and student mentors who work with our students. Since launching in 2009, we have developed six research sites, each with a distinct research focus and a curriculum tailored to that focus, across Pitt’s two-mile wide campus ( Table 1 ). While the composition of each site varies—it can be a single department, a specialized research center, or multiple departments—all sites have a central hub for activities as well as a unique research identity. A seventh site in ophthalmology will be added in 2022 after a pilot year in 2021. Each research site has a faculty member lead who serves to recruit mentors within their departments/centers, organize a local educational team, review applications, and develop and implement a curriculum tailored to the students of that site. The creation of individual sites with distinct research approaches was the key to expansion of the Hillman Academy over time as it has allowed for a snowballing effect of mentor recruiting. The distributed site model has also diversified the research projects and provided students more choice in how they can engage in research. Finally, individual sites create small learning environments of around 10 students. Though students are working on individual projects, this format enables them to engage in didactic and other enrichment activities in small groups, which facilitates collaboration and cohort building while avoiding the risk of isolation. Students also benefit from integration into the larger social network that is formed by the composite of the six sites under the umbrella of the entire program. The six areas of research are explained on our website to applicants, who rank the sites in order of preference. Every effort is made to assign students to one of their two top sites upon acceptance into the program. Students remain at the assigned site throughout the eight weeks of each summer research experience along with students from the DDCF and the HCC volunteer program, creating truly diverse cadres of students in an integrated and specialized learning environment. The students work a full-time schedule of about 35 hours per week in their assigned labs or research groups when not engaged with the summer curriculum. In the second year, YES students, who participate in two consecutive summer research experiences, have the option to return to the same site for a deeper research experience or to attend a different site for exposure to a new curriculum, mentor, and research project. The research groups at each site are equipped with resources and facilities that provide students with the opportunity to complete an authentic cutting-edge research project in a variety of specialized areas of basic, translational, or clinical cancer investigation. All projects are chosen by the mentor to be something related to ongoing research within their lab. Sometimes students are provided with a list of projects, and they choose which is of interest to them, but for others they are given a single project from their mentor. Our multi-site structure provides a wide option of the types of research projects. Table 2 lists two example projects from each site in 2021. Some students will work in wet labs spending their summer pipetting and conducting cell and molecular biology experiments such as CRISPR, PCR, proliferation assays ( Figure 3 , left – images of students working in labs) perhaps studying the impact of a gene knockout on proliferation of cancer cells. Others will learn how to program in R or Python and conduct computational genomic or clinical research on datasets provided by their mentoring teams to determine if machine learning can aid clinical decisions on what targeted therapy patients should receive. The students are always under close supervision, but by the end of the summer acquire varied levels of independence depending on the student, mentor, and ethical and safety concerns of the project. The Hillman Academy uses a multi-level mentoring structure as outlined in Figure 4 . We engage highly motivated and NIH- and NSF-funded mentors to serve as research advisors and mentors for our students. Each student is placed into a research lab based on their indicated interest in their application. The breadth of the Hillman Academy across multiple sites and with over 200 unique participating faculty mentors allows for the students’ interests to drive their research experience, facilitating project ownership. In addition to the primary faculty member, most Hillman Academy students have at least one near-peer mentor, who is typically a graduate or medical student, postdoctoral fellow, or research specialist. The near-peer mentors teach students basic research skills, guide their daily research, assist with understanding primary literature articles, aid in writing an abstract, and prepare the students for their final presentations and beyond. First-year Hillman Academy students get the added benefit of interacting with second-year Hillman Academy students in the YES program as well as Hillman Academy alumni who are now undergraduates and returning to complete research during the summer. This gives all Hillman Academy students the opportunity to serve in a leadership role within the program and adds another layer of near-peer mentors for new students who may require more attention and support. A broader mentoring layer is provided by the six research site leaders, who all have at least five years of leadership and mentoring experience from prior Hillman Academy cohorts. Site leaders organize the active-learning site-specific curricula, and both facilitate and monitor communication between students and mentors. Site-specific curricula are carried out by an additional set of graduate student and postdoctoral mentors that we call our educational team. Lastly, the program leadership is yet another layer of mentorship. The leadership team (Program Director, Associate Director, and Program Coordinator) is the first to communicate with students and maintains continuous communication during and for years after the program to provide advice, guidance, and support. On the first day of the summer research program, all students meet for orientation. Here, students participate in a program that includes mandatory safety and responsible conduct of research training sessions, including blood-borne pathogen and chemical hygiene safety training, proper experimental record keeping, and responsible data management, that prepare them for work in a lab or research group. There is also an overview of program expectations, general guidelines for success in the program, introductions of the six site leaders and their research sites, and “icebreaker” activities to familiarize students with program leaders and mentors and foster an engaged and supportive local learning environment. Finally, during the first week of the program, we offer workshop sessions based on the Entering Research ( Balster et al., 2010 ; Branchaw et al., 2020 ) curriculum that actively engage students in conversations about common concerns and barriers for those entering research along with strategies for bolstering self-efficacy and success ( Table 3 ). These sessions are taught by a Master Facilitator of the Entering Research Training materials that are made available through the Center for the Improvement of Mentored Experiences in Research (CIMER). Much of the summer curriculum focuses on teaching scientific skills. All students attend workshops at their research sites with small group instruction and peer–peer learning on how to effectively find, read, organize, and discuss research papers for a journal club and how to give high-quality oral and poster presentations. The small-group format is also used for research roundtable discussions where each student discusses their research project every one to two weeks with other students at their site. Outside of the basic scientific skills curriculum, each of the six research sites has a customized syllabus with didactic sessions on topics relevant to the site’s area of research ( Table 1 ). Additionally, students have the option to tour key facilities, including the Pitt Zebrafish Facility, Pitt Flow Cytometry Facility, Center for Biologic Imaging, and UPMC operating rooms. While most of the students’ academic and professional development occurs in a laboratory setting under the supervision of their mentors, we engage all students as a group in several enrichment activities that promote career awareness, college readiness, and peer engagement ( Figure 3 , middle) . Each student attends a series of spotlight discussions/seminars in which accomplished and diverse professionals in research and clinical fields talk with students about the work that they do and their career paths. In the virtual and hybrid settings of 2020 and 2021, most of these talks were live streamed through YouTube to allow students outside of the Hillman Academy to benefit as well. To build on this, we are now partnering with PPS to bring these talks to classrooms to expose students to scientific and clinical role models who come from similar backgrounds. Additional enrichment activities throughout the summer include a weekly keynote speaker series in which students from all sites come together to explore a variety of professional development topics including but not limited to: Diversity Awareness, led by the Assistant Vice Chancellor for Health Science Diversity Leadership and Purpose, led by the Coro Institute Basics of Effective Science Communication, led by the Pitt Center for Teaching and Learning Mindfulness, led by a therapist at the University’s Stress-Free Zone Introduction to MD/PhD programs, led by the Pitt Medical Scientist Training Program Director Discussion about financial aid and college admissions, led by the Pitt Admissions Office and the former Director of College Counseling at Collegewise. STEM education and career experiences from a student perspective, led by Hillman Academy alumni Maintaining a CV and LinkedIn profile, led by Hillman Academy staff We value the transition from secondary to post-secondary education as a critical development in the lives of our students. To facilitate this, we hold a session in which staff from the Admissions and Financial Aid Departments of Pitt discuss the programs and resources available at the university. We also connect students with the former Director of College Counseling at Collegwise to learn application strategies and have one-on-one editing of college essays. In addition to this formal approach to college preparation, we host specific meetings for students to regularly interface with the undergraduate and graduate students within the Hillman Academy network to provide guidance on how to navigate the next steps of their academic journey. For instance, summer undergraduate students and graduate students in the Department of Computational and Systems Biology, which houses the CompBio Site, host meetings between these groups during the summer. Each summer research experience of the Hillman Academy culminates in oral presentations at each research site ( Figure 3 , right ) and at an HCC-wide poster session that serves as students’ foremost opportunity to exhibit their research and communication skills and expand their network in the Pitt/UPMC/HCC community. Posters are visited by Pitt/UPMC/HCC faculty and lab personnel, as well as community partners, parents, and friends. Unfortunately, due to the COVID-19 pandemic, final presentations in 2020 and 2021 were held via Zoom. The talks of all students are available on our YouTube channel. In addition to the summer research experiences, students who are accepted into the YES program participate in an academic-year professional development program in two consecutive years. Since many students have multiple responsibilities during the school year, including homework, sports, music, jobs, and family obligations, we created a curriculum during the academic year that is primarily asynchronous and focused on student-generated content related to their summer projects and an extension of their final presentations. For example, we require all students to apply to two science fairs or symposia per year. Research demonstrates that students who compete in science fairs are more likely to complete a STEM major than those who do not, even when controlling for prior STEM interest and college readiness, and students who participate in more than one science fair are even more likely to persist in STEM (Miller et al., 2017; Sahin et al., 2017 ). Since our academic-year program is still in its first few years and all within the COVID-19 pandemic, it is premature to examine data from YES students to determine if that finding holds. However, competing in these fairs provides a platform to discuss their work with other scientists and be a part of the larger scientific community. Program Directors mentor students through the science fair process by giving workshops about once a month on how to write about and present their projects. They also work directly with students and mentors to provide critical feedback on student compositions. Several of our students have won prizes at these fairs. Students are compensated for their work in the academic-year program like they are for their summer research. Additionally, all students who do not have a laptop are given a loaner for the duration of the program. The loaner laptops are loaded with all necessary software for videoconferencing and designing presentations. We also purchase network cards for students that do not have reliable internet access in their homes and bus passes when needed to attempt to remove barriers that might hinder students from participating in the program. The COVID-19 pandemic shut down labs throughout the University of Pittsburgh. Although dramatically impacted, computational research continued. Thus, we pivoted from our normal structure of six research sites on campus and instead leaned on our strengths in computational research projects to provide a remote mentoring setting for students willing and interested to engage in that way. Students were paired with mentoring teams that were enthusiastic to work remotely with the students. A technology survey was sent to each student, and we provided laptops and internet access to those in need. One-on-one mentorship occurred via Zoom between the students and faculty/trainee teams full-time for seven weeks. Additionally, we hosted all didactic and professional development activities through Zoom and connected the students socially through Discord. We also constructed a team of eight Hillman Academy alumni that were in college to help build social capital and provide advice through the summer via the Discord server. Furthermore, a team of 10 individuals with strong computational background were employed to serve as coding teaching assistants who hosted office hours throughout the summer and were also in the Discord server. The intent was to replace the constant access to graduate students and postdocs that the students would normally have if they were in-person. We also increased the number of lectures and other activities to maintain significant contact with the students and the Program Director and Coordinator scheduled weekly check-ins with all students. The summer ended with each student giving a 10-minute talk on their work after being introduced by their mentor. This event, which was open to the public, was held on Zoom and broadcasted more broadly on our YouTube page. More information about the pivot to virtual programming including assessment data was reported elsewhere ( Fung et al. 2021 ). The ongoing pandemic impacted us again in the summer of 2021. We were permitted to have students in labs, but because of University of Pittsburgh regulations, were not allowed to host in-person meetings with the students and could not use university housing. Because of this, we pivoted once again to a hybrid program. Traditional wet-lab research internships were conducted as they were prior to the pandemic with each student commuting to their host mentor’s lab each day. All computational projects were conducted remotely as outlined above during 2020. As always, all students were paired with mentoring teams that met either in-person or virtually through Zoom. Remote students in 2020 ranked our group talks and events higher than in any previous year. In response, we hosted a one-hour talk, social, or professional development event each day of the seven-week program. We asked the students performing research remotely to attend daily to build cohesion but permitted students that were in labs to attend only two of the events per week to maximize their time in lab (which we know from previous years is the most important aspect of the program). All events happened via Zoom and when the speakers agreed, these events were recorded and posted to our public YouTube channel. Once again, the students presented final talks at seven concurrent Zoom sessions, most of which are available on our YouTube channel. At the Hillman Academy, we attempt to use culturally responsive pedagogy and provide students with opportunities to learn about or engage in research that affects them and their community. As an example, here, we highlight the work of our Deaf and Hard of Hearing (Deaf/HoH) students and their mentors in our YES program . People who identify as Deaf/HoH are highly underrepresented in research ( Lynn et al., 2020 ). For example, only 1.3% of the 39,435 doctorates awarded in 2017 went to people who were Deaf/HoH ( Jackal, 2019 ; National Center for Science and Engineering Statistics, 2019 ). Moreover, of those doctorates to Deaf/HoH adults, only 40% were earned in life sciences, physical sciences, or engineering compared to 78% of the doctorates to hearing individuals ( Solomon CM, 2012 ). Deaf/HoH students face significant obstacles to advancing in STEM, including issues in accommodations, self-advocacy, and establishment of a support system ( Listman & Dingus-Eason, 2018 ; Madhusoodanan J, 2016 ; Solomon CM, 2012 ). The extent to which these obstacles present barriers varies greatly because Deaf/HoH students are highly diverse in many respects, including the etiology of deafness, possible benefits of auditory interventions, necessary exposure and acquisition to American Sign Language (ASL) and English, and access to accommodations ( Solomon CM, 2012 ). Deaf/HoH students whose primary language is ASL with a natural interest and acumen in science and math may face challenges in these disciplines where English is the dominant language. Students are often not given the opportunity to become bilingual by learning the English language through age-appropriate instruction starting at a young age. This limits the students’ access to information in English and reduces confidence and the ability to develop independent learning skills in science that are needed to advance skills in research. These challenges are exacerbated due to the national shortage of interpreters, and specifically, the lack of sign language interpreters who specialize in certain disciplines with specific jargon. Also, there are relatively few signing teachers of science, and many science teachers are unaware of appropriate methods for teaching Deaf/HoH students The Hillman Academy YES program has sought to address these obstacles to the advancement of future Deaf/HoH scientists by bringing together experts in Deaf studies, artificial intelligence, communication science, and biomedical informatics from both Pitt and Gallaudet University, which specifically caters to Deaf/HoH students. The collaboration began in 2018 with the goal of increasing the inclusion of Deaf/HoH students in biomedical science. Since 2018, the program has involved a total of 10 high school students who identify as Deaf/HoH. Two students have returned for multiple years and one student is continuing to receive research mentorship from faculty on the Hillman Academy team as an undergraduate at the University of Pittsburgh. Including Deaf/HoH students requires special recruiting and preparation. One of the first activities to recruit is to create and send by email, social media, and mail a promotional flyer that includes links to an informational video in ASL and to the program’s website and application. The team also works closely with the Office of Disability Resources and Services at Pitt to schedule ASL interpreters for all didactic sessions during the summer and ensure all facilities have appropriate accommodations. Mentors whose first language is English work out a mentor–mentee meeting schedule in advance of the program to ensure that there is adequate interpreter coverage. However, for the first time in 2021, two Deaf students worked with mentors from Gallaudet University who were Deaf themselves or teach primarily in ASL. We did learn from this experience that interpreters should still be included within these workgroups so that the interpreters can learn about the research to effectively voice for the Deaf students in group activities with hearing individuals. Finally, the team invites scientists from Deaf/HoH communities to give keynote talks to all students in the program. These scientists also meet directly with the Deaf/HoH students for a career-focused discussion. While the Deaf/HoH students have worked on a variety of topics, one project that has been advanced by these students each year is a prototype learning aid designed for Deaf/HoH students to use while reading scientific content on a web page or watching a video. Figure 5 shows screenshots of the working prototype and features designed to reduce language obstacles that Deaf/HoH face when attempting to learn from scientific content written in English. The tool is a web browser plug-in that presents the user with ASL videos and pictures expanding on science jargon in English. The primary feature of the tool is that certain English words within the material are highlighted and clickable. Clicking on a highlighted word updates a pane on the bottom of the web page that will show 1) a short and carefully written English definition, 2) one or more images about the concept that the word is about, and 3) a brief ASL video that finger spells the word and summarizes the concept. Prior work by students on the prototype has demonstrated proof of concept for the technology. Currently the study team consisting of a Deaf Gallaudet undergraduate and a Deaf Hillman Academy YES program alumnus is conducting a formative evaluation of the approach to test if its use is associated with an increase in the accuracy of answers to Advanced Placement Biology questions by Deaf students.

For 13 years, the Hillman Academy has provided unique foundational training, educational, and development opportunities to diverse groups of students with an emphasis on reaching students from URM/DA backgrounds and preparing them for successful careers in STEM. Students in the Hillman Academy are immersed in mentored, cutting-edge research projects that are the centerpiece of the program. Complementing the research experience is a comprehensive curriculum that covers essential knowledge and skills that help develop our students scientifically, professionally, and personally. YES funding, combined with our innovative structure with 6 specific research sites/foci led by 6 different faculty, allowed for our program to grow from 6 to 70 students who participate in a specialized site-specific curriculum that complements their individual research projects. It also simultaneously allows for the students to network across a large number of peers through the whole program (~70 students), while developing deep bonds and having small local learning environments within their sites that contain ~10 students each. Likewise, our innovative new recruiting plan with guaranteed admission to students from key schools and community partners who focus on diversity, equity, and inclusion, has more than tripled the number of URM/DA students applying to and attending the Hillman Academy. This has led to a truly diverse set of students from all backgrounds (based on race, ethnicity, socioeconomic status, and disability) who are underrepresented in cancer research. Our newly incorporated YES program, which began in September of 2019, adds an additional academic-year curriculum that engages students in additional professional development that helps prepare them for scientific presentations at formal scientific conferences as well as for award and college applications. Hillman students report significant gains in key areas related to thinking and feeling like a scientist and have almost exclusively pursued further degrees in STEM fields. Our program has also forged important community partnerships that dramatically increased the number of URM/DA students, with an additional emphasis on Deaf/HoH students, participating in our program. Lastly, the Hillman Academy, together with its many institutional and community partners, aims to continue its ongoing mission of training the next generation of diverse scientists, physicians, teachers, and others working in STEM, and positioning them to be change leaders in their field.

The University of Pittsburgh Medical Center (UPMC) Hillman Cancer Center (HCC) Academy (Hillman Academy) mentors and trains high school students, mostly from underrepresented minority and disadvantaged (URM/DA) backgrounds each summer in an immersive, 8-week, cancer-focused, research program complemented with a host of professional development and enrichment activities throughout the year. These efforts are geared towards increasing the diversity of the scientific and medical workforces and addressing the disparities that exist in academia, medicine, and for certain diseases such as cancer, which hits URM/DA communities disproportionately with relatively few members of this community representing the scientists and physicians who study and treat it. This exclusion has led to a mostly homogeneous clinical and biomedical workforce that is lacking in diverse perspectives and contributions, fails to take advantage of the many benefits seen with diverse teams and organizations ( Cohen et al., 2002 ; Freeman & Huang, 2015 ; Hong & Page, 2004 ), and exacerbates health disparities in the excluded groups, especially when it comes to treating diseases like cancer that affect minority and economically disadvantaged individuals at different frequencies ( SEER, 2017 ; Singh & Jemal, 2017 ; Talcott et al., 2007 ; Ward et al., 2004 ). Over the past 13 years, the Hillman Academy has trained 239 URM/DA students, nearly all of whom have gone on to pursue undergraduate degrees. Hillman Academy scholars have also had other successes during their time in the program and beyond, such as making contributions to published manuscripts, presenting at national conferences, and winning state and regional science fairs. The Hillman Academy itself was recognized with accolades including the Carnegie Science Center Leadership in STEM Education award and being named as one of the top 100 global education innovations by Hundred.org . There are many reasons for the growth and success of the program, but primary among them is the dedication of the faculty and senior trainees who make it their mission to help train and prepare a diverse, next-generation workforce, as well as the strong base of support from our partners including local schools and community-based organizations. Funding from individual donors, charitable foundations, and government agencies were essential in providing the framework and support for financially supporting students who are largely left out of opportunities like this at the high school level. The NIH and the National Cancer Institute (NCI) recognized the disparities in health outcomes, education, and training efforts and created the Youth Enjoy Science (YES) program to help train and sponsor educational programs that aim to broaden participation by providing educational and hands-on training opportunities to URM/DA students. Other federal (ex. NSF, NIH) agencies and foundations (ex. Doris Duke Charitable Foundation, Hillman Foundation, Pittsburgh Foundation, etc.) have funded similar efforts because studies demonstrate that impactful research experimentation tethered to mentorship, professional skills development, and protected learning environments are instrumental to increasing the number of UMR/DA students in research and STEM careers ( Fung et al 2021 , Kobrak, 1992 ; Campbell, 1997 ; Maton et al., 2000 ; Nakamura and Shernoff, 2009 ; Kim and Sax, 2011 ; Noy and Ray, 2011 ; Toven-Lindsey et al., 2015 ). With successful internship programs building students’ confidence, science identity, research skills, and scientific networks ( Toven-Lindsey et al., 2015 ). Publications examining the impacts of undergraduate internship programs are more common ( Lopatto 2007 , Lopatto 2004 , Junge et al. 2010 ), but examples from high school interventions show similar benefits ( Page et al. 2019 , Eeds et. al. 2014 , King et. al 2015)) and literature suggests most students who major in STEM make the decision during high school ( Norris and Agodoa, 2006 ) highlighting the importance of precollege STEM programs. In this paper, we provide a description of the Hillman Academy, focusing on its uniqueness in breadth, size, diversity of student populations including students from the Deaf and Hard of Hearing community, longevity, recruitment strategies, assessment, and how we cultivated, evolved, and expanded our program to ultimately become an NCI YES-supported program. More information about the steps to implement a program like ours was previously described ( Livshits, S. and Boone, DN 2020 ). Other models of YES programs can be found in accompanying articles within this special edition.

Over the past 13 years, the Hillman Academy has provided 294 paid internships to 239 URM/DA students. In addition to these URM/DA students, the Academy has also provided an additional 286 unpaid internships to 267 students from non-URM/DA backgrounds. Many of these URM/DA scholars now attend highly selective universities and colleges including Harvard, Yale, Princeton, Stanford, and MIT. Excitingly, approximately 90% of URM/DA Hillman Academy alumni with known and declared majors are in STEM fields, including biological sciences, chemistry, engineering, and pre-medicine. Of the 182 underrepresented Hillman Academy alumni that have graduated high school, we know that at least 176 matriculated into higher education. Of the 164 underrepresented Hillman Academy alumni that we know have declared a major, 151, or 92%, are in STEM or healthcare-related fields. This excludes fields considered STEM by the NSF such as Political Science, Economics, and Sociology which are most of the majors of the “non-STEM” alumni. While this evidence suggests the Hillman Academy at least perpetuates an interest in STEM leading to the high proportion of students in these majors, we are cautious in our interpretations. A formal causal evaluation of the longitudinal impact of the Hillman Academy on persistence in STEM is ongoing and will be reported in a future publication after more Hillman Academy students progress into, through, and beyond undergraduate education. In addition to the longitudinal assessment, we assess the summer research component each year using a modified pre/post survey ( King et al., 2017 ) adapted from 2 published sources—the Summer Undergraduate Research Experience (SURE) and one used by the School for Science and Math at Vanderbilt ( Eeds et al., 2014 ; Lopatto, 2004 , 2007 ). These surveys are administered online through Qualtrics, are anonymous, and are completed before the start of the summer program and during/after the end of the research symposium. Each question can be skipped if a student chooses not to answer, and the post survey includes some questions with a ‘retro-pre’ evaluation of the student where they are asked to rate themselves at the beginning of the program now that they have had the experience. This has been used by others ( Nimon et al., 2011 ) as students often overestimate themselves before educational experiences. We saw similar results that retro-pre ratings were often lower than true pre ratings and therefore rely on the retro-pre in this analysis. Given the vast changes in programming due to the COVID-19 pandemic that began just a few months after the starts of our YES program, we are hesitant to make sweeping judgements based strictly on these surveys. To increase the sample size, here we report results of a similar survey administered to all URM/DA Hillman Academy students since 2016. At the Hillman Academy, we hope to provide authentic research experiences and career development opportunities to clarify what a scientist is, what a scientist does, and how to become a scientist, in order to broaden participation in the biomedical workforce. Of the 163 URM/DA students that responded to the surveys, 96% reported that their idea of a career scientist was clearer after the Hillman Academy with over 83% stating they have a ‘good’ or ‘very clear’ idea of what a scientist is. The students were asked, both before and after the program, about what their plans were after undergrad. The largest changes were an increase in ‘pursuing a PhD in a STEM field’ and a decrease in ‘undecided’ or ‘have not considering grad school ( Table 4 – highlighted in red ).’ Moreover, 94% of students said they are ‘likely’ or ‘very likely’ to seek another research experience. Together these results suggest the Hillman Academy is successful at influencing students plans to pursue paths leading to career scientists. Importantly, these numbers were very similar during the pandemic-influenced programs ( Table 4 left vs right ). In 2020, the Hillman Academy was held completely virtually. The students still completed authentic research projects under the mentorship of faculty and trainees, but only in computational projects that could be completed entirely remotely. In 2021, the Hillman Academy was a hybrid program where half of the students were in labs on campus and the other half completed their computational research at home as in 2020. During the 2021 program, all didactic activities also happened via Zoom as University policy prevented students from being together in groups. For both of these years, we saw similar encouraging results for students self-perceived ratings as in previous years, which we previously reported in 2020 ( Fung et al., 2021 ). For example, on the post survey, we asked students to rate themselves on a scale of ‘1 - I know nothing’ to ‘10 - I am an expert’ on a series of skills important to becoming a scientist. Positive gains were reported in all areas with the largest gain in mean change per student (> 2.61 +/− 0.18) seen in the categories of ‘communicating with scientists using a scientific vocabulary,’ ‘gathering and organizing data through the use of scientific tools and technology,’ and ‘critically analyzing the quality of data generated in my own research’ ( Table 5 ). The only appreciable difference between the 2016–2019 cohorts and the pandemic 2020–2021 cohorts was a larger gain in ‘using software to create scientific models to make predictions.’ This is likely due to an increased proportion of students completing computational projects given that all students in 2020 and half of the students in 2021 engaged in computational research, but only 33–40% in the pre-pandemic cohorts. When we asked the students about their experience in the Hillman Academy and to rate different components, they overwhelmingly indicated that working with their mentor is the most rewarding and important part of the program. 96% of students rated working with their mentor as a 4 or 5 on a Likert scale of ‘1 - Useless’ to ‘5 - Very Useful’. This was clear from themes extracted from open-ended questions as well. The second highest rated component of the program was career and research talks (86% scoring 4 or 5; average = 4.3). During these career talks, we try to expose students to successful scientists of similar backgrounds. These talks are typically engaging and personal and are rated higher than straight didactic lectures/lessons. Finally, in a question with a Likert scale of ‘1 - Not True’ to ‘5 - Very True,’ we asked students about their experience and possible gains. Table 6 shows that most students responded with a 4 or 5 for all questions. The statements with over 90% of students responding 4 or 5 were: ‘I would recommend the Hillman Academy to other students,’ ‘I learned new concepts and skills in science,’ ‘I increased my proficiency at using materials, equipment, or technology in performing job tasks’, and ‘my research experience enhanced my critical thinking-skills’. These results suggest that the Hillman Academy is increasing students’ skills important to becoming a successful scientist.