Evaluating the Impact of a Dutch Sexual Health Intervention for Adolescents: Think-Aloud and Semi-Structured Interview Study

Background: Engagement with and potential impact of web-based interventions is often studied by unobtrusively tracking user behavior with web analytics. These metrics do provide insights into how users behave, but not why they behave as such. Objectives: This article demonstrates how a mixed-methods approach consisting of (i) a theoretical analysis of intended use, (ii) a subsequent analysis of actual use, and (iii) a further exploration of user perceptions can be used to gain insight into engagement with and potential impact of web-based interventions. This paper focuses on the exploration of user perceptions. The chlamydia page of the Dutch sexual health intervention Sense.info is used as a demonstration case. This prevention-focused website serves as the primary platform for young people aged 12 to 25 in the Netherlands for information (and care if needed) regarding the whole spectrum of sexual and reproductive health. Methods: First, Acyclic Behavior Change Diagrams (ABCDs) were used to carry out a theoretical analysis of intended use of the chlamydia page. ABCDs display how behavior change principles are applied in an intervention and which sub-behaviors and target behaviors are (aimed to be) influenced. This analysis indicated that one of the main aims of the page is to motivate STI testing. Second, actual use of the chlamydia page was analyzed with the web analytics tool Matomo. Despite the page’s aim of promoting STI testing, a relatively small percentage of the 35,347 transfers from this page were to the STI testing page (14%). Based on these two phases, preliminary assumptions about use and impact were formulated. Third, to further explore these assumptions, a study combining the think-aloud method and semi-structured interviews was executed with 15 young individuals aged 16-25 (Mage = 20, SDage = 2.5). Template analysis was used to analyze interview transcripts. Results: Participants perceived the information present on the Sense.info chlamydia page as reliable and would visit it mostly for self-diagnosis purposes if they would experience symptoms of a potential STI transmission. A perceived facilitator for STI testing was the possibility to learn about the symptoms and consequences of chlamydia through the page. Several barriers were mentioned, such as an easily overlooked link to the STI testing page and use of language not meeting the needs of participants. Participants offered suggestions for lowering the threshold for STI testing. Conclusions: The mixed methods approach used provided detailed insights into the engagement with and potential impact of the Sense.info chlamydia page, as well as strategies to further engage end-users and increase the potential impact of the page. We conclude that this approach, which triangulates findings from theoretical analysis with web analytics and a think-aloud study combined with semi-structured interviews, may also have potential for the evaluation of web-based interventions in general.