Improving the Process & Product of Intensive Intervention Through Formative Triangulation

Improving the Process & Product of Intensive Intervention Through Formative Triangulation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Improving the Process & Product of Intensive Intervention Through Formative Triangulation Joseph Lambert, Angel Fettig, Jennifer Ledford This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7005035/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 14 You are reading this latest preprint version Abstract Service-delivery models that promote collaboration with families and systematically program for generalization may produce socially significant and generalized outcomes. However, valid empirical demonstrations of generality are difficult to generate. For practitioners invested in evaluating the molar impact of their effort, quantitative measures reflecting caregiver perception of change could be a valuable metric for guiding formative triangulation processes. In this paper, we presented a fully integrated mixed methods case study using convergent-design elements to demonstrate how incorporating qualitative and quantitative reports of caregiver perceptions of the frequency and intensity of aggression outside of formal therapy sessions can inform socially valid and generalized outcomes through formative triangulation. Generalization functional analysis intensive intervention data based decision making social validity Figures Figure 1 Introduction Formative assessment has been described as a process (as opposed to a product) by which observers use repeated measures (e.g., direct observation) to sample learner performance throughout the entirety of a treatment program; as opposed to summative assessment, which is designed to evaluate outcomes at the end of a program. The purpose of formative assessment is to offer tentative and ongoing empirical evidence of the effectiveness of a prescribed treatment plan, and to justify adaptation when learners prove non-responsive to the program-as-intended (Boston, 2002 ). Relevant to this point, behavior analysts are ethically bound to engage in data-based iterative instructional design. Undesirable or “counter-therapuetic” behavior patterns should lead to adaptations to treatment programs that are justified by operant learning theory (Behavior Analyst Certification Board, 2020 ; Lambert, Paranczak et al., 2022 ). That is, procedural modifications are triggered by poor empirical outcomes or undesirable trajectories, are informed by a theory of change, and are capable of producing new empirical outputs which allow for direct evaluation of the success of the adaptation (Robertson et al., 2022 ). Thus, almost by definition (e.g., Bennett, 2011 ), Board Certified Behavior Analysts (BCBA) are specialists in formative assessment processes. Limitations of Empirical Systems Based in Direct Observation Boundedness is a term which has been used to describe the extent to which patterns of behavior change documented by researchers extend beyond the research context and into socially valid domains (Yoder et al., 2018 ). Although the term is useful for describing socially invalid behavior change (i.e., context-bound demonstrations of effect not relevant to day-to-day life), it is also useful for labeling data systems which are incapable of evaluating such generality—those that fail to assess the construct altogether, as well as those whose methodology implicitly precludes valid assessment (see also the concept of ecological validity; Fahmie et al., 2023 ). Contemporary approaches to measurement in single case design render it difficult to produce compelling demonstrations of generality. That is, strategies for ensuring measurement integrity and validity (e.g., interobserver agreement, frequent reliability checks) are often carried out by agents (e.g., research assistants) not relevant to typical contexts (i.e. practitioners such as BCBAs). Because the presence of external observers can serve discriminative functions for both treatment implementation and for challenging behavior, demonstrations of generality documented in the presence of such observers tend to lack validity (Ledford et al., 2023 ). As a result, these, and other, problems associated with measurement have plagued single-case research for decades (cf. Kazdin 1979 ; Kennedy, 2002 ; Halle, 2019 ). Unfortunately, the generality and social validity of context-bound demonstrations of effect which are typical to peer-reviewed publication (i.e., with tightly controlled parameters and high internal validity), often go unassessed and/or unreported in the literature (Carr et al., 1999 ; Conroy et al., 2005 ; D’Agostino et al., 2019 ; Ferguson et al., 2018; Ghaemmaghami et al., 2021 ; Hurley, 2012 ; Ledford et al.; 2016 ; Park & Blair, 2019 ; Shepley et al., 2025 ; Snodgrass et al., 2018 ). As a result, their value to effective therapuetic processes is relatively unknown (Ghaemmaghami et al., 2021 ). That is, measurement limitations and design oversights ubiquitous in intervention research have made it difficult to distinguish practices which promote socially valid and generalized outcomes from those that merely produce context-bound demonstrations (Sandbank et al., 2021 ). Relevance to Intensive Intervention for Severe Challenging Behavior Importantly, the problem posed by boundedness extends beyond research, and into effectiveness appraisals of any scientist-practitioner who leverages rigorous empirical methodologies in their own practice (e.g., BCBAs). For example, effective approaches to intensive intervention for challenging behavior often entail therapies which begin in analogue contexts explicitly designed to accentuate the intensities and salience of relevant controlling variables targeted for analysis (e.g., fixed ratio schedules during functional analysis; cf. Beavers et al., 2013 ; Herzinger & Campbell, 2007 ; Hurl et al., 2016 ). After desirable response patterns have been established via intervention, interventionists then begin a systematic process of stimulus shaping and fading (e.g., programming common stimuli, sequential modification) to ensure that therapeutic effects generalize to the circumstances under which they are needed (i.e., the unsystematic presentation of establishing operations by careproviders and other stakeholders across a variety of typical settings; Lambert, Copeland et al., 2022 ). Despite the potential of these interventions to improve patterns of challenging behavior, the generality and social validity of established gains is not automatically entailed in the process and there are circumstances under which favorable empirical outputs are challenged by caregiver reports which indicate the data are invalid (Lambert, Sandstrom et al., 2022 ). Notwithstanding, some context-bound demonstrations of effect remain useful. On the one hand, orderly demonstrations in controlled contexts (i.e., those contrived during formal therapy) hold substantial value to formative assessment and intervention processes. Specifically, terminal stages of intensive intervention are often designed to mirror the lean and unpredictable contingencies common to typical day-to-day contexts (e.g., Muharib et al., 2021 ). However, success in these latter stages is the product of systematic shaping and is predicated on a participant’s success with less complex and more predictable contingencies arranged during initial stages of therapy, when opportunities to respond are presented in a massed trial, session based, single-operant paradigm. Interventionists who advance from initial stages to latter stages of therapy without first obtaining compelling evidence of initial-stage successes are likely to encounter relapses in challenging behavior and an overall degradation of treatment effects (Kimball et al., 2023 ; cf. Hanley et al., 2014 ; Volkert et al., 2009 ). Thus, measurement systems capable of detecting responsiveness to intervention during initial development and validation processes hold clear value. On the other hand, response patterns established across small periods of time (e.g., 5-min sessions) are not always representative of patterns under similar circumstances across larger periods of time (e.g., 2-hr sessions; Lindberg et al., 2003 ). Likewise, variables that control responding in analogue settings may not be present in typical ones (Ledford et al., 2016 ). Thus, it can be appropriate to question the social validity of context-bound demonstrations which are ubiquitous in intervention research (e.g., Ghaemmaghami et al., 2021 ; Lambert, Sandstrom et al., 2022 ; Sandbank et al., 2021 ). For interventionists invested in assessing the generality and social validity of their programming in ways which are unconstrained to contexts which happen to be convenient to rigorous measurement systems, triangulating across complementary data sources can help ascertain the degree to which desired outcomes established in contrived scenarios (e.g., formal therapy sessions) translate into socially significant improvements under typical circumstances (i.e., those large enough to be noticed by those who originally sought services). Consequently, triangulation can provide critical information which can be used to qualify and substantiate empirical demonstrations of effect established in more controlled settings (Kazdin, 1977 ; Schwartz & Baer, 1991 ). Triangulation In trigonometry, triangulation is a strategy by which the specific location of a point-of-interest is deduced based on its geometric relation to other points-of-interest with known locations (Triangulation, 2025 ). Qualitative and mixed-methods researchers who adopt a pragmatic philosophical lens and employ a similar principle when they enlist multiple methods and data sources to enhance their understanding of a phenomenon-of-interest (Denzin, 2012 ; Patton, 1999 ; Creswell & Clark, 2017 ). One goal of triangulation is to overcome the limitations intrinsic to any single methodological approach or measurement system (Kimchi et al., 1991 ). For example, to increase confidence in the extent to which a study has confirmed a proposition—behavior change has occurred and has generalized to all relevant contexts outside of the constraints of therapy, and is thus socially valid—a mixed methods researcher would use different approaches to gather different kinds of data to evaluate the extent to which socially valid generalized outcomes occurred, and then would triangulate outcomes across methods to identify commonalities across measures, time points, spaces, and perspectives (Creswell & Clark, 2017 ; Heale & Forbes, 2013 ; Kimchi et al, 1991 ). When the results of all methodologies converge (i.e., lead to the same conclusions), there can be more confidence in statements about the generality of treatment outcomes. By contrast, when they diverge (i.e., conclusions contradict across planes of analysis), a need for problem solving arises (in order to reconcile paradoxical outcomes) and the product of this process is often a richer understanding of the phenomenon of interest. Although many types of triangulation have been described in the literature (Patton, 1999 ; Kimchi et al., 1991 ), methodological triangulation may be most relevant to the problem of measuring generalized and socially valid behavior change. Specifically, in this triangulation approach, researchers use more than one method to acquire data about a phenomenon of interest. The intent is to identify gaps or inconsistencies in the conclusions drawn from data derived from either methodology in isoloation, and then shore up said limitations with the strengths offered through the alternative methodology(ies). Within-methods triangulation entails combining similar data-collection approaches to measure the same variable of interest (e.g., frequency counts of individual instances of aggression during environmentally controlled 5-min sessions collected by researchers during formal therapy sessions vs. frequency counts of bouts of aggression collected by parents in uncontrolled naturalistic contexts across the entire day). Between-methods triangulation entails comparing data across both quantitative (e.g., frequency counts, intensity scales) and qualitative approaches (e.g., field notes; open-ended narrative reports). Multiple triangulation describes research which employs more than one type of triangulation (e.g., within- and between-methods triangulation; Kimchi et al., 1991 ). Multiple Triangulation as a Formative Process During Intensive Intervention Although triangulation is typically discussed within the context of research, its relevance to practice can’t be overstated. For example, Lambert, Sandstrom et al. ( 2022 ) recently described case studies in which their research team served two clients during the same calendar year. These clients shared similar diagnoses and relatively similar topographical (i.e., aggression) and functional (i.e., escape, escape & tangible) profiles for challenging behavior. In both cases, functional analysis (FA) unambiguously confirmed controlling variables and function-informed treatments achieved a commonly evoked standard of high efficacy (i.e., better than 90% reductions in challenging behavior, cf. Saini et al., 2019 ). However, surveys designed to explore the social validity of the treatment model (i.e., Lambert, Copeland et al., 2022 ; Lambert, Paranczak et al., 2022 ) suggested that demonstrations of effect established during formal therapy generalized and maintained for one client (i.e., BAC 7 in Lambert, Copeland et al., 2022 ), but not the other (i.e., BAC 4 in Lambert, Copeland et al., 2022 ). One apparent difference between these two cases rested in each team’s responsiveness to circumstantial evidence. In the case of BAC 4, interventionists reported being fairly inflexible in their treatment approach and only considered data obtained through the measurement system contrived by the research team during formal appointments (hereafter referred to as the “supervised context”). In the case of BAC 7, the research team reported being more responsive to parent reports of events which occurred when the team was not present (hereafter referred to as the “unsupervised context”), and more intentionally engaging in an iterative process of multiple triangulation and treatment design which was responsive both to objective data obtained in the supervised context, as well as subjective reports of what occurred in the unsupervised context (i.e., “formative triangulation”). Despite the clear utility of using stakeholder input to modify planned interventions, and the supposed value of this type of analysis in applied behavior analysis (i.e., socially important), these data are not typically included in published behavior analytic work, eschewed for more objective direct measures in tightly controlled situations. The omission of these types of ongoing and formative data has been the subject of recent informal (e.g., blogs, social media) and formal critiques of behavior analysis (Penney et al., 2023 ; Pritchett et al., 2022). Mixed Methods Case Study Reports As noted above, when considered in isolation, empirical demonstrations of effect common to the formative assessment processes of behavior analysts are often limited in their capacity to assess generalized and socially valid behavior change. This in turn, has sometimes limited interventionists’ abilities to engage in responsive programming. Formative triangulation apparently offers a practical solution to this problem which honors the strengths of traditional single-case design while also addressing its limitations. Stated differently, exigencies of highly responsive clinical practice are often at odds with contemporary exigencies of evidence in single case design. Further, individuals who engage in severe challenging behavior often represent low incidence and heterogonous populations, and treatment design is highly individualized. Thus, valid replication of treatment effects at the level of complexity required by practicing behavior analysts has often escaped the clutches of single case researchers (Carr et al., 1999 ; Durand et al., 2008; 2010; Lambert, Copeland et al., 2022 ). With that said, recent calls for dynamic evaluative criteria in single case design (i.e., Ledford et al., 2023 ) make space for consumers of research to adjust standards of rigor in ways which prioritize protecting against threats to validity most likely to challenge the primary contributions of a paper, while marginalizing unlikely threats that might impede a project’s ability to meaningfully explore critical topics (e.g., generality, social validity). In the spirit of this guidance, the purpose of this project was to expand, through mixed methods case study analysis, dialogue about the potential value of formative triangulation to socially valid intevention programming. Method The BAC (not an acronym; pronounced B-A-C) was a university-sponsored practicum experience designed to provide pre-service practitioners of special education and applied behavior analysis with exposure to a focused process of intensive intervention for severe challenging behavior (i.e, Lambert, Paranczak, et al., 2022 ). Services were provided by a team comprised of four interns (i.e., graduate-level students studying special education and behavior analysis) and one Board Certified Behavior Analyst. Standard goals for the experience entailed: (1) conceptual clarity about the functions of challenging behavior (accomplished through functional analysis), (2) response elimination (accomplished through functional communication training paired with extinction), (3) tolerance of brief but mandatory exposures to challenging behavior’s establishing operations (EO) (accomplished through schedule thinning organized into compound-schedule arrangements), and (4) generalization (achieved by incorporating typical implementers into intervention sessions held in typical settings). All deviations were constrained to well-understood operant learning mechanisms (see Supplemental Figs. 1–5) and were deliberated by team members between formal appointments (Lambert, Copeland et al. 2022 ). Deliberations adhered to a number of guiding principles (Supplemental Tables 1 & 2) and a general problem-solving framework (Supplemental Fig. 5). Research Design Using a pragmatic philosophical lens and guided by the baseline assumptions and guiding principles described in Supplemental Table 1, we conducted a fully integrated mixed methods case study design to generate a case report in which our quantitative (freqeuency counts in supervised and unsupervised planes of analysis in the context of single case designs) and qualitative (open-ended parent reports, field notes) data were collected and compared multiple times per week across a three month period (Creswell & Clark, 2017 ). Within the context of the overarching program evaluation (see Lambert Paranczak et al., 2022; Lambert, Copeland et al., 2022 ; Lambert, Sandstrom et al., 2022 ); this study can be classified as an emergent (as opposed to fixed) design, because the introduction of formative triangulation occurred in response to clinical oversights described in detail in Lambert, Sandstrom et al. ( 2022 ). That is, intervention methodology evolved across the relevant time frame in response to clinical need and was not set (fixed) at the onset of the overarching project. Participant and Setting Jared (“BAC 32” in Lambert, Copeland et al., 2022 ) was a 10-year-old White male, reported by his mother to be diagnosed with KBG syndrome, Attention-Deficit Hyperactivity Disorder (ADHD), Speech Sound Disorder, Anxiety Disorder (unspecified), and a sleep disorder. We chose to highlight Jared’s case in this report because he represented a case for which 1) data were collected in both supervised and unsupervised planes of analysis, 2) conclusions about treatment effectiveness diverged across planes of analysis while services were being provided, and 3) the treatment team proved responsive to this divergence and meaningfully incorporated formative triangulation in a collaborative process which resulted in eventual convergence across data sources. For context, Jared had previously received behavior analytic services from BCBAs. Previous intervention plans were based on a purported attention function and included differential reinforcement, planned ignoring, and choice-making opportunities. At the time of the study, Jared was on summer break, did not attend school during assessment or intervention, and spent most of his time with his family at home or in the community. Jared could fluently speak English using full sentences. All stages of therapy occurred in Jared’s home. Initially, appointments were held exclusively in his bedroom after considerable environmental re-arrangement (e.g., removing furniture, securing breakables). Later, appointments were conducted across the home with no environmental re-arrangement. Dependent Variables We concurrently tracked the frequency of instances (supervised) or bouts (unsupervised) of Jared’s aggression (hitting, kicking, scratching, choking, pinching, spitting, pushing, and property destruction) across supervised and unsupervised contexts throughout all phases of this study. We also tracked the episodic severity (i.e., above average, average, below average) of the most intense instance of Jared’s aggressive behavior in the unsupervised context (cf. Lavigna & Willis, 2005 ; Lavigna et al., 2012), as well as qualitative data on contextualizing variables for bouts of aggression reported in the unsupervised plane of analysis. Data from supervised contexts were collected by trained research asssistants during formal appointments. Data from unsupervised contexts were reported by Jared’s mother. Prior to study onset, Jared’s mother was trained on all operational definitions. Measurement in the Supervised Context In the supervised context, trained observers collected frequency data using continuous recording procedures on iPods with data-collection software. Sessions were initially 5 min and eventually increased to 10 min to accommodate increases in the durations of response requirements during schedule thinning. Interobserver agreement (IOA) . A second trained observer independently collected reliability data across assessment and intervention phases in the supervised context. To calculate IOA, we divided each session into 10-s bins and scored agreements between primary and reliability observers about the the frequency of aggressive behavior scored within each bin. We then divided agreeements by the sum of agreements and disagreements and multipled by 100, averaged bin scores across the session, and reported a single percentage-agreement score for each session evaluated (Cooper et al., 2020 ). Mean IOA during Jared’s FA was 99.9% (range 98–100%) and was calculated for 37.5% (9 of 24) of his FA sessions. Mean IOA during Jared’s supervised intervention was 98% (range 71.7–100%) and was calculated during 45.9% (73 of 159) of Jared’s supervised intervention sessions. Measurement in Unsupervised Context We asked Jared’s mother to report unsupervised data using a low-tech (paper and pencil) data-collection system. In addition to asking Jared’s mother to report the frequency and intensity of problem behavior in unsupervised contexts, we asked her to contextualize her reports of aggression and identify topics she wished to discuss during subsequent appointments in an open-ended “notes” section. That is, we developed a data system that allowed Jared’s mother to answer three questions during every day of the consultation: 1. How many bouts of problem behavior were observed today? 2. What was the intensity of the most intense bout observed today? 3. What else do you want us to know? We asked Jared’s mother to begin reporting the day before therapy in the supervised context began. To protect against retroactive reporting, her team consulted her data sheet with her during every appointment, and Jared’s case manager regularly contacted her between appointments to ask about Jared, and to obtain a description of the reports she had written into the paper-and-pencil system. Throughout the duration of this study, Jared’s mother reported on his behavior in the unsupervised context across 100% (57 of 57) of opportunities. Procedural Fidelity We used a whole interval recording system to assess fidelity to programmed procedures during sessions conducted in the supervised context. Specifically, we divided each session into 10-s bins and evaluated adherence to programmed setting events and contingencies on a bin-by-bin basis. Bins in which all procedures were implemented as intended were scored as “yes”. Bins in which any error was observed were scored as “no.” To calculate fidelity scores, we divided “yes” marks by the sum of “yes” and “no” and multiplied by 100. Mean fidelity during Jared’s FA was 98.5% (range 92.6–100%) and was assessed across 54.2% (13 of 24) of all FA sessions. Mean fidelity during Jared’s supervised intervention was 99.2% (range 93–100%) and was assessed across 61.6% (98 of 159) of all supervised intervention sessions. Procedural fidelity was not assessed in unsupervised contexts. Procedures Functional Analysis We conducted a multielement functional analysis (FA; Iwata, Dorsey et al., 1994 ) of Jared’s aggression. To promote discrimination between FA conditions (Conners et al., 2000 ), we assigned a different colored t-shirt to each test and control condition of each FA. We presented FA conditions in the order suggested by Hammond et al. ( 2013 ) to control for carry-over effects and capitalize on relevant motivating operations. Programmed EOs (e.g., denied access to tangibles) were presented at session onset and programmed consequences entailed 30-s access to hypothesized reinforcers (e.g., access to tangibles). All variables suspected to influence challenging behavior (e.g., attention, escape) were abolished during tests in which alternative hypotheses were tested (e.g., tangible conditions). During the control (play) condition, all hypothesized controlling variables were purportedly abolished through noncontingent access (i.e., continuous attention, preferred tangible items, no demands). Intervention Supervised Intervention. Intervention appointments for Jared initially occurred twice weekly and lasted approximately 2 hrs—during which time we conducted multiple consecutive 5-min sessions to evaluate Jared’s responsiveness to programmed contingencies. Therapy began with the goal of response elimination via functional communication training (FCT: Carr & Durand, 1985 ; mechanism 3 in Supplemental Fig. 1). It then progressed to the goal of EO-tolerance via a chained-schedules arrangement (i.e., discrimination training; see Supplemental Fig. 2) in which functional reinforcers were available contingent on manding during one schedule element (i.e., an unsignaled reinforcement condition) and were unavailable in the other (i.e., a signaled S ∆ condition; e.g., Hagopian et al., 2011 ; Lambert et al., 2017 ). Transitions from S ∆ to S D elements depended on satisfaction of a conjunctive contingency (i.e., both DRO and DRA schedule requirements; e.g., Lambert, Copeland et al., 2022 ; Muharib et al., 2021 ). For reasons described below, we introduced a function-based punisher (see mechanism 6 in Supplemental Fig. 1) through an individualized levels system (Hagopian et al., 2002 ; Randall et al., 2018 ; Lambert et al., 2021 ). Lower levels of the system were only contacted contingent on challenging behavior and entailed immediate presentation of challenging behavior’s establishing operations (e.g., low quality attention, denied access to tangible items, demand presentation) and a signal (i.e., the color “red” and a visual timer). Levels ascension entailed satisfaction of a conjunctive-schedules arrangement which entailed both a 30-s momentary resetting DRO for problem behavior, and a performance criterion (i.e., simple compliances; see Supplemental Fig. 3). We then began shaping routines which were amenable to Jared’s family’s home circumstance. Jared’s mother first began implementing sessions in his bedroom, and then across the house. As Jared’s mother began to trust that he would not destroy valuable household objects, we replaced arbitrary work requirements (e.g., simple compliances with gross motor demands) with functional ones (chores, academics). Unsupervised Intervention. We asked Jared’s mother to wait to implement the intervention in the unsupervised context until after she had been trained to fidelity (via behavior skills training) in the supervised context. At that time, we helped her plan when, where, and how to implement the intervention in our absence. All supports following the onset of intervention in the unsupervised context focused on helping Jared’s mother practice implementing with fidelity the exact procedures validated in the supervised context. Following the onset of unsupervised intervention implementation, Jared’s team rearranged appointment schedules to visit the home more frequently than twice per week. Mechanisms-Based Problem Solving Informed by Formative Triangulation Jared’s team continuously monitored data obtained from the unsupervised context and engaged with it during an ongoing process of formative triangulation across available data sources, and problem solving which conformed to the principles displayed in Supplemental Figs. 1–5. That is, throughout service provision, the intervention team serving Jared’s family met for an hour between every formal appointment to review quantitative data from supervised and unsupervised contexts, as well as qualitative contextual descriptions from Jared’s mother. Specifically, the team reviewed and coded qualitative descriptions and derived themes intended to contextualize and qualify quantitative data. These themes were presented back to Jared’s mother to confirm the team’s understanding of her insight into intervention effectiveness or noneffectiveness. Once confirmed, the team then identified convergence and divergence across data sources, discussed whether then-current behavioral trends across data sources aligned with anticipated trends, and engaged in problem solving when they did not (Lambert, Paranczak et al., 2022 ). All problem-solving efforts were guided by the baseline assumptions and guiding principles, and technique-selection guides (e.g., Berg et al., 2016; Geiger et al., 2010; Grow et al., 2009), of the FIMB framework and proposed procedural modifications were discussed with Jared’s mother before implemented during formal appointments (Supplemental Fig. 5). The remainder of each meeting was dedicated to logistics; including planning the content and sequencing of sessions, delegating tasks, and materials preparation. During each meeting, a scribe took notes on meeting content and these notes served as the basis for “lesson plans” that coordinated all elements of each appointment. Results & Discussion Jared’s FA was completed in three appointments. Aggression was maintained by contingent access to attention and tangible items, and escape from demands (see Supplemental Fig. 6). Research assistants served as therapists during Day 1 and Jared’s mother served as therapist during Days 2 & 3 (see baseline in Fig. 1 ). The top panel of Fig. 1 shows data collected in supervised contexts and bottom panels show data from unsupervised contexts. In supervised contexts, shaded regions delineate all sessions conducted during each appointment. Appointment numbers are centered above each shaded region. Solid phase-change lines indicate transitions from baseline to intervention. During appointment 4 (i.e. the first intervention appointment), Jared’s aggression spiked, relative to baseline, but decressed to zero levels before therapists left. Similar patterns were observed during the next two appointments; with general levels of aggression decreasing across appointments. When we introduced Jared’s discrimination training protocol during appointment 7, we saw a relapse in aggression but rates again decreased to zero. During appointments 8 and 9, we did not see aggression. Outcomes Disparity Across the same span of time and with the exception of a three-day stretch during which Jared had contracted the flu, we noted a concerning increasing trend in Jared’s aggression in the unsupervised context (bottom panels of Fig. 1 ). As per narrative reports which accompanied frequency and intensity data, much of this aggression was directed toward siblings and was abnormally intense and occasionally sexual. As a result, Jared’s mother asked us to develop an effective punishment procedure which she could consistently use in the unsupervised context. Whereas punishment (colloquially defined) is avoided on principle at the BAC, benign and effective reactive procedures (e.g., those that minimize escalations of behavior and would technically be categorized as punishment, according to scientific defintions of the operation), may be used as a last resort in specific situations to manage problematic behavior while prioritizing the use of proactive and positive strategies (Lavigna et al., 2012). Consequently, we agreed to develop an intervention plan that incorporated function-based punishment (Lambert, Copeland et al., 2022 ). Specifically, during session 10 in the supervised context, we piloted a function-based individualized levels system for Jared in which levels descension entailed the presentation of aggression’s EOs (i.e., denied tangibles, neutral attention, simple gross-motor demands) contingent on aggression. Levels ascension entailed access to the original demands (e.g., chores), followed by breaks and access to high-preferred technology (e.g., an iPad ) . Research assistants first implemented the system while Jared’s mother observed. This continued until challenging behavior occurred and triggered levels descension; thus allowing us to observe Jared’s reaction to the system’s lower level and assess whether contingencies for levels ascension were sufficiently reinforcing to sustain ascension requirements (i.e., 30 s of compliance with simple gross motor demands in the absence of aggression). Once Jared’s aggression contacted the system and we observed evidence of promise (i.e., Jared independently worked through performance requirements to re-ascend to the higher level), we asked Jared’s mother to take over as primary implementer in the supervised plane to observe whether she could successfully and independently enforce all aspects of levels descension and re-ascension when Jared was aggressive (cf. Cheremshynski et al., 2012). After observing Jared’s mother successfully and independently respond to multiple bouts of aggression over the course of the next two appointments (i.e., 11 & 12), we helped her plan when, where, and how to begin implementing the system in the unsupervised context. We warned her that Jared’s aggression might temporarily worsen before it got better, provided words of encouragement (both vocal and written), and then left so that she could pilot Jared’s procedures in our absence; thus beginning intervention in the unsupervised context. That same night, Jared refused to stay in bed and Jared’s mother reported first attempting, and then desisting, in her effort to enforce the system. The next day, Jared stole and hid the laminated token board associated with his system in the outside trashcan. He then woke up his mother at midnight; and again at 2:00 am (the second time by striking her in the face repeatedly; this was atypical). In response to this aggression, his mother reported following through with programmed contingencies for two consecutive hours until eventually observing compliance at 4:00 am. During the 13th appointment in the supervised context, Jared’s mother again independently implemented the system with fidelity in our presence. However, she was noticeably fatigued and reported feeling discouraged. Responding to Qualitative Reports with Treatment Adaptations After reviewing her data with her, we asked Jared’s mother whether she thought the intervention was working and whether she wanted to continue implementing it, or if she wanted something to change. We also asked a series of probing questions intended to clarify conceptual issues essential to our problem-solving process (Supplemental Fig. 5), and intended to explore which aspects of the established system were incompatible with home life, and which elements of the treatment she felt were ineffective when employed across large periods of time. From these interactions, it became clear that Jared’s mother thought the intervention would work and wanted to keep trying it. We also learned that she felt that the two positive reinforcers programmed into the system (i.e., attention, tangible) abolished fairly quickly when used throughout the day, albeit for different reasons. With respect to aggression’s tangible function, the levels system allotted Jared substantially more screen time (i.e., approximately once every 10 min) than he would otherwise have been given. At the same time, the quality of each access period (typically limited to 1 min) was low. From her perspective, the interaction between these two variables abolished the value of access to technology. This, in turn, abolished compliance with low-probability demands (e.g., chores) and thus increased the percentage of her interactions with Jared which entailed demand presentation (i.e., escape extinction). With respect to Jared’s attention function, his mother indicated that she felt guilty implementing the system because it required her to attend to Jared substantially more than to his siblings. To compensate, she would capitalize on times during which Jared was not being aggressive to attend to her other childrens’ needs, and to other household demands. The result of this decision was unbalanced implementation of certain aspects of the system. Specifically, Jared’s mother maintained high fidelity to the system’s punishment-based elements (i.e., levels descension) and decreased fidelity to elements intended to reinforce Jared’s appropriate behavior (i.e., high quality social interactions). By her own report, the only time that Jared’s mother was providing Jared with her undivided attention was when he engaged in challenging behavior; attention which entailed variables meant not only to address aggression’s attention and tangible functions (i.e., denied access to tangible items; neutral attention), but also to address its escape function (i.e., the continuous presentation of demands). Relevant to this point, Jared’s mother estimated that it took twice as long for him to ascend from the lower level of the system when his siblings were present and observing implementation. From these reports, we hypothesized that low-quality interactions between Jared and his mother throughout the day had established contingent access to demand presentation as a reinforcer for aggression’s attention function (e.g., Iwata, Pace, et al., 1994 ). In response to empirical reports from the unsupervised context (reviewed at the onset of every formal appointment from a clipboard Jared’s mother used to store her data), and the supplemental narrative reports and dialogue which accompanied them, we changed important parameters of Jared’s system. For instance, compliance with demands continued to earn Jared a break and access to preferred toys. However, access to his iPad and other electronic items was restricted to two distinct times of day (i.e., times when his siblings were also allowed to access their technology). Following each successful work-cycle completion (e.g., one in which Jared completed all chores without aggression), he accumulated “minutes” which could be exchanged for uninterrupted access to technology at designated times (e.g., he could access his iPad for one uninterrupted 16-min period; rather than sixteen, 1-min periods). We also re-iterated the role of high-quality social interactions in abolishing Jared’s attention function, and encouraged Jared’s mother to increase the amount and quality of attention she provided to Jared by adding two elements to his system. First, when Jared was on “green” (the highest level of the system), he was allowed to request 1:1 playtime with his mother, without sibling interference (a stipulation which was not programmed into the original contingency). Next, we asked Jared’s mother to use a vibrating app on her phone to prompt her to provide Jared with behavior-specific praise every 5–7 min throughout the entire day. Finally, whenever aggression occurred and Jared descended to the lower level of his system, his mother instructed his siblings to leave the room to disallow Jared uncontrolled access to their attention. Following these modifications, the rate and intensity of Jared’s aggression in the unsupervised context dropped precipitously and narrative reports suggested Jared had become “calmer”. Although aggression was never fully eliminated, rates and intensities remained consistently low for the remainder of the study and for the first time since study onset, Jared’s mother provided a positive assessment in the open-ended section of her data system (i.e., “Great day! Excited about Cub Scout hike!”). By the end of the consultation, Jared’s mother stopped adding narrative reports altogether (a strategy she typically used to identify topics she wanted to troubleshoot during formal appointments) and her demeanor during appointments 14–18 improved substantially. During appointment 18, Jared’s mother ran the system without occasioning aggression. With evidence of success in both supervised and unsupervised contexts, as well as both empirical and anecdotal evidence that Jared’s mother could independently implement the system with fidelity, research assistants replaced Jared’s mother as the primary therapist and used the final two appointments of the consultation (i.e., 19 and 20) to embed a small social-skills curriculum (i.e.,, turn-taking, rule following, and commenting while playing board games) into the system in order to create a context for Jared to have positive and meaningful interactions with his siblings. This last addition was included in an attempt to maintain the dosage of attention Jared apparently needed while also decreasing the burden of this load on his mother (i.e., by decreasing EO for Jared’s mands for 1:1 play with his mother). During the six days following our final formal appointment, rates and intensities of challenging behavior remained low. Practical constraints, and assurances from Jared’s mother that she no longer felt she needed our support to be successful, precluded us from obtaining additional maintenance data. General Discussion Previous commentaries on intervention research have warned that prioritizing internal over external and social validity can lead to biased appraisals of evidence that favor short-term and limited interventions validated in controlled environments by trained researchers—at the expense of interventions with demonstrated practical effectiveness in real-world settings (e.g., Carr, 1999; Carr et al., 1999 ; Dunlap et al., 2010). Emphasis of macro-level variables (e.g., metacontingencies, cultural influences, family dynamics), and an embrace of multiple perspectives and methodologies (including non-experimental sources of evidence such as anecdotes and case study reports) can lead to improvements in the ecological and social validity of treatment outcomes (i.e., effective, durable, and feasible interventions applied in typical settings by typical intervention agents; Carr, 1999; Cheremshynski et al., 2012; Dunlap et al., 2008; Lavigna et al., 2012; Lucshyn et al., 2007; 2010; Muskowitz et al., 2011). In this case-study-faciliated discussion, we recruited ongoing empirical samples of parent-perceptions of the prevalence and intensity of challenging behavior throughout all phases of a standard consultation provided through a university-based internship experience (i.e., Lambert, Paranczak et al., 2022 ). We considered these data as a proxy measure for the degree to which gains made in the supervised context generalized to the unsupervised context, and a direct measure of the social validity of those gains. Through this lens, we view appointments 5–11 as a cautionary tale because they indicate that generality and social validity of treatment gains established in a supervised context should not be assumed to have occurred even after systematic efforts to promote it have been made (i.e., programming common stimuli, sequential modification; Stokes & Baer, 1977 ). Specifically, after evidence of therepuetic effect emerged, we trained Jared’s mother to implement the intervention to fidelity during supervised appointments only; when she had access to coaching, feedback, and supports intended to ensure some degree of fidelity even when she contacted high rates of challenging behavior. Despite evidence that she could implement with fidelity and that the intervention could be effective, problem behavior in the unsupervised context continued to worsen (for one possible explanation, see Boyle et al., 2018 ). We thus concluded that a compelling demonstration of effect produced in Jared’s supervised context was not socially valid. However, because Jared’s team was aware of the limitations of their primary analytic strategy (i.e., visual analysis of data obtained in the supervised context), they took steps to mitigate those limitations through data triangulation and problem solving (e.g., Lindberg et al., 2003 ). Thus, one notable advantage available to Jared’s case not available to some of the other cases described in in the overarching validation project (e.g., BAC 4 in Lambert, Sandstrom et al., 2022 ) is that Jared’s team continuously consulted, and responded to, data provided by Jared’s mother. These data served important discriminative functions which prompted Jared’s team to ask probing and detailed follow up questions about contextual fit beyond what Jared’s mother knew to report during open-ended and informal opportunities to do so (e.g., “how has implementation gone for you these past few days?”). This process of triangulation did not disregard evidence of controlling variables confirmed through FA, nor did it ignore evidence which suggested the treatment could be effective in appointments 5, 6, 8, 9, & 18. Rather, it leveraged these details, in conjunction with empirical and anecdotal reports provided by Jared’s mother about his performance in the unsupervised context, to engage in a conceptually systematic and mechanisms-based approach to problem solving (i.e., Supplemental Figs. 1–5) to individualize treatment adaptations which better fit the family’s day-to-day circumstances (see participatory research practices; Pritchett et al., 2021 ). Limitations The limitations of this project should be noted. First, although data in the supervised context were protected by frequent assessments of reliability and procedural fidelity, data in the unsupervised context were not. Importantly, these omissions occurred by design and were intended to provide estimates of perceptions of impact which were not influenced by reactivity (a problem we felt was not obviously resolved by asking perents to video record their interactions for later scoring), or other problematic contextual factors unique to formal therapy. To mitigate the limitations of this approach, we trained Jared’s mother on all aspects of their measurement system, operational definitions of aggression, and then put protections in place to detect and discourage retroactive reporting (i.e., regular phone calls, ongoing data interaction). Even still, what we measured is most accurately conceptualized as Jared’s mother’s perception and recall of his challenging behavior, rather than the behavior itself. We argue that this is nonetheless a critical variable to measure, as caregiver perception and recall is ultimately responsible for initial requests for services, and for substantive appraisals of treatment impact during assessments of social validity. Moreover, perception of challenging beahvior is important in and of itself because it impacts environmental interactions, access to certain opportunities, and relationship development. Second, we did not collect data on Jared’s aggression in the unsupervised context prior to the onset of formal therapy appointments in the supervised context. The representativeness of his baseline sample is thus limited. Third, our inability to update Jared’s contact information precluded us from including his family in the social validity study reported by Lambert, Sandstrom, et al. ( 2022 ). Thus, we do not have long term summative assessments of social validity to corroborate conclusions drawn from formative assessments reported in this study. Finally, because we did not control our effects, we cannot assert with confidence that considering parent reports of impact led to a perceived reduction in the frequency and intensity of challenging behavior in the unsupervised context for Jared. With that said, the asymmetry between contexts throughout Jared’s consultation (up until the above-described modifications) demonstrates a potential which justifies challenges to the social validity of any context-bound demonstration for which generality is not assessed (e.g., Sandbank et al., 2021 ). While some might question the rigor of the approach taken for qualitative data collection and analysis in this mixed methods case study, it provides a critical case example that underscores the value of evaluating therapuetic impact in relation to multiple, varied, and concurrently operating data sources, including those which sample the opinions, perspectives, and perceptions of key stakeholders (Lambert et al., 2024 ). In this vein, other researchers have also suggested that interventions are more effective when there is ongoing support for implementation, continuous progress monitoring, and clear contextual fit with family daily life and values (Dempsey & Keen, 2008 ; Fettig & Barton, 2014 ; Lucyshyn et al., 2007 ; Lucyshyn et al., 2015 ). Intervention models that systematically program for generalization and promote collaboration with families thus seem particularly well poised to produce meaningful outcomes (Nock & Kazdin, 2005 ). However, although previous discussion papers have proposed models to improve collaboroative processes that expand the social validity and generality of treatment effects (e.g., Moore & Amado, 2021 ), few are accompanied with empirical demonstrations of impact. The model through which services were provided to Jared (i.e., Lambert, Copeland et al., 2022 ) is firmly rooted in a foundation of collaboration and responsiveness to unmeasured contextual factors (see Supplemental Table 1). Although the data presented by Lambert, Sandstrom et al. ( 2022 ) suggest informal and unmeasured approaches to iterative collaborative programming with parents may sometimes be insufficient to establish socially valid outcomes, we presented this case to highlight the potential increase in impact that interventionists may have when they engage with clinically useful empirical markers of generality in formative ways. As social validity is often linked to generalization and maintenance of treatment effects (Kennedy, 2002 ; Moore & Amado, 2021 ; Moore & Symons, 2011 ), empirical proxies of these constructs (e.g., data obtained from unsupervised contexts) may prove invaluable to socially valid programming. That is, as the field develops valid measurement systems useful to formative triangulation, it will likely become increasingly possible to develop a replicable technology of socially valid programming. We thus propose that the data we offer, and our methods for obtaining and analyzing it, represent a substantial step forward in transforming effective clinical practice into a technology of behavior change. Declarations Conflict of Interest All authors declare no conflict of interest. Compliance with Ethical Standards This research has been approved by the appropriate institutional research ethics committee and has been performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments. This manuscript is not under review, nor has it been published, elsewhere. This submission has been approved by the responsible authorities where the work was carried out. The participant’s guardians provided informed consent for participation before we initiated study-related activities. Author Contribution JML conceptualized the project, lead implementation and data collection, and took the lead on writing. AF and JRL supported conceptualization, researched topics critical to the study's framing and discussion, and supported writing. Acknowledgement We thank Abigail Morgan, Ipshita Banerjee, Nealetta Houchins-Juarez, & Bailey Copeland for their critical thinking and insight about how to solve problems thematically related to those presented in this study. Data Availability This study adhered to ethical guidelines, and was approved by Vanderbilt’s IRB. Informed consent was obtained and data have been anonymized to protect participant privacy and confidentiality. All authors declare no conflicts of interest. Data are available from the first author upon reasonable request. References Beavers, G. A., Iwata, B. A., & Lerman, D. C. (2013). Thirty years of research on the functional analysis of problem behavior. Journal of Applied Behavior Analysis , 46 (1), 1-21. https://doi.org/10.1002/jaba.30 Behavior Analyst Certification Board. (2020). Ethics code for behavior analysts . https://bacb.com/wp-content/ethics-code-for-behavior-analysts/ Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice , 18 (1), 5-25 Boston, C. (2002). The concept of formative assessment. Practical Assessment, Research, and Evaluation , 8 (1), 9. https://doi.org/10.7275/kmcq-dj31 Boyle, M. A., Hoffmann, A. N., & Lambert, J. M. (2018). Behavioral contrast: Research and areas for investigation. Journal of Applied Behavior Analysis , 51 (3), 702-718. https://doi.org/10.1002/jaba.469 Carr, J. E., Austin, J. L., Britton, L. N., Kellum, K. K., & Bailey, J. S. (1999). An assessment of social validity trends in applied behavior analysis. Behavioral Interventions: Theory & Practice in Residential & Community‐Based Clinical Programs , 14 (4), 223-231. Carr, E. G., & Durand, V. M. (1985). Reducing behavior problems through functional communication training. Journal of Applied Behavior Analysis , 18 (2), 111-126. https://doi.org/10.1901/jaba.1985.18-111 Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research . Sage Publication. Conners, J., Iwata, B. A., Kahng, S. W., Hanley, G. P., Worsdell, A. S., & Thompson, R. H. (2000). Differential responding in the presence and absence of discriminative stimuli during multielement functional analyses. Journal of Applied Behavior Analysis, 33, 299–308. http://dx.doi.org/10 .1901/jaba.2000.33-299 Conroy, M. A., Dunlap, G., Clarke, S., & Alter, P. J. (2005). A descriptive analysis of positive behavioral intervention research with young children with challenging behavior. Topics in Early Childhood Special Education, 25, 157–166. https://doi.org/10.1177/02711214050250030301 Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3 rd ed.). Upper Saddle River, NJ: Pearson. D’Agostino, S. R., Douglas, S. N., & Duenas, A. D. (2019). Practitioner-implemented naturalistic developmental behavioral interventions: Systematic review of social validity practices. Topics in Early Childhood Special Education , 39 (3), 170-182. https://doi.org/10.1177/0271121419854803 Dempsey, I., & Keen, D. (2008). A review of processes and outcomes in family-centered services for children with a disability. Topics in Early Childhood Special Education , 28 (1), 42-52. https://doi.org/10.1177/0271121408316699 Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research , 6 (2), 80-88. https://doi.org/10.1177/1558689812437186 Fahmie, T. A., Rodriguez, N. M., Luczynski, K. C., Rahaman, J. A., Charles, B. M., & Zangrillo, A. N. (2023). Toward an explicit technology of ecological validity. Journal of Applied Behavior Analysis , 56 (2), 302-322. https://doi.org/10.1002/jaba.972 Ferguson, J. L., Cihon, J. H., Leaf, J. B., Van Meter, S. M., McEachin, J., & Leaf, R. (2019). Assessment of social validity trends in the journal of applied behavior analysis. European Journal of Behavior Analysis , 20 (1), 146-157. https://doi.org/10.1080/15021149.2018.1534771 Fettig, A., & Barton, E. E. (2014). Parent implementation of function-based intervention to reduce children’s problem behaviors: A literature review. Topics in Early Childhood Special Education, 34 , 49-61. https://doi.org/10.1177/0271121413513037 Ghaemmaghami, M., Hanley, G. P., & Jessel, J. (2021). Functional communication training: From efficacy to effectiveness. Journal of Applied Behavior Analysis , 54 (1), 122-143. https://doi.org/10.1177/13623613211065540 Hagopian, L. P., Boelter, E. W., & Jarmolowicz, D. P. (2011). Reinforcement schedule thinning following functional communication training: Review and recommendations. Behavior Analysis in Practice , 4 (1), 4-16. https://doi.org/10.1007/BF03391770 Hagopian, L. P., Rush, K. S., Richman, D. M., Kurtz, P. F., Contrucci, S.A., & Crosland, K. (2002). The development and application of individualized levels systems for the intervention of severe problem behavior. Behavior Therapy, 33 , 65-86. https://doi.org/10.1016/S0005-7894(02)80006-5 Halle, J. (2019). Avoiding the humdrum: Recommendations for improving how we conceptualize and assess social validity in ECSE. Topics in Early Childhood Special Education , 39 (3), 139-143. https://doi.org/10.1177/0271121419873525 Hanley, G. P., Jin, C. S., Vanselow, N. R., & Hanratty, L. A. (2014). Producing meaningful improvements in problem behavior of children with autism via synthesized analyses and interventions. Journal of Applied Behavior Analysis, 47 , 16-36. doi: 10.1002/jaba.106 Hammond, J.L., Iwata, B.A., Rooker, G.W., Fritz, J.N. & Bloom, S.E. (2013). Effects of fixed versus random condition sequencing during multielement functional analyses. Journal of Applied Behavior Analysis, 46, 22-30. doi: 10.1002/jaba.7 Herzinger, C. V., & Campbell, J. M. (2007). Comparing functional assessment methodologies: A quantitative synthesis. Journal of Autism and Developmental Disorders , 37 (8), 1430-1445. https://doi.org/10.1007/s10803-006-0219-6 Heale, R., & Forbes, D. (2013). Understanding triangulation in research. Evidence-Based Nursing , 16 (4), 98-98. http://dx.doi.org/10.1136/eb-2013-101494 Hurl, K., Wightman, J., Haynes, S. N., & Virues-Ortega, J. (2016). Does a pre-intervention functional assessment increase intervention effectiveness? A meta-analysis of within-subject interrupted time-series studies. Clinical Psychology Review , 47 , 71-84. https://doi.org/10.1016/j.cpr.2016.05.003 Hurley, J. J. (2012). Social validity assessment in social competence interventions for preschool children: A review. Topics in Early Childhood Special Education, 32 (3) , 164–174. https://doi.org/10.1177/0271121412440186 Iwata, B.A., Dorsey M.F., Slifer K.J., Bauman K.E., Richman G.S. (1994). Toward a functional analysis of self-injury . Journal of Applied Behavior Analysis, 27, 197–209. (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3–20, 1982). https://doi.org/10.1901/jaba.1994.27-197 Iwata, B. A., Pace, G. M., Cowdery, G. E., & Miltenberger, R. G. (1994). What makes extinction work: An analysis of procedural form and function. Journal of Applied Behavior Analysis , 27 (1), 131-144. https://doi.org/10.1901/jaba.1994.27-131 Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification , 1 (4), 427-452. https://doi.org/10.1177/014544557714001 Kazdin, A. E. (1979). Unobtrusive measures in behavioral assessment. Journal of Applied Behavior Analysis, 12 (4), 713-724. https://doi.org/10.1901/jaba.1979.12-713 Kennedy, C. H. (2002). The maintenance of behavior change as an indicator of social validity. Behavior Modification , 26 (5), 594-60. https://doi.org/10.1177/014544502236652 Kimball, R. T., Greer, B. D., Fuhrman, A. M., & Lambert, J. M. (2023). Relapse and its mitigation: Toward behavioral inoculation. Journal of Applied Behavior Analysis , 56(2), 282-301. https://doi.org/10.1002/jaba.971 Kimchi, J., Polivka, B., & Stevenson, J. S. (1991). Triangulation: operational definitions. Nursing Research , 40 (6), 364-366 Lambert, J.M., Clohisy, A.M., Barrows, S.B., & Houchins-Juarez, N.J. (2017). Compound-schedules approaches to noncompliance: Teaching children when to ask and when to work. Journal of Behavioral Education, 26, 201-220. doi: 10.1007/s10864-016-9260-5 Lambert, J.M., Copeland, B.A., Paranczak, J.L., Macdonald, M.J., Torelli, J.T., Houchins-Juarez N.J. (2022). Description and valuation of a function informed and mechanisms-based framework for treating challenging behavior. Journal of Applied Behavior Analysis, 55 (4), 1193-1219. https://doi.org/10.1002/jaba.940 Lambert, J.M., Lamers, M.E.., Copeland, B.A., & Banerjee, I. (2021). Multiple-operant discrimination training to increase the precision of stimulus deltas following treatment of challenging behavior . Behavioral Interventions. https://doi.org/10.1002/bin.1788 Lambert, J. M., Meadan, H., & Ledford, J. R. (2024). Measuring generality and social validity in single case research. In J. R. Ledford & D. L. Gast (Eds.), Single case research methodology: Applications in special education and behavioral sciences (4th ed., pp. 96-118). Routledge. Lambert, J. M., Paranczak, J. L., Copeland, B. A., Houchins-Juarez N. J., & Macdonald, M. J. (2022). Exploring the validity of university-based practicum tailored to develop expertise in addressing challenging behavior. Journal of Applied Behavior Analysis, 55 (4), 1172-1192. https://doi.org/10.1002/jaba.941 Lambert, J.M., Sandstrom, A., Hodapp, R., Copeland, B.A., Paranczak, J.L., Macdonald, M.J., & Houchins-Juarez (2022). Revisiting the validity of university-based practicum tailored to develop expertise in addressing challenging behavior. Journal of Applied Behavior Analysis, 55 (4), 1220-1238. https://doi.org/10.1002/jaba.939 LaVigna, G. W., & Willis, T. J. (2005). Episodic severity: An overlooked dependent variable in the application of behavior analysis to challenging behavior. Journal of Positive Behavior Intervention, 7 , 47 – 54. doi:10.1177/10983007050070010501 Ledford, J. R., Hall, E., Conder, E., & Lane, J. D. (2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education , 35 (4), 223-233. doi: 10.1177/0271121415585956 Ledford, J. R., Lambert, J. M., Pustejovsky, J. E., Zimmerman, K. N., Hollins, N., & Barton, E. E. (2023). Single-case-design research in special education: Next-generation guidelines and considerations. Exceptional Children , 89 (4), 379-396 Lindberg, J. S., Iwata, B. A., Roscoe, E. M., Worsdell, A.S., Hanley, G. P. (2003). Treatment efficacy of noncontingent reinforcement during brief and extended application. Journal of Applied Behavior Analysis, 36 , 1-19. doi: 10.1901/jaba.2003.36-1 Lucyshyn, J. M., Albin, R. W., Horner, R. H., Mann, J. C., Mann, J. A., & Wadsworth, G. (2007). Family implementation of positive behavior support for a child with autism: Longitudinal, single-case, experimental, and descriptive replication and extension. Journal of Positive Behavior Interventions, 9 , 131-150. Lucyshyn, J. M., Fossett, B., Bakeman, R., Cheremshynski, C., Miller, L., Lohrmann, S., Binnendyk, L., Khan, S., Chinn, S., Kwon, S., Irvin, L. K. (2015). Transforming parent–child interaction in family routines: Longitudinal analysis with families of children with developmental disabilities. Journal of Child and Family Studies , 24 , 3526-3541. doi: 10.1007/s10826-015-0154-2 Moore, T. R., & Amado, R. S. (2021). A conceptual model of treatment adherence in a behavior analytic framework. Education and Treatment of Children , 44 (1), 1-17. Moore, T. R., & Symons, F. J. (2011). Adherence to treatment in a behavioral intervention curriculum for parents of children with autism spectrum disorder. Behavior Modification , 35 (6), 570-594. Muharib, R., Dowdy, A., Rajaraman, A., & Jessel, J. (2021). Contingency-based delay to reinforcement following functional communication training for autistic individuals: A multilevel meta-analysis. Autism , https://doi.org/10.1177/13623613211065540 Nock, M. K., & Kazdin, A. E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology , 73 (5), 872. Park, E. Y., & Blair, K. S. C. (2019). Social validity assessment in behavior interventions for young children: A systematic review. Topics in Early Childhood Special Education , 39 (3), 156-169. https://doi.org/10.1177/0271121419860195 Patton, M.Q. (1999). Enhancing the quality and credibility of qualitative analysis. Health Sciences Research, 34, 1189–1208. Penney, A. M., Bateman, K. J., Veverka, Y., Luna, A., & Schwartz, I. S. (2023). Compassion: The eighth dimension of applied behavior analysis. Behavior Analysis in Practice , 1-15. Pritchett, M., Ala’i-Rosales, S., Cruz, A. R., & Cihon, T. M. (2021). Social justice is the spirit and aim of an applied science of human behavior: Moving from colonial to participatory research practices. Behavior Analysis in Practice , 1-19. https://doi.org/10.1007/s40617-021-00591-7 Randall, K. R., Lambert, J. M., Matthews, M. P., & Houchins-Juarez, N. J. (2018). Individualized levels system and systematic stimulus pairing to reduce multiply controlled aggression of a child with autism spectrum disorder. Behavior Modification , 42 (3), 422-440. https://doi.org/10.1177/0145445517741473 Robertson, R. E., Lambert, J. M., Buonomo, K., & Copeland, B. A. (2022). Theoretical foundations of applied behavior analysis and applications in special education research and practice. In Handbook of Special Education Research (pp. 10-23). Routledge. Sandbank, M., Chow, J., Bottema‐Beutel, K., & Woynaroski, T. (2021). Evaluating evidence‐based practice in light of the boundedness and proximity of outcomes: Capturing the scope of change. Autism Research: Official Journal of the International Society for Autism Research , 14 (8), 1536–1542. https://doi.org/10.1002/aur.2527 Saini, V., Jessel, J., Iannaccone, J., & Agnew, C. (2019). Efficacy of functional analysis for informing behavioral treatment of inappropriate mealtime behavior: A systematic review and meta‐analysis. Behavioral Interventions , 34 (2), 231–247. https://doi.org/10.1002/bin.1664 Schwartz, I. S., & Baer, D. M. (1991). Social validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24 (2), 189–204. https://doi.org/10.1901/jaba.1991.24-189 Shepley, C., Shepley, S., Spriggs, A., & O'Neill, K. (2025). Generalization in Single-Case Research: A Review of Inclusion, Prevalence, and Rigor. Single Case in the Social Sciences , 2 (2), 38-53. https://doi.org/10.15763/issn.1936-9298.2025.2.2.38-53 Snodgrass, M. R., Chung, M. Y., Meadan, H., & Halle, J. W. (2018). Social validity in single-case research: A systematic literature review of prevalence and application. Research in Developmental Disabilities, 74 , 160–173. https://doi.org/10.1016/j.ridd.2018.01.007 Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis , 10 , 349-367. doi:10.1901/jaba.1977.10-349 Triangulation. 2025. In Merriam-Webster.com . Retrieved May 8, 2025, from https://www.merriam-webster.com/dictionary/triangulation Volkert, V. M., Lerman, D. C., Call, N. A., & Trosclair‐Lasserre, N. (2009). An evaluation of resurgence during treatment with functional communication training. Journal of Applied Behavior Analysis , 42 (1), 145-160. doi:10.1901/jaba.2009.42-145 Yoder, P., Lloyd, B., & Symons, F. (2018). Observational measurement of behavior (2 nd Edition). Paul H. Brookes Publishing Co Additional Declarations No competing interests reported. Supplementary Files SupplementalMaterials6.26.25.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 14 Dec, 2025 Reviews received at journal 01 Dec, 2025 Reviewers agreed at journal 27 Oct, 2025 Reviewers agreed at journal 19 Oct, 2025 Reviews received at journal 13 Oct, 2025 Reviewers agreed at journal 02 Oct, 2025 Reviews received at journal 03 Sep, 2025 Reviewers agreed at journal 11 Jul, 2025 Reviewers agreed at journal 09 Jul, 2025 Reviewers agreed at journal 09 Jul, 2025 Reviewers invited by journal 09 Jul, 2025 Editor assigned by journal 07 Jul, 2025 Submission checks completed at journal 30 Jun, 2025 First submitted to journal 29 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7005035","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":484256940,"identity":"f295af6a-ae73-4b3d-8093-6b73cff12f2f","order_by":0,"name":"Joseph Lambert","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABN0lEQVRIiWNgGAWjYBACxgYYwc7AwAwS4QcRPAwMCWzsIBYbHi3MEC0Skg0wLczMWLUg9MG0GByAamHAoYW5vf3h48IdDNFABU83F1TcqzO+kfzswZuKe3l8zPwHGD6UHcawoOeMsfHMMwy5DcxsZrdnnCmWMLuRZm4450xxMchhjDPOYWqZkcMmzdsG0sJgdpu3LUHC7MwBM6BIQmIbUAszbxsWLenPf0O0sH+7zfsvQcK45/g3aSADouUvNi0JZswQLTxAWxoSJAzYe4C2NEC1MGLRAvSL9Mw2idw2Zp6y2zOOJUjOON5TJjnnWALILwYHe86lo2sxBIbY58I2m9x+9vZttwtqEvj5m9m3SbypSciTb298+OBHmTWGlgZwdEjgiLIDWMTkGaDpZBSMglEwCkYBTgAA8qZnqXuOp0EAAAAASUVORK5CYII=","orcid":"","institution":"Vanderbilt University","correspondingAuthor":true,"prefix":"","firstName":"Joseph","middleName":"","lastName":"Lambert","suffix":""},{"id":484256941,"identity":"eed69e72-0627-4e2b-baa5-535db4f2c648","order_by":1,"name":"Angel Fettig","email":"","orcid":"","institution":"University of Washington","correspondingAuthor":false,"prefix":"","firstName":"Angel","middleName":"","lastName":"Fettig","suffix":""},{"id":484256942,"identity":"7a6ccc95-e67f-4496-87a7-cfc4cf84840b","order_by":2,"name":"Jennifer Ledford","email":"","orcid":"","institution":"Vanderbilt University","correspondingAuthor":false,"prefix":"","firstName":"Jennifer","middleName":"","lastName":"Ledford","suffix":""}],"badges":[],"createdAt":"2025-06-29 23:38:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7005035/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7005035/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":86672901,"identity":"3c0bd1f5-82ae-432f-95dd-919a878853cd","added_by":"auto","created_at":"2025-07-14 11:44:09","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":162847,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eJoint Display of Empirical Outputs tht Informed Formative Triangulation During Intervention\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7005035/v1/a053dca5f4dd44497b54fe79.png"},{"id":86674168,"identity":"91ac2ba1-379c-4c64-a61c-9e4078bf0853","added_by":"auto","created_at":"2025-07-14 11:52:09","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1029047,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7005035/v1/ac75bdd8-748c-4ed4-9240-b8c6448b3a20.pdf"},{"id":86672902,"identity":"c52a77aa-8c92-40bf-a04d-ebb9afd4cda3","added_by":"auto","created_at":"2025-07-14 11:44:09","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1559064,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementalMaterials6.26.25.docx","url":"https://assets-eu.researchsquare.com/files/rs-7005035/v1/4b8a27dbe744fafb0a64b3c6.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Improving the Process \u0026 Product of Intensive Intervention Through Formative Triangulation ","fulltext":[{"header":"Introduction","content":"\u003cp\u003eFormative assessment has been described as a process (as opposed to a product) by which observers use repeated measures (e.g., direct observation) to sample learner performance throughout the entirety of a treatment program; as opposed to summative assessment, which is designed to evaluate outcomes at the end of a program. The purpose of formative assessment is to offer tentative and ongoing empirical evidence of the effectiveness of a prescribed treatment plan, and to justify adaptation when learners prove non-responsive to the program-as-intended (Boston, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2002\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eRelevant to this point, behavior analysts are ethically bound to engage in data-based iterative instructional design. Undesirable or \u0026ldquo;counter-therapuetic\u0026rdquo; behavior patterns should lead to adaptations to treatment programs that are justified by operant learning theory (Behavior Analyst Certification Board, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Lambert, Paranczak et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). That is, procedural modifications are triggered by poor empirical outcomes or undesirable trajectories, are informed by a theory of change, and are capable of producing new empirical outputs which allow for direct evaluation of the success of the adaptation (Robertson et al., \u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Thus, almost by definition (e.g., Bennett, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2011\u003c/span\u003e), Board Certified Behavior Analysts (BCBA) are specialists in formative assessment processes.\u003c/p\u003e\n\u003ch3\u003eLimitations of Empirical Systems Based in Direct Observation\u003c/h3\u003e\n\u003cp\u003e\u003cem\u003eBoundedness\u003c/em\u003e is a term which has been used to describe the extent to which patterns of behavior change documented by researchers extend beyond the research context and into socially valid domains (Yoder et al., \u003cspan citationid=\"CR68\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). Although the term is useful for describing socially invalid behavior change (i.e., \u003cem\u003econtext-bound\u003c/em\u003e demonstrations of effect not relevant to day-to-day life), it is also useful for labeling data systems which are incapable of evaluating such generality\u0026mdash;those that fail to assess the construct altogether, as well as those whose methodology implicitly precludes valid assessment (see also the concept of ecological validity; Fahmie et al., \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eContemporary approaches to measurement in single case design render it difficult to produce compelling demonstrations of generality. That is, strategies for ensuring measurement integrity and validity (e.g., interobserver agreement, frequent reliability checks) are often carried out by agents (e.g., research assistants) not relevant to typical contexts (i.e. practitioners such as BCBAs). Because the presence of external observers can serve discriminative functions for both treatment implementation and for challenging behavior, demonstrations of generality documented in the presence of such observers tend to lack validity (Ledford et al., \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). As a result, these, and other, problems associated with measurement have plagued single-case research for decades (cf. Kazdin \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e1979\u003c/span\u003e; Kennedy, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2002\u003c/span\u003e; Halle, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e2019\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eUnfortunately, the generality and social validity of context-bound demonstrations of effect which are typical to peer-reviewed publication (i.e., with tightly controlled parameters and high internal validity), often go unassessed and/or unreported in the literature (Carr et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Conroy et al., \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2005\u003c/span\u003e; D\u0026rsquo;Agostino et al., \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Ferguson et al., 2018; Ghaemmaghami et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Hurley, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2012\u003c/span\u003e; Ledford et al.; \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e2016\u003c/span\u003e; Park \u0026amp; Blair, \u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Shepley et al., \u003cspan citationid=\"CR63\" class=\"CitationRef\"\u003e2025\u003c/span\u003e; Snodgrass et al., \u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). As a result, their value to effective therapuetic processes is relatively unknown (Ghaemmaghami et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). That is, measurement limitations and design oversights ubiquitous in intervention research have made it difficult to distinguish practices which promote socially valid and generalized outcomes from those that merely produce context-bound demonstrations (Sandbank et al., \u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eRelevance to Intensive Intervention for Severe Challenging Behavior\u003c/h2\u003e\u003cp\u003eImportantly, the problem posed by boundedness extends beyond research, and into effectiveness appraisals of any scientist-practitioner who leverages rigorous empirical methodologies in their own practice (e.g., BCBAs). For example, effective approaches to intensive intervention for challenging behavior often entail therapies which begin in analogue contexts explicitly designed to accentuate the intensities and salience of relevant controlling variables targeted for analysis (e.g., fixed ratio schedules during functional analysis; cf. Beavers et al., \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Herzinger \u0026amp; Campbell, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2007\u003c/span\u003e; Hurl et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). After desirable response patterns have been established via intervention, interventionists then begin a systematic process of stimulus shaping and fading (e.g., programming common stimuli, sequential modification) to ensure that therapeutic effects generalize to the circumstances under which they are needed (i.e., the unsystematic presentation of establishing operations by careproviders and other stakeholders across a variety of typical settings; Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eDespite the potential of these interventions to improve patterns of challenging behavior, the generality and social validity of established gains is not automatically entailed in the process and there are circumstances under which favorable empirical outputs are challenged by caregiver reports which indicate the data are invalid (Lambert, Sandstrom et al., \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Notwithstanding, some context-bound demonstrations of effect remain useful. On the one hand, orderly demonstrations in controlled contexts (i.e., those contrived during formal therapy) hold substantial value to formative assessment and intervention processes. Specifically, terminal stages of intensive intervention are often designed to mirror the lean and unpredictable contingencies common to typical day-to-day contexts (e.g., Muharib et al., \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). However, success in these latter stages is the product of systematic shaping and is predicated on a participant\u0026rsquo;s success with less complex and more predictable contingencies arranged during initial stages of therapy, when opportunities to respond are presented in a massed trial, session based, single-operant paradigm. Interventionists who advance from initial stages to latter stages of therapy without first obtaining compelling evidence of initial-stage successes are likely to encounter relapses in challenging behavior and an overall degradation of treatment effects (Kimball et al., \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e2023\u003c/span\u003e; cf. Hanley et al., \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Volkert et al., \u003cspan citationid=\"CR67\" class=\"CitationRef\"\u003e2009\u003c/span\u003e). Thus, measurement systems capable of detecting responsiveness to intervention during initial development and validation processes hold clear value.\u003c/p\u003e\u003cp\u003eOn the other hand, response patterns established across small periods of time (e.g., 5-min sessions) are not always representative of patterns under similar circumstances across larger periods of time (e.g., 2-hr sessions; Lindberg et al., \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e2003\u003c/span\u003e). Likewise, variables that control responding in analogue settings may not be present in typical ones (Ledford et al., \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). Thus, it can be appropriate to question the social validity of context-bound demonstrations which are ubiquitous in intervention research (e.g., Ghaemmaghami et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Lambert, Sandstrom et al., \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Sandbank et al., \u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eFor interventionists invested in assessing the generality and social validity of their programming in ways which are unconstrained to contexts which happen to be convenient to rigorous measurement systems, triangulating across complementary data sources can help ascertain the degree to which desired outcomes established in contrived scenarios (e.g., formal therapy sessions) translate into socially significant improvements under typical circumstances (i.e., those large enough to be noticed by those who originally sought services). Consequently, triangulation can provide critical information which can be used to qualify and substantiate empirical demonstrations of effect established in more controlled settings (Kazdin, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e1977\u003c/span\u003e; Schwartz \u0026amp; Baer, \u003cspan citationid=\"CR62\" class=\"CitationRef\"\u003e1991\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eTriangulation\u003c/h3\u003e\n\u003cp\u003eIn trigonometry, triangulation is a strategy by which the specific location of a point-of-interest is deduced based on its geometric relation to other points-of-interest with known locations (Triangulation, \u003cspan citationid=\"CR66\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). Qualitative and mixed-methods researchers who adopt a pragmatic philosophical lens and employ a similar principle when they enlist multiple methods and data sources to enhance their understanding of a phenomenon-of-interest (Denzin, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2012\u003c/span\u003e; Patton, \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Creswell \u0026amp; Clark, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). One goal of triangulation is to overcome the limitations intrinsic to any single methodological approach or measurement system (Kimchi et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e1991\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eFor example, to increase confidence in the extent to which a study has confirmed a proposition\u0026mdash;behavior change has occurred and has generalized to all relevant contexts outside of the constraints of therapy, and is thus socially valid\u0026mdash;a mixed methods researcher would use different approaches to gather different kinds of data to evaluate the extent to which socially valid generalized outcomes occurred, and then would triangulate outcomes across methods to identify commonalities across measures, time points, spaces, and perspectives (Creswell \u0026amp; Clark, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Heale \u0026amp; Forbes, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Kimchi et al, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e1991\u003c/span\u003e). When the results of all methodologies converge (i.e., lead to the same conclusions), there can be more confidence in statements about the generality of treatment outcomes. By contrast, when they diverge (i.e., conclusions contradict across planes of analysis), a need for problem solving arises (in order to reconcile paradoxical outcomes) and the product of this process is often a richer understanding of the phenomenon of interest.\u003c/p\u003e\u003cp\u003eAlthough many types of triangulation have been described in the literature (Patton, \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Kimchi et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e1991\u003c/span\u003e), \u003cem\u003emethodological triangulation\u003c/em\u003e may be most relevant to the problem of measuring generalized and socially valid behavior change. Specifically, in this triangulation approach, researchers use more than one method to acquire data about a phenomenon of interest. The intent is to identify gaps or inconsistencies in the conclusions drawn from data derived from either methodology in isoloation, and then shore up said limitations with the strengths offered through the alternative methodology(ies). \u003cem\u003eWithin-methods\u003c/em\u003e triangulation entails combining similar data-collection approaches to measure the same variable of interest (e.g., frequency counts of individual instances of aggression during environmentally controlled 5-min sessions collected by researchers during formal therapy sessions vs. frequency counts of bouts of aggression collected by parents in uncontrolled naturalistic contexts across the entire day). \u003cem\u003eBetween-methods\u003c/em\u003e triangulation entails comparing data across both quantitative (e.g., frequency counts, intensity scales) and qualitative approaches (e.g., field notes; open-ended narrative reports). \u003cem\u003eMultiple triangulation\u003c/em\u003e describes research which employs more than one type of triangulation (e.g., within- and between-methods triangulation; Kimchi et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e1991\u003c/span\u003e).\u003c/p\u003e\n\u003ch3\u003eMultiple Triangulation as a Formative Process During Intensive Intervention\u003c/h3\u003e\n\u003cp\u003eAlthough triangulation is typically discussed within the context of research, its relevance to practice can\u0026rsquo;t be overstated. For example, Lambert, Sandstrom et al. (\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) recently described case studies in which their research team served two clients during the same calendar year. These clients shared similar diagnoses and relatively similar topographical (i.e., aggression) and functional (i.e., escape, escape \u0026amp; tangible) profiles for challenging behavior. In both cases, functional analysis (FA) unambiguously confirmed controlling variables and function-informed treatments achieved a commonly evoked standard of high efficacy (i.e., better than 90% reductions in challenging behavior, cf. Saini et al., \u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). However, surveys designed to explore the social validity of the treatment model (i.e., Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Lambert, Paranczak et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) suggested that demonstrations of effect established during formal therapy generalized and maintained for one client (i.e., BAC 7 in Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), but not the other (i.e., BAC 4 in Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eOne apparent difference between these two cases rested in each team\u0026rsquo;s responsiveness to circumstantial evidence. In the case of BAC 4, interventionists reported being fairly inflexible in their treatment approach and only considered data obtained through the measurement system contrived by the research team during formal appointments (hereafter referred to as the \u0026ldquo;supervised context\u0026rdquo;). In the case of BAC 7, the research team reported being more responsive to parent reports of events which occurred when the team was not present (hereafter referred to as the \u0026ldquo;unsupervised context\u0026rdquo;), and more intentionally engaging in an iterative process of multiple triangulation and treatment design which was responsive both to objective data obtained in the supervised context, as well as subjective reports of what occurred in the unsupervised context (i.e., \u0026ldquo;formative triangulation\u0026rdquo;). Despite the clear utility of using stakeholder input to modify planned interventions, and the supposed value of this type of analysis in applied behavior analysis (i.e., socially important), these data are not typically included in published behavior analytic work, eschewed for more objective direct measures in tightly controlled situations. The omission of these types of ongoing and formative data has been the subject of recent informal (e.g., blogs, social media) and formal critiques of behavior analysis (Penney et al., \u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e2023\u003c/span\u003e; Pritchett et al., 2022).\u003c/p\u003e\n\u003ch3\u003eMixed Methods Case Study Reports\u003c/h3\u003e\n\u003cp\u003eAs noted above, when considered in isolation, empirical demonstrations of effect common to the formative assessment processes of behavior analysts are often limited in their capacity to assess generalized and socially valid behavior change. This in turn, has sometimes limited interventionists\u0026rsquo; abilities to engage in responsive programming. Formative triangulation apparently offers a practical solution to this problem which honors the strengths of traditional single-case design while also addressing its limitations. Stated differently, exigencies of highly responsive clinical practice are often at odds with contemporary exigencies of evidence in single case design. Further, individuals who engage in severe challenging behavior often represent low incidence and heterogonous populations, and treatment design is highly individualized. Thus, valid replication of treatment effects at the level of complexity required by practicing behavior analysts has often escaped the clutches of single case researchers (Carr et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Durand et al., 2008; 2010; Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eWith that said, recent calls for dynamic evaluative criteria in single case design (i.e., Ledford et al., \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2023\u003c/span\u003e) make space for consumers of research to adjust standards of rigor in ways which prioritize protecting against threats to validity most likely to challenge the primary contributions of a paper, while marginalizing unlikely threats that might impede a project\u0026rsquo;s ability to meaningfully explore critical topics (e.g., generality, social validity). In the spirit of this guidance, the purpose of this project was to expand, through mixed methods case study analysis, dialogue about the potential value of formative triangulation to socially valid intevention programming.\u003c/p\u003e"},{"header":"Method","content":"\u003cp\u003eThe BAC (not an acronym; pronounced B-A-C) was a university-sponsored practicum experience designed to provide pre-service practitioners of special education and applied behavior analysis with exposure to a focused process of intensive intervention for severe challenging behavior (i.e, Lambert, Paranczak, et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Services were provided by a team comprised of four interns (i.e., graduate-level students studying special education and behavior analysis) and one Board Certified Behavior Analyst. Standard goals for the experience entailed: (1) conceptual clarity about the functions of challenging behavior (accomplished through functional analysis), (2) response elimination (accomplished through functional communication training paired with extinction), (3) tolerance of brief but mandatory exposures to challenging behavior’s establishing operations (EO) (accomplished through schedule thinning organized into compound-schedule arrangements), and (4) generalization (achieved by incorporating typical implementers into intervention sessions held in typical settings). All deviations were constrained to well-understood operant learning mechanisms (see Supplemental Figs.\u0026nbsp;1–5) and were deliberated by team members between formal appointments (Lambert, Copeland et al. \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Deliberations adhered to a number of guiding principles (Supplemental Tables\u0026nbsp;1 \u0026amp; 2) and a general problem-solving framework (Supplemental Fig.\u0026nbsp;5).\u003c/p\u003e\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\u003ch2\u003eResearch Design\u003c/h2\u003e\u003cp\u003eUsing a pragmatic philosophical lens and guided by the baseline assumptions and guiding principles described in Supplemental Table\u0026nbsp;1, we conducted a fully integrated mixed methods case study design to generate a case report in which our quantitative (freqeuency counts in supervised and unsupervised planes of analysis in the context of single case designs) and qualitative (open-ended parent reports, field notes) data were collected and compared multiple times per week across a three month period (Creswell \u0026amp; Clark, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Within the context of the overarching program evaluation (see Lambert Paranczak et al., 2022; Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Lambert, Sandstrom et al., \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e); this study can be classified as an emergent (as opposed to fixed) design, because the introduction of formative triangulation occurred in response to clinical oversights described in detail in Lambert, Sandstrom et al. (\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). That is, intervention methodology evolved across the relevant time frame in response to clinical need and was not set (fixed) at the onset of the overarching project.\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eParticipant and Setting\u003c/h3\u003e\n\u003cp\u003eJared (“BAC 32” in Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) was a 10-year-old White male, reported by his mother to be diagnosed with KBG syndrome, Attention-Deficit Hyperactivity Disorder (ADHD), Speech Sound Disorder, Anxiety Disorder (unspecified), and a sleep disorder. We chose to highlight Jared’s case in this report because he represented a case for which 1) data were collected in both supervised and unsupervised planes of analysis, 2) conclusions about treatment effectiveness diverged across planes of analysis while services were being provided, and 3) the treatment team proved responsive to this divergence and meaningfully incorporated formative triangulation in a collaborative process which resulted in eventual convergence across data sources.\u003c/p\u003e\u003cp\u003eFor context, Jared had previously received behavior analytic services from BCBAs. Previous intervention plans were based on a purported attention function and included differential reinforcement, planned ignoring, and choice-making opportunities. At the time of the study, Jared was on summer break, did not attend school during assessment or intervention, and spent most of his time with his family at home or in the community. Jared could fluently speak English using full sentences.\u003c/p\u003e\u003cp\u003eAll stages of therapy occurred in Jared’s home. Initially, appointments were held exclusively in his bedroom after considerable environmental re-arrangement (e.g., removing furniture, securing breakables). Later, appointments were conducted across the home with no environmental re-arrangement.\u003c/p\u003e\n\u003ch3\u003eDependent Variables\u003c/h3\u003e\n\u003cp\u003eWe concurrently tracked the frequency of instances (supervised) or bouts (unsupervised) of Jared’s aggression (hitting, kicking, scratching, choking, pinching, spitting, pushing, and property destruction) across supervised and unsupervised contexts throughout all phases of this study. We also tracked the episodic severity (i.e., above average, average, below average) of the most intense instance of Jared’s aggressive behavior in the unsupervised context (cf. Lavigna \u0026amp; Willis, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e2005\u003c/span\u003e; Lavigna et al., 2012), as well as qualitative data on contextualizing variables for bouts of aggression reported in the unsupervised plane of analysis. Data from supervised contexts were collected by trained research asssistants during formal appointments. Data from unsupervised contexts were reported by Jared’s mother. Prior to study onset, Jared’s mother was trained on all operational definitions.\u003c/p\u003e\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003eMeasurement in the Supervised Context\u003c/h2\u003e\u003cp\u003eIn the supervised context, trained observers collected frequency data using continuous recording procedures on iPods with data-collection software. Sessions were initially 5 min and eventually increased to 10 min to accommodate increases in the durations of response requirements during schedule thinning.\u003c/p\u003e\u003cp\u003e\u003cb\u003eInterobserver agreement (IOA)\u003c/b\u003e. A second trained observer independently collected reliability data across assessment and intervention phases in the supervised context. To calculate IOA, we divided each session into 10-s bins and scored agreements between primary and reliability observers about the the frequency of aggressive behavior scored within each bin. We then divided agreeements by the sum of agreements and disagreements and multipled by 100, averaged bin scores across the session, and reported a single percentage-agreement score for each session evaluated (Cooper et al., \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). Mean IOA during Jared’s FA was 99.9% (range 98–100%) and was calculated for 37.5% (9 of 24) of his FA sessions. Mean IOA during Jared’s supervised intervention was 98% (range 71.7–100%) and was calculated during 45.9% (73 of 159) of Jared’s supervised intervention sessions.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003eMeasurement in Unsupervised Context\u003c/h2\u003e\u003cp\u003eWe asked Jared’s mother to report unsupervised data using a low-tech (paper and pencil) data-collection system. In addition to asking Jared’s mother to report the frequency and intensity of problem behavior in unsupervised contexts, we asked her to contextualize her reports of aggression and identify topics she wished to discuss during subsequent appointments in an open-ended “notes” section. That is, we developed a data system that allowed Jared’s mother to answer three questions during every day of the consultation: 1. How many bouts of problem behavior were observed today? 2. What was the intensity of the most intense bout observed today? 3. What else do you want us to know?\u003c/p\u003e\u003cp\u003eWe asked Jared’s mother to begin reporting the day before therapy in the supervised context began. To protect against retroactive reporting, her team consulted her data sheet with her during every appointment, and Jared’s case manager regularly contacted her between appointments to ask about Jared, and to obtain a description of the reports she had written into the paper-and-pencil system. Throughout the duration of this study, Jared’s mother reported on his behavior in the unsupervised context across 100% (57 of 57) of opportunities.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\u003ch2\u003eProcedural Fidelity\u003c/h2\u003e\u003cp\u003eWe used a whole interval recording system to assess fidelity to programmed procedures during sessions conducted in the supervised context. Specifically, we divided each session into 10-s bins and evaluated adherence to programmed setting events and contingencies on a bin-by-bin basis. Bins in which all procedures were implemented as intended were scored as “yes”. Bins in which any error was observed were scored as “no.” To calculate fidelity scores, we divided “yes” marks by the sum of “yes” and “no” and multiplied by 100. Mean fidelity during Jared’s FA was 98.5% (range 92.6–100%) and was assessed across 54.2% (13 of 24) of all FA sessions. Mean fidelity during Jared’s supervised intervention was 99.2% (range 93–100%) and was assessed across 61.6% (98 of 159) of all supervised intervention sessions. Procedural fidelity was not assessed in unsupervised contexts.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\u003ch2\u003eProcedures\u003c/h2\u003e\u003cdiv id=\"Sec15\" class=\"Section3\"\u003e\u003ch2\u003eFunctional Analysis\u003c/h2\u003e\u003cp\u003eWe conducted a multielement functional analysis (FA; Iwata, Dorsey et al., \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e1994\u003c/span\u003e) of Jared’s aggression. To promote discrimination between FA conditions (Conners et al., \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2000\u003c/span\u003e), we assigned a different colored t-shirt to each test and control condition of each FA. We presented FA conditions in the order suggested by Hammond et al. (\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2013\u003c/span\u003e) to control for carry-over effects and capitalize on relevant motivating operations. Programmed EOs (e.g., denied access to tangibles) were presented at session onset and programmed consequences entailed 30-s access to hypothesized reinforcers (e.g., access to tangibles). All variables suspected to influence challenging behavior (e.g., attention, escape) were abolished during tests in which alternative hypotheses were tested (e.g., tangible conditions). During the control (play) condition, all hypothesized controlling variables were purportedly abolished through noncontingent access (i.e., continuous attention, preferred tangible items, no demands).\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\u003ch2\u003eIntervention\u003c/h2\u003e\u003cp\u003e\u003cb\u003eSupervised Intervention.\u003c/b\u003e Intervention appointments for Jared initially occurred twice weekly and lasted approximately 2 hrs—during which time we conducted multiple consecutive 5-min sessions to evaluate Jared’s responsiveness to programmed contingencies. Therapy began with the goal of response elimination via functional communication training (FCT: Carr \u0026amp; Durand, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e1985\u003c/span\u003e; mechanism 3 in Supplemental Fig.\u0026nbsp;1). It then progressed to the goal of EO-tolerance via a chained-schedules arrangement (i.e., discrimination training; see Supplemental Fig.\u0026nbsp;2) in which functional reinforcers were available contingent on manding during one schedule element (i.e., an unsignaled reinforcement condition) and were unavailable in the other (i.e., a signaled S\u003csup\u003e∆\u003c/sup\u003e condition; e.g., Hagopian et al., \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2011\u003c/span\u003e; Lambert et al., \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Transitions from S\u003csup\u003e∆\u003c/sup\u003e to S\u003csup\u003eD\u003c/sup\u003e elements depended on satisfaction of a conjunctive contingency (i.e., both DRO and DRA schedule requirements; e.g., Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Muharib et al., \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). For reasons described below, we introduced a function-based punisher (see mechanism 6 in Supplemental Fig.\u0026nbsp;1) through an individualized levels system (Hagopian et al., \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2002\u003c/span\u003e; Randall et al., \u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Lambert et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). Lower levels of the system were only contacted contingent on challenging behavior and entailed immediate presentation of challenging behavior’s establishing operations (e.g., low quality attention, denied access to tangible items, demand presentation) and a signal (i.e., the color “red” and a visual timer). Levels ascension entailed satisfaction of a conjunctive-schedules arrangement which entailed both a 30-s momentary resetting DRO for problem behavior, and a performance criterion (i.e., simple compliances; see Supplemental Fig.\u0026nbsp;3).\u003c/p\u003e\u003cp\u003eWe then began shaping routines which were amenable to Jared’s family’s home circumstance. Jared’s mother first began implementing sessions in his bedroom, and then across the house. As Jared’s mother began to trust that he would not destroy valuable household objects, we replaced arbitrary work requirements (e.g., simple compliances with gross motor demands) with functional ones (chores, academics).\u003c/p\u003e\u003cp\u003e\u003cb\u003eUnsupervised Intervention.\u003c/b\u003e We asked Jared’s mother to wait to implement the intervention in the unsupervised context until after she had been trained to fidelity (via behavior skills training) in the supervised context. At that time, we helped her plan when, where, and how to implement the intervention in our absence. All supports following the onset of intervention in the unsupervised context focused on helping Jared’s mother practice implementing with fidelity the exact procedures validated in the supervised context. Following the onset of unsupervised intervention implementation, Jared’s team rearranged appointment schedules to visit the home more frequently than twice per week.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\u003ch2\u003eMechanisms-Based Problem Solving Informed by Formative Triangulation\u003c/h2\u003e\u003cp\u003eJared’s team continuously monitored data obtained from the unsupervised context and engaged with it during an ongoing process of formative triangulation across available data sources, and problem solving which conformed to the principles displayed in Supplemental Figs.\u0026nbsp;1–5. That is, throughout service provision, the intervention team serving Jared’s family met for an hour between every formal appointment to review quantitative data from supervised and unsupervised contexts, as well as qualitative contextual descriptions from Jared’s mother. Specifically, the team reviewed and coded qualitative descriptions and derived themes intended to contextualize and qualify quantitative data. These themes were presented back to Jared’s mother to confirm the team’s understanding of her insight into intervention effectiveness or noneffectiveness. Once confirmed, the team then identified convergence and divergence across data sources, discussed whether then-current behavioral trends across data sources aligned with anticipated trends, and engaged in problem solving when they did not (Lambert, Paranczak et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). All problem-solving efforts were guided by the baseline assumptions and guiding principles, and technique-selection guides (e.g., Berg et al., 2016; Geiger et al., 2010; Grow et al., 2009), of the FIMB framework and proposed procedural modifications were discussed with Jared’s mother before implemented during formal appointments (Supplemental Fig.\u0026nbsp;5).\u003c/p\u003e\u003cp\u003eThe remainder of each meeting was dedicated to logistics; including planning the content and sequencing of sessions, delegating tasks, and materials preparation. During each meeting, a scribe took notes on meeting content and these notes served as the basis for “lesson plans” that coordinated all elements of each appointment.\u003c/p\u003e\u003c/div\u003e"},{"header":"Results \u0026 Discussion","content":"\u003cp\u003eJared’s FA was completed in three appointments. Aggression was maintained by contingent access to attention and tangible items, and escape from demands (see Supplemental Fig.\u0026nbsp;6). Research assistants served as therapists during Day 1 and Jared’s mother served as therapist during Days 2 \u0026amp; 3 (see baseline in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThe top panel of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows data collected in supervised contexts and bottom panels show data from unsupervised contexts. In supervised contexts, shaded regions delineate all sessions conducted during each appointment. Appointment numbers are centered above each shaded region. Solid phase-change lines indicate transitions from baseline to intervention. During appointment 4 (i.e. the first intervention appointment), Jared’s aggression spiked, relative to baseline, but decressed to zero levels before therapists left. Similar patterns were observed during the next two appointments; with general levels of aggression decreasing across appointments. When we introduced Jared’s discrimination training protocol during appointment 7, we saw a relapse in aggression but rates again decreased to zero. During appointments 8 and 9, we did not see aggression.\u003c/p\u003e\u003ch2\u003eOutcomes Disparity\u003c/h2\u003e\u003cp\u003eAcross the same span of time and with the exception of a three-day stretch during which Jared had contracted the flu, we noted a concerning increasing trend in Jared’s aggression in the unsupervised context (bottom panels of Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). As per narrative reports which accompanied frequency and intensity data, much of this aggression was directed toward siblings and was abnormally intense and occasionally sexual. As a result, Jared’s mother asked us to develop an effective punishment procedure which she could consistently use in the unsupervised context. Whereas punishment (colloquially defined) is avoided on principle at the BAC, benign and effective reactive procedures (e.g., those that minimize escalations of behavior and would technically be categorized as punishment, according to scientific defintions of the operation), may be used as a last resort in specific situations to manage problematic behavior while prioritizing the use of proactive and positive strategies (Lavigna et al., 2012). Consequently, we agreed to develop an intervention plan that incorporated function-based punishment (Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eSpecifically, during session 10 in the supervised context, we piloted a function-based individualized levels system for Jared in which levels descension entailed the presentation of aggression’s EOs (i.e., denied tangibles, neutral attention, simple gross-motor demands) contingent on aggression. Levels ascension entailed access to the original demands (e.g., chores), followed by breaks and access to high-preferred technology (e.g., an iPad\u003cb\u003e)\u003c/b\u003e. Research assistants first implemented the system while Jared’s mother observed. This continued until challenging behavior occurred and triggered levels descension; thus allowing us to observe Jared’s reaction to the system’s lower level and assess whether contingencies for levels ascension were sufficiently reinforcing to sustain ascension requirements (i.e., 30 s of compliance with simple gross motor demands in the absence of aggression).\u003c/p\u003e\u003cp\u003eOnce Jared’s aggression contacted the system and we observed evidence of promise (i.e., Jared independently worked through performance requirements to re-ascend to the higher level), we asked Jared’s mother to take over as primary implementer in the supervised plane to observe whether she could successfully and independently enforce all aspects of levels descension and re-ascension when Jared was aggressive (cf. Cheremshynski et al., 2012). After observing Jared’s mother successfully and independently respond to multiple bouts of aggression over the course of the next two appointments (i.e., 11 \u0026amp; 12), we helped her plan when, where, and how to begin implementing the system in the unsupervised context. We warned her that Jared’s aggression might temporarily worsen before it got better, provided words of encouragement (both vocal and written), and then left so that she could pilot Jared’s procedures in our absence; thus beginning intervention in the unsupervised context.\u003c/p\u003e\u003cp\u003eThat same night, Jared refused to stay in bed and Jared’s mother reported first attempting, and then desisting, in her effort to enforce the system. The next day, Jared stole and hid the laminated token board associated with his system in the outside trashcan. He then woke up his mother at midnight; and again at 2:00 am (the second time by striking her in the face repeatedly; this was atypical). In response to this aggression, his mother reported following through with programmed contingencies for two consecutive hours until eventually observing compliance at 4:00 am. During the 13th appointment in the supervised context, Jared’s mother again independently implemented the system with fidelity in our presence. However, she was noticeably fatigued and reported feeling discouraged.\u003c/p\u003e\u003ch2\u003eResponding to Qualitative Reports with Treatment Adaptations\u003c/h2\u003e\u003cp\u003eAfter reviewing her data with her, we asked Jared’s mother whether she thought the intervention was working and whether she wanted to continue implementing it, or if she wanted something to change. We also asked a series of probing questions intended to clarify conceptual issues essential to our problem-solving process (Supplemental Fig.\u0026nbsp;5), and intended to explore which aspects of the established system were incompatible with home life, and which elements of the treatment she felt were ineffective when employed across large periods of time.\u003c/p\u003e\u003cp\u003eFrom these interactions, it became clear that Jared’s mother thought the intervention would work and wanted to keep trying it. We also learned that she felt that the two positive reinforcers programmed into the system (i.e., attention, tangible) abolished fairly quickly when used throughout the day, albeit for different reasons. With respect to aggression’s tangible function, the levels system allotted Jared substantially more screen time (i.e., approximately once every 10 min) than he would otherwise have been given. At the same time, the quality of each access period (typically limited to 1 min) was low. From her perspective, the interaction between these two variables abolished the value of access to technology. This, in turn, abolished compliance with low-probability demands (e.g., chores) and thus increased the percentage of her interactions with Jared which entailed demand presentation (i.e., escape extinction).\u003c/p\u003e\u003cp\u003eWith respect to Jared’s attention function, his mother indicated that she felt guilty implementing the system because it required her to attend to Jared substantially more than to his siblings. To compensate, she would capitalize on times during which Jared was not being aggressive to attend to her other childrens’ needs, and to other household demands. The result of this decision was unbalanced implementation of certain aspects of the system. Specifically, Jared’s mother maintained high fidelity to the system’s punishment-based elements (i.e., levels descension) and decreased fidelity to elements intended to reinforce Jared’s appropriate behavior (i.e., high quality social interactions). By her own report, the only time that Jared’s mother was providing Jared with her undivided attention was when he engaged in challenging behavior; attention which entailed variables meant not only to address aggression’s attention and tangible functions (i.e., denied access to tangible items; neutral attention), but also to address its escape function (i.e., the continuous presentation of demands). Relevant to this point, Jared’s mother estimated that it took twice as long for him to ascend from the lower level of the system when his siblings were present and observing implementation. From these reports, we hypothesized that low-quality interactions between Jared and his mother throughout the day had established contingent access to demand presentation as a reinforcer for aggression’s attention function (e.g., Iwata, Pace, et al., \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e1994\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e In response to empirical reports from the unsupervised context (reviewed at the onset of every formal appointment from a clipboard Jared’s mother used to store her data), and the supplemental narrative reports and dialogue which accompanied them, we changed important parameters of Jared’s system. For instance, compliance with demands continued to earn Jared a break and access to preferred toys. However, access to his iPad and other electronic items was restricted to two distinct times of day (i.e., times when his siblings were also allowed to access their technology). Following each successful work-cycle completion (e.g., one in which Jared completed all chores without aggression), he accumulated “minutes” which could be exchanged for uninterrupted access to technology at designated times (e.g., he could access his iPad for one uninterrupted 16-min period; rather than sixteen, 1-min periods).\u003c/p\u003e\u003cp\u003eWe also re-iterated the role of high-quality social interactions in abolishing Jared’s attention function, and encouraged Jared’s mother to increase the amount and quality of attention she provided to Jared by adding two elements to his system. First, when Jared was on “green” (the highest level of the system), he was allowed to request 1:1 playtime with his mother, without sibling interference (a stipulation which was not programmed into the original contingency). Next, we asked Jared’s mother to use a vibrating app on her phone to prompt her to provide Jared with behavior-specific praise every 5–7 min throughout the entire day. Finally, whenever aggression occurred and Jared descended to the lower level of his system, his mother instructed his siblings to leave the room to disallow Jared uncontrolled access to their attention.\u003c/p\u003e\u003cp\u003eFollowing these modifications, the rate and intensity of Jared’s aggression in the unsupervised context dropped precipitously and narrative reports suggested Jared had become “calmer”. Although aggression was never fully eliminated, rates and intensities remained consistently low for the remainder of the study and for the first time since study onset, Jared’s mother provided a positive assessment in the open-ended section of her data system (i.e., “Great day! Excited about Cub Scout hike!”). By the end of the consultation, Jared’s mother stopped adding narrative reports altogether (a strategy she typically used to identify topics she wanted to troubleshoot during formal appointments) and her demeanor during appointments 14–18 improved substantially. During appointment 18, Jared’s mother ran the system without occasioning aggression.\u003c/p\u003e\u003cp\u003eWith evidence of success in both supervised and unsupervised contexts, as well as both empirical and anecdotal evidence that Jared’s mother could independently implement the system with fidelity, research assistants replaced Jared’s mother as the primary therapist and used the final two appointments of the consultation (i.e., 19 and 20) to embed a small social-skills curriculum (i.e.,, turn-taking, rule following, and commenting while playing board games) into the system in order to create a context for Jared to have positive and meaningful interactions with his siblings. This last addition was included in an attempt to maintain the dosage of attention Jared apparently needed while also decreasing the burden of this load on his mother (i.e., by decreasing EO for Jared’s mands for 1:1 play with his mother).\u003c/p\u003e\u003cp\u003eDuring the six days following our final formal appointment, rates and intensities of challenging behavior remained low. Practical constraints, and assurances from Jared’s mother that she no longer felt she needed our support to be successful, precluded us from obtaining additional maintenance data.\u003c/p\u003e"},{"header":"General Discussion","content":"\u003cp\u003ePrevious commentaries on intervention research have warned that prioritizing internal over external and social validity can lead to biased appraisals of evidence that favor short-term and limited interventions validated in controlled environments by trained researchers—at the expense of interventions with demonstrated practical effectiveness in real-world settings (e.g., Carr, 1999; Carr et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Dunlap et al., 2010). Emphasis of macro-level variables (e.g., metacontingencies, cultural influences, family dynamics), and an embrace of multiple perspectives and methodologies (including non-experimental sources of evidence such as anecdotes and case study reports) can lead to improvements in the ecological and social validity of treatment outcomes (i.e., effective, durable, and feasible interventions applied in typical settings by typical intervention agents; Carr, 1999; Cheremshynski et al., 2012; Dunlap et al., 2008; Lavigna et al., 2012; Lucshyn et al., 2007; 2010; Muskowitz et al., 2011).\u003c/p\u003e\u003cp\u003eIn this case-study-faciliated discussion, we recruited ongoing empirical samples of parent-perceptions of the prevalence and intensity of challenging behavior throughout all phases of a standard consultation provided through a university-based internship experience (i.e., Lambert, Paranczak et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). We considered these data as a proxy measure for the degree to which gains made in the supervised context generalized to the unsupervised context, and a direct measure of the social validity of those gains. Through this lens, we view appointments 5–11 as a cautionary tale because they indicate that generality and social validity of treatment gains established in a supervised context should not be assumed to have occurred even after systematic efforts to promote it have been made (i.e., programming common stimuli, sequential modification; Stokes \u0026amp; Baer, \u003cspan citationid=\"CR65\" class=\"CitationRef\"\u003e1977\u003c/span\u003e). Specifically, after evidence of therepuetic effect emerged, we trained Jared’s mother to implement the intervention to fidelity during supervised appointments only; when she had access to coaching, feedback, and supports intended to ensure some degree of fidelity even when she contacted high rates of challenging behavior. Despite evidence that she could implement with fidelity and that the intervention could be effective, problem behavior in the unsupervised context continued to worsen (for one possible explanation, see Boyle et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). We thus concluded that a compelling demonstration of effect produced in Jared’s supervised context was not socially valid. However, because Jared’s team was aware of the limitations of their primary analytic strategy (i.e., visual analysis of data obtained in the supervised context), they took steps to mitigate those limitations through data triangulation and problem solving (e.g., Lindberg et al., \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e2003\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThus, one notable advantage available to Jared’s case not available to some of the other cases described in in the overarching validation project (e.g., BAC 4 in Lambert, Sandstrom et al., \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) is that Jared’s team continuously consulted, and responded to, data provided by Jared’s mother. These data served important discriminative functions which prompted Jared’s team to ask probing and detailed follow up questions about contextual fit beyond what Jared’s mother knew to report during open-ended and informal opportunities to do so (e.g., “how has implementation gone for you these past few days?”). This process of triangulation did not disregard evidence of controlling variables confirmed through FA, nor did it ignore evidence which suggested the treatment could be effective in appointments 5, 6, 8, 9, \u0026amp; 18. Rather, it leveraged these details, in conjunction with empirical and anecdotal reports provided by Jared’s mother about his performance in the unsupervised context, to engage in a conceptually systematic and mechanisms-based approach to problem solving (i.e., Supplemental Figs.\u0026nbsp;1–5) to individualize treatment adaptations which better fit the family’s day-to-day circumstances (see participatory research practices; Pritchett et al., \u003cspan citationid=\"CR57\" class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e\u003ch2\u003eLimitations\u003c/h2\u003e\u003cp\u003eThe limitations of this project should be noted. First, although data in the supervised context were protected by frequent assessments of reliability and procedural fidelity, data in the unsupervised context were not. Importantly, these omissions occurred by design and were intended to provide estimates of perceptions of impact which were not influenced by reactivity (a problem we felt was not obviously resolved by asking perents to video record their interactions for later scoring), or other problematic contextual factors unique to formal therapy. To mitigate the limitations of this approach, we trained Jared’s mother on all aspects of their measurement system, operational definitions of aggression, and then put protections in place to detect and discourage retroactive reporting (i.e., regular phone calls, ongoing data interaction). Even still, what we measured is most accurately conceptualized as Jared’s mother’s \u003cem\u003eperception and recall\u003c/em\u003e of his challenging behavior, rather than the behavior itself. We argue that this is nonetheless a critical variable to measure, as caregiver perception and recall is ultimately responsible for initial requests for services, and for substantive appraisals of treatment impact during assessments of social validity. Moreover, perception of challenging beahvior is important in and of itself because it impacts environmental interactions, access to certain opportunities, and relationship development.\u003c/p\u003e\u003cp\u003eSecond, we did not collect data on Jared’s aggression in the unsupervised context prior to the onset of formal therapy appointments in the supervised context. The representativeness of his baseline sample is thus limited. Third, our inability to update Jared’s contact information precluded us from including his family in the social validity study reported by Lambert, Sandstrom, et al. (\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Thus, we do not have long term summative assessments of social validity to corroborate conclusions drawn from formative assessments reported in this study.\u003c/p\u003e\u003cp\u003eFinally, because we did not control our effects, we cannot assert with confidence that considering parent reports of impact led to a perceived reduction in the frequency and intensity of challenging behavior in the unsupervised context for Jared. With that said, the asymmetry between contexts throughout Jared’s consultation (up until the above-described modifications) demonstrates a potential which justifies challenges to the social validity of any context-bound demonstration for which generality is not assessed (e.g., Sandbank et al., \u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). While some might question the rigor of the approach taken for qualitative data collection and analysis in this mixed methods case study, it provides a critical case example that underscores the value of evaluating therapuetic impact in relation to multiple, varied, and concurrently operating data sources, including those which sample the opinions, perspectives, and perceptions of key stakeholders (Lambert et al., \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e2024\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn this vein, other researchers have also suggested that interventions are more effective when there is ongoing support for implementation, continuous progress monitoring, and clear contextual fit with family daily life and values (Dempsey \u0026amp; Keen, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2008\u003c/span\u003e; Fettig \u0026amp; Barton, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Lucyshyn et al., \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e2007\u003c/span\u003e; Lucyshyn et al., \u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e2015\u003c/span\u003e). Intervention models that systematically program for generalization and promote collaboration with families thus seem particularly well poised to produce meaningful outcomes (Nock \u0026amp; Kazdin, \u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e2005\u003c/span\u003e). However, although previous discussion papers have proposed models to improve collaboroative processes that expand the social validity and generality of treatment effects (e.g., Moore \u0026amp; Amado, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e2021\u003c/span\u003e), few are accompanied with empirical demonstrations of impact.\u003c/p\u003e\u003cp\u003eThe model through which services were provided to Jared (i.e., Lambert, Copeland et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) is firmly rooted in a foundation of collaboration and responsiveness to unmeasured contextual factors (see Supplemental Table\u0026nbsp;1). Although the data presented by Lambert, Sandstrom et al. (\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) suggest informal and unmeasured approaches to iterative collaborative programming with parents may sometimes be insufficient to establish socially valid outcomes, we presented this case to highlight the potential increase in impact that interventionists may have when they engage with clinically useful empirical markers of generality in formative ways. As social validity is often linked to generalization and maintenance of treatment effects (Kennedy, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2002\u003c/span\u003e; Moore \u0026amp; Amado, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Moore \u0026amp; Symons, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e2011\u003c/span\u003e), empirical proxies of these constructs (e.g., data obtained from unsupervised contexts) may prove invaluable to socially valid programming. That is, as the field develops valid measurement systems useful to formative triangulation, it will likely become increasingly possible to develop a replicable technology of socially valid programming. We thus propose that the data we offer, and our methods for obtaining and analyzing it, represent a substantial step forward in transforming effective clinical practice into a technology of behavior change.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003ch2\u003eConflict of Interest\u003c/h2\u003e\u003cp\u003eAll authors declare no conflict of interest.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCompliance with Ethical Standards\u003c/strong\u003e\u003c/p\u003e\u003cp\u003e This research has been approved by the appropriate institutional research ethics committee and has been performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments. This manuscript is not under review, nor has it been published, elsewhere. This submission has been approved by the responsible authorities where the work was carried out. The participant\u0026rsquo;s guardians provided informed consent for participation before we initiated study-related activities.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eJML conceptualized the project, lead implementation and data collection, and took the lead on writing. AF and JRL supported conceptualization, researched topics critical to the study's framing and discussion, and supported writing.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eWe thank Abigail Morgan, Ipshita Banerjee, Nealetta Houchins-Juarez, \u0026amp; Bailey Copeland for their critical thinking and insight about how to solve problems thematically related to those presented in this study.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThis study adhered to ethical guidelines, and was approved by Vanderbilt\u0026rsquo;s IRB. Informed consent was obtained and data have been anonymized to protect participant privacy and confidentiality. All authors declare no conflicts of interest. Data are available from the first author upon reasonable request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBeavers, G. A., Iwata, B. A., \u0026amp; Lerman, D. C. (2013). Thirty years of research on the functional analysis of problem behavior. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e46\u003c/em\u003e(1), 1-21. https://doi.org/10.1002/jaba.30\u003c/li\u003e\n\u003cli\u003eBehavior Analyst Certification Board. (2020). \u003cem\u003eEthics code for behavior analysts\u003c/em\u003e. https://bacb.com/wp-content/ethics-code-for-behavior-analysts/\u003c/li\u003e\n\u003cli\u003eBennett, R. E. (2011). Formative assessment: A critical review. \u003cem\u003eAssessment in Education: Principles, Policy \u0026amp; Practice\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e(1), 5-25\u003c/li\u003e\n\u003cli\u003eBoston, C. (2002). The concept of formative assessment. \u003cem\u003ePractical Assessment, Research, and Evaluation\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e(1), 9. https://doi.org/10.7275/kmcq-dj31\u003c/li\u003e\n\u003cli\u003eBoyle, M. A., Hoffmann, A. N., \u0026amp; Lambert, J. M. (2018). Behavioral contrast: Research and areas for investigation. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e51\u003c/em\u003e(3), 702-718. https://doi.org/10.1002/jaba.469\u003c/li\u003e\n\u003cli\u003eCarr, J. E., Austin, J. L., Britton, L. N., Kellum, K. K., \u0026amp; Bailey, J. S. (1999). An assessment of social validity trends in applied behavior analysis. \u003cem\u003eBehavioral Interventions: Theory \u0026amp; Practice in Residential \u0026amp; Community‐Based Clinical Programs\u003c/em\u003e, \u003cem\u003e14\u003c/em\u003e(4), 223-231. \u003c/li\u003e\n\u003cli\u003eCarr, E. G., \u0026amp; Durand, V. M. (1985). Reducing behavior problems through functional communication training. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e(2), 111-126. https://doi.org/10.1901/jaba.1985.18-111\u003c/li\u003e\n\u003cli\u003eCreswell, J. W., \u0026amp; Clark, V. L. P. (2017). \u003cem\u003eDesigning and conducting mixed methods research\u003c/em\u003e. Sage Publication.\u003c/li\u003e\n\u003cli\u003eConners, J., Iwata, B. A., Kahng, S. W., Hanley, G. P., Worsdell, A. S., \u0026amp; Thompson, R. H. (2000). Differential responding in the presence and absence of discriminative stimuli during multielement functional analyses. \u003cem\u003eJournal of Applied Behavior Analysis, 33, \u003c/em\u003e299\u0026ndash;308. http://dx.doi.org/10 .1901/jaba.2000.33-299 \u003c/li\u003e\n\u003cli\u003eConroy, M. A., Dunlap, G., Clarke, S., \u0026amp; Alter, P. J. (2005). A descriptive analysis of positive behavioral intervention research with young children with challenging behavior. \u003cem\u003eTopics in Early Childhood Special Education, 25,\u003c/em\u003e 157\u0026ndash;166. https://doi.org/10.1177/02711214050250030301\u003c/li\u003e\n\u003cli\u003eCooper, J. O., Heron, T. E., \u0026amp; Heward, W. L. (2020). \u003cem\u003eApplied behavior analysis \u003c/em\u003e(3\u003csup\u003erd\u003c/sup\u003e ed.). Upper Saddle River, NJ: Pearson. \u003c/li\u003e\n\u003cli\u003eD\u0026rsquo;Agostino, S. R., Douglas, S. N., \u0026amp; Duenas, A. D. (2019). Practitioner-implemented naturalistic developmental behavioral interventions: Systematic review of social validity practices. \u003cem\u003eTopics in Early Childhood Special Education\u003c/em\u003e, \u003cem\u003e39\u003c/em\u003e(3), 170-182. https://doi.org/10.1177/0271121419854803\u003c/li\u003e\n\u003cli\u003eDempsey, I., \u0026amp; Keen, D. (2008). A review of processes and outcomes in family-centered services for children with a disability. \u003cem\u003eTopics in Early Childhood Special Education\u003c/em\u003e, \u003cem\u003e28\u003c/em\u003e(1), 42-52. https://doi.org/10.1177/0271121408316699\u003c/li\u003e\n\u003cli\u003eDenzin, N. K. (2012). Triangulation 2.0. \u003cem\u003eJournal of Mixed Methods Research\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(2), 80-88. https://doi.org/10.1177/1558689812437186\u003c/li\u003e\n\u003cli\u003eFahmie, T. A., Rodriguez, N. M., Luczynski, K. C., Rahaman, J. A., Charles, B. M., \u0026amp; Zangrillo, A. N. (2023). Toward an explicit technology of ecological validity. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e56\u003c/em\u003e(2), 302-322. https://doi.org/10.1002/jaba.972\u003c/li\u003e\n\u003cli\u003eFerguson, J. L., Cihon, J. H., Leaf, J. B., Van Meter, S. M., McEachin, J., \u0026amp; Leaf, R. (2019). Assessment of social validity trends in the journal of applied behavior analysis. \u003cem\u003eEuropean Journal of Behavior Analysis\u003c/em\u003e, \u003cem\u003e20\u003c/em\u003e(1), 146-157. https://doi.org/10.1080/15021149.2018.1534771\u003c/li\u003e\n\u003cli\u003eFettig, A., \u0026amp; Barton, E. E. (2014). Parent implementation of function-based intervention to reduce children\u0026rsquo;s problem behaviors: A literature review. \u003cem\u003eTopics in Early Childhood Special Education, 34\u003c/em\u003e, 49-61. https://doi.org/10.1177/0271121413513037\u003c/li\u003e\n\u003cli\u003eGhaemmaghami, M., Hanley, G. P., \u0026amp; Jessel, J. (2021). Functional communication training: From efficacy to effectiveness. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e54\u003c/em\u003e(1), 122-143. https://doi.org/10.1177/13623613211065540\u003c/li\u003e\n\u003cli\u003eHagopian, L. P., Boelter, E. W., \u0026amp; Jarmolowicz, D. P. (2011). Reinforcement schedule thinning following functional communication training: Review and recommendations. \u003cem\u003eBehavior Analysis in Practice\u003c/em\u003e, \u003cem\u003e4\u003c/em\u003e(1), 4-16. https://doi.org/10.1007/BF03391770\u003c/li\u003e\n\u003cli\u003eHagopian, L. P., Rush, K. S., Richman, D. M., Kurtz, P. F., Contrucci, S.A., \u0026amp; Crosland, K. (2002). The development and application of individualized levels systems for the intervention of severe problem behavior. \u003cem\u003eBehavior Therapy, 33\u003c/em\u003e, 65-86. https://doi.org/10.1016/S0005-7894(02)80006-5\u003c/li\u003e\n\u003cli\u003eHalle, J. (2019). Avoiding the humdrum: Recommendations for improving how we conceptualize and assess social validity in ECSE. \u003cem\u003eTopics in Early Childhood Special Education\u003c/em\u003e, \u003cem\u003e39\u003c/em\u003e(3), 139-143. https://doi.org/10.1177/0271121419873525\u003c/li\u003e\n\u003cli\u003eHanley, G. P., Jin, C. S., Vanselow, N. R., \u0026amp; Hanratty, L. A. (2014). Producing meaningful improvements in problem behavior of children with autism via synthesized analyses and interventions. \u003cem\u003eJournal of Applied Behavior Analysis, 47\u003c/em\u003e, 16-36. doi: 10.1002/jaba.106 \u003c/li\u003e\n\u003cli\u003eHammond, J.L., Iwata, B.A., Rooker, G.W., Fritz, J.N. \u0026amp; Bloom, S.E. (2013). Effects of fixed versus random condition sequencing during multielement functional analyses. \u003cem\u003eJournal of Applied Behavior Analysis, 46,\u003c/em\u003e 22-30. doi: 10.1002/jaba.7\u003c/li\u003e\n\u003cli\u003eHerzinger, C. V., \u0026amp; Campbell, J. M. (2007). Comparing functional assessment methodologies: A quantitative synthesis. \u003cem\u003eJournal of Autism and Developmental Disorders\u003c/em\u003e, \u003cem\u003e37\u003c/em\u003e(8), 1430-1445. https://doi.org/10.1007/s10803-006-0219-6\u003c/li\u003e\n\u003cli\u003eHeale, R., \u0026amp; Forbes, D. (2013). Understanding triangulation in research. \u003cem\u003eEvidence-Based Nursing\u003c/em\u003e, \u003cem\u003e16\u003c/em\u003e(4), 98-98. http://dx.doi.org/10.1136/eb-2013-101494\u003c/li\u003e\n\u003cli\u003eHurl, K., Wightman, J., Haynes, S. N., \u0026amp; Virues-Ortega, J. (2016). Does a pre-intervention functional assessment increase intervention effectiveness? A meta-analysis of within-subject interrupted time-series studies. \u003cem\u003eClinical Psychology Review\u003c/em\u003e, \u003cem\u003e47\u003c/em\u003e, 71-84. https://doi.org/10.1016/j.cpr.2016.05.003\u003c/li\u003e\n\u003cli\u003eHurley, J. J. (2012). Social validity assessment in social competence interventions for preschool children: A review. \u003cem\u003eTopics in Early Childhood Special Education, 32\u003c/em\u003e(3)\u003cem\u003e, \u003c/em\u003e164\u0026ndash;174. https://doi.org/10.1177/0271121412440186\u003c/li\u003e\n\u003cli\u003eIwata, B.A., Dorsey M.F., Slifer K.J., Bauman K.E., Richman G.S. (1994). Toward a functional analysis of self-injury\u003cem\u003e. Journal of Applied Behavior Analysis,\u003c/em\u003e 27, 197\u0026ndash;209. (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3\u0026ndash;20, 1982). https://doi.org/10.1901/jaba.1994.27-197\u003c/li\u003e\n\u003cli\u003eIwata, B. A., Pace, G. M., Cowdery, G. E., \u0026amp; Miltenberger, R. G. (1994). What makes extinction work: An analysis of procedural form and function. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e27\u003c/em\u003e(1), 131-144. https://doi.org/10.1901/jaba.1994.27-131\u003c/li\u003e\n\u003cli\u003eKazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. \u003cem\u003eBehavior Modification\u003c/em\u003e, \u003cem\u003e1\u003c/em\u003e(4), 427-452. https://doi.org/10.1177/014544557714001\u003c/li\u003e\n\u003cli\u003eKazdin, A. E. (1979). Unobtrusive measures in behavioral assessment. \u003cem\u003eJournal of Applied Behavior Analysis, 12\u003c/em\u003e(4), 713-724. https://doi.org/10.1901/jaba.1979.12-713\u003c/li\u003e\n\u003cli\u003eKennedy, C. H. (2002). The maintenance of behavior change as an indicator of social validity. \u003cem\u003eBehavior Modification\u003c/em\u003e, \u003cem\u003e26\u003c/em\u003e(5), 594-60. https://doi.org/10.1177/014544502236652\u003c/li\u003e\n\u003cli\u003eKimball, R. T., Greer, B. D., Fuhrman, A. M., \u0026amp; Lambert, J. M. (2023). Relapse and its mitigation: Toward behavioral inoculation. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, 56(2), 282-301. https://doi.org/10.1002/jaba.971\u003c/li\u003e\n\u003cli\u003eKimchi, J., Polivka, B., \u0026amp; Stevenson, J. S. (1991). Triangulation: operational definitions. \u003cem\u003eNursing Research\u003c/em\u003e, \u003cem\u003e40\u003c/em\u003e(6), 364-366\u003c/li\u003e\n\u003cli\u003eLambert, J.M., Clohisy, A.M., Barrows, S.B., \u0026amp; Houchins-Juarez, N.J. (2017). Compound-schedules approaches to noncompliance: Teaching children when to ask and when to work. \u003cem\u003eJournal of Behavioral Education, 26,\u003c/em\u003e 201-220. doi: 10.1007/s10864-016-9260-5\u003c/li\u003e\n\u003cli\u003eLambert, J.M., Copeland, B.A., Paranczak, J.L., Macdonald, M.J., Torelli, J.T., Houchins-Juarez N.J. (2022). Description and valuation of a function informed and mechanisms-based framework for treating challenging behavior. \u003cem\u003eJournal of Applied Behavior Analysis, 55\u003c/em\u003e(4), 1193-1219. https://doi.org/10.1002/jaba.940\u003c/li\u003e\n\u003cli\u003eLambert, J.M., Lamers, M.E.., Copeland, B.A., \u0026amp; Banerjee, I. (2021). Multiple-operant discrimination training to increase the precision of stimulus deltas following treatment of challenging behavior\u003cem\u003e. Behavioral Interventions. \u003c/em\u003ehttps://doi.org/10.1002/bin.1788\u003c/li\u003e\n\u003cli\u003eLambert, J. M., Meadan, H., \u0026amp; Ledford, J. R. (2024). Measuring generality and social validity in single case research. In J. R. Ledford \u0026amp; D. L. Gast (Eds.), Single case research methodology: Applications in special education and behavioral sciences (4th ed., pp. 96-118). Routledge.\u003c/li\u003e\n\u003cli\u003eLambert, J. M., Paranczak, J. L., Copeland, B. A., Houchins-Juarez N. J., \u0026amp; Macdonald, M. J. (2022). Exploring the validity of university-based practicum tailored to develop expertise in addressing challenging behavior. \u003cem\u003eJournal of Applied Behavior Analysis, 55\u003c/em\u003e(4), 1172-1192. https://doi.org/10.1002/jaba.941\u003c/li\u003e\n\u003cli\u003eLambert, J.M., Sandstrom, A., Hodapp, R., Copeland, B.A., Paranczak, J.L., Macdonald, M.J., \u0026amp; Houchins-Juarez (2022). Revisiting the validity of university-based practicum tailored to develop expertise in addressing challenging behavior. \u003cem\u003eJournal of Applied Behavior Analysis, 55\u003c/em\u003e(4), 1220-1238. https://doi.org/10.1002/jaba.939\u003c/li\u003e\n\u003cli\u003eLaVigna, G. W., \u0026amp; Willis, T. J. (2005). Episodic severity: An overlooked dependent variable in the application of behavior analysis to challenging behavior. \u003cem\u003eJournal of Positive Behavior Intervention, 7\u003c/em\u003e, 47 \u0026ndash; 54. doi:10.1177/10983007050070010501\u003c/li\u003e\n\u003cli\u003eLedford, J. R., Hall, E., Conder, E., \u0026amp; Lane, J. D. (2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. \u003cem\u003eTopics in Early Childhood Special Education\u003c/em\u003e, \u003cem\u003e35\u003c/em\u003e(4), 223-233. doi: 10.1177/0271121415585956\u003c/li\u003e\n\u003cli\u003eLedford, J. R., Lambert, J. M., Pustejovsky, J. E., Zimmerman, K. N., Hollins, N., \u0026amp; Barton, E. E. (2023). Single-case-design research in special education: Next-generation guidelines and considerations. \u003cem\u003eExceptional Children\u003c/em\u003e, \u003cem\u003e89\u003c/em\u003e(4), 379-396\u003c/li\u003e\n\u003cli\u003eLindberg, J. S., Iwata, B. A., Roscoe, E. M., Worsdell, A.S., Hanley, G. P. (2003). Treatment efficacy of noncontingent reinforcement during brief and extended application. \u003cem\u003eJournal of Applied Behavior Analysis, 36\u003c/em\u003e, 1-19. doi: 10.1901/jaba.2003.36-1\u003c/li\u003e\n\u003cli\u003eLucyshyn, J. M., Albin, R. W., Horner, R. H., Mann, J. C., Mann, J. A., \u0026amp; Wadsworth, G. (2007). Family implementation of positive behavior support for a child with autism: Longitudinal, single-case, experimental, and descriptive replication and extension. \u003cem\u003eJournal of Positive Behavior Interventions, 9\u003c/em\u003e, 131-150.\u003c/li\u003e\n\u003cli\u003eLucyshyn, J. M., Fossett, B., Bakeman, R., Cheremshynski, C., Miller, L., Lohrmann, S., Binnendyk, L., Khan, S., Chinn, S., Kwon, S., Irvin, L. K. (2015). Transforming parent\u0026ndash;child interaction in family routines: Longitudinal analysis with families of children with developmental disabilities. \u003cem\u003eJournal of Child and Family Studies\u003c/em\u003e, \u003cem\u003e24\u003c/em\u003e, 3526-3541. doi: 10.1007/s10826-015-0154-2\u003c/li\u003e\n\u003cli\u003eMoore, T. R., \u0026amp; Amado, R. S. (2021). A conceptual model of treatment adherence in a behavior analytic framework. \u003cem\u003eEducation and Treatment of Children\u003c/em\u003e, \u003cem\u003e44\u003c/em\u003e(1), 1-17.\u003c/li\u003e\n\u003cli\u003eMoore, T. R., \u0026amp; Symons, F. J. (2011). Adherence to treatment in a behavioral intervention curriculum for parents of children with autism spectrum disorder. \u003cem\u003eBehavior Modification\u003c/em\u003e, \u003cem\u003e35\u003c/em\u003e(6), 570-594.\u003c/li\u003e\n\u003cli\u003eMuharib, R., Dowdy, A., Rajaraman, A., \u0026amp; Jessel, J. (2021). Contingency-based delay to reinforcement following functional communication training for autistic individuals: A multilevel meta-analysis. \u003cem\u003eAutism\u003c/em\u003e, https://doi.org/10.1177/13623613211065540\u003c/li\u003e\n\u003cli\u003eNock, M. K., \u0026amp; Kazdin, A. E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. \u003cem\u003eJournal of Consulting and Clinical Psychology\u003c/em\u003e, \u003cem\u003e73\u003c/em\u003e(5), 872.\u003c/li\u003e\n\u003cli\u003ePark, E. Y., \u0026amp; Blair, K. S. C. (2019). Social validity assessment in behavior interventions for young children: A systematic review. \u003cem\u003eTopics in Early Childhood Special Education\u003c/em\u003e, \u003cem\u003e39\u003c/em\u003e(3), 156-169. https://doi.org/10.1177/0271121419860195\u003c/li\u003e\n\u003cli\u003ePatton, M.Q. (1999). Enhancing the quality and credibility of qualitative analysis. \u003cem\u003eHealth Sciences Research, 34, \u003c/em\u003e1189\u0026ndash;1208.\u003c/li\u003e\n\u003cli\u003ePenney, A. M., Bateman, K. J., Veverka, Y., Luna, A., \u0026amp; Schwartz, I. S. (2023). Compassion: The eighth dimension of applied behavior analysis. \u003cem\u003eBehavior Analysis in Practice\u003c/em\u003e, 1-15.\u003c/li\u003e\n\u003cli\u003ePritchett, M., Ala\u0026rsquo;i-Rosales, S., Cruz, A. R., \u0026amp; Cihon, T. M. (2021). Social justice is the spirit and aim of an applied science of human behavior: Moving from colonial to participatory research practices. \u003cem\u003eBehavior Analysis in Practice\u003c/em\u003e, 1-19. https://doi.org/10.1007/s40617-021-00591-7\u003c/li\u003e\n\u003cli\u003eRandall, K. R., Lambert, J. M., Matthews, M. P., \u0026amp; Houchins-Juarez, N. J. (2018). Individualized levels system and systematic stimulus pairing to reduce multiply controlled aggression of a child with autism spectrum disorder. \u003cem\u003eBehavior Modification\u003c/em\u003e, \u003cem\u003e42\u003c/em\u003e(3), 422-440. https://doi.org/10.1177/0145445517741473\u003c/li\u003e\n\u003cli\u003eRobertson, R. E., Lambert, J. M., Buonomo, K., \u0026amp; Copeland, B. A. (2022). Theoretical foundations of applied behavior analysis and applications in special education research and practice. In \u003cem\u003eHandbook of Special Education Research \u003c/em\u003e(pp. 10-23). Routledge.\u003c/li\u003e\n\u003cli\u003eSandbank, M., Chow, J., Bottema‐Beutel, K., \u0026amp; Woynaroski, T. (2021). Evaluating evidence‐based practice in light of the boundedness and proximity of outcomes: Capturing the scope of change. \u003cem\u003eAutism Research: Official Journal of the International Society for Autism Research\u003c/em\u003e, \u003cem\u003e14\u003c/em\u003e(8), 1536\u0026ndash;1542. https://doi.org/10.1002/aur.2527\u003c/li\u003e\n\u003cli\u003eSaini, V., Jessel, J., Iannaccone, J., \u0026amp; Agnew, C. (2019). Efficacy of functional analysis for informing behavioral treatment of inappropriate mealtime behavior: A systematic review and meta‐analysis. \u003cem\u003eBehavioral Interventions\u003c/em\u003e, \u003cem\u003e34\u003c/em\u003e(2), 231\u0026ndash;247. https://doi.org/10.1002/bin.1664\u003c/li\u003e\n\u003cli\u003eSchwartz, I. S., \u0026amp; Baer, D. M. (1991). Social validity assessments: Is current practice state of the art? \u003cem\u003eJournal of Applied Behavior Analysis,\u003c/em\u003e \u003cem\u003e24\u003c/em\u003e(2), 189\u0026ndash;204. https://doi.org/10.1901/jaba.1991.24-189\u003c/li\u003e\n\u003cli\u003eShepley, C., Shepley, S., Spriggs, A., \u0026amp; O\u0026apos;Neill, K. (2025). Generalization in Single-Case Research: A Review of Inclusion, Prevalence, and Rigor. \u003cem\u003eSingle Case in the Social Sciences\u003c/em\u003e, \u003cem\u003e2\u003c/em\u003e(2), 38-53. https://doi.org/10.15763/issn.1936-9298.2025.2.2.38-53\u003c/li\u003e\n\u003cli\u003eSnodgrass, M. R., Chung, M. Y., Meadan, H., \u0026amp; Halle, J. W. (2018). Social validity in single-case research: A systematic literature review of prevalence and application. \u003cem\u003eResearch in Developmental Disabilities, 74\u003c/em\u003e, 160\u0026ndash;173. https://doi.org/10.1016/j.ridd.2018.01.007\u003c/li\u003e\n\u003cli\u003eStokes, T. F., \u0026amp; Baer, D. M. (1977). An implicit technology of generalization. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e, 349-367. doi:10.1901/jaba.1977.10-349\u003c/li\u003e\n\u003cli\u003eTriangulation. 2025. In \u003cem\u003eMerriam-Webster.com\u003c/em\u003e. Retrieved May 8, 2025, from https://www.merriam-webster.com/dictionary/triangulation\u003c/li\u003e\n\u003cli\u003eVolkert, V. M., Lerman, D. C., Call, N. A., \u0026amp; Trosclair‐Lasserre, N. (2009). An evaluation of resurgence during treatment with functional communication training. \u003cem\u003eJournal of Applied Behavior Analysis\u003c/em\u003e, \u003cem\u003e42\u003c/em\u003e(1), 145-160. doi:10.1901/jaba.2009.42-145\u003c/li\u003e\n\u003cli\u003eYoder, P., Lloyd, B., \u0026amp; Symons, F. (2018). \u003cem\u003eObservational measurement of behavior\u003c/em\u003e (2\u003csup\u003end\u003c/sup\u003e Edition). Paul H. Brookes Publishing Co\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-behavioral-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jobe","sideBox":"Learn more about [Journal of Behavioral Education](http://link.springer.com/journal/10864)","snPcode":"10864","submissionUrl":"https://submission.springernature.com/new-submission/10864/3","title":"Journal of Behavioral Education","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Generalization, functional analysis, intensive intervention, data based decision making, social validity","lastPublishedDoi":"10.21203/rs.3.rs-7005035/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7005035/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Service-delivery models that promote collaboration with families and systematically program for generalization may produce socially significant and generalized outcomes. However, valid empirical demonstrations of generality are difficult to generate. For practitioners invested in evaluating the molar impact of their effort, quantitative measures reflecting caregiver perception of change could be a valuable metric for guiding formative triangulation processes. In this paper, we presented a fully integrated mixed methods case study using convergent-design elements to demonstrate how incorporating qualitative and quantitative reports of caregiver perceptions of the frequency and intensity of aggression outside of formal therapy sessions can inform socially valid and generalized outcomes through formative triangulation.","manuscriptTitle":"Improving the Process \u0026amp; Product of Intensive Intervention Through Formative Triangulation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-14 11:36:04","doi":"10.21203/rs.3.rs-7005035/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-12-14T20:57:48+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-01T19:00:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"158859133467755424321690798530872041268","date":"2025-10-27T13:19:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"179982681689335953540504371682065802241","date":"2025-10-19T15:21:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-13T12:06:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"222821865772980301994511862848006807914","date":"2025-10-02T14:32:36+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-03T19:35:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"319142551888613498303766428228138735490","date":"2025-07-11T22:30:16+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"163380464418983911320651814333358782621","date":"2025-07-10T01:21:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"219079342899701089423105897015809930679","date":"2025-07-09T21:25:24+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-09T21:06:43+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-07T16:26:12+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-30T12:57:31+00:00","index":"","fulltext":""},{"type":"submitted","content":"Journal of Behavioral Education","date":"2025-06-29T23:24:35+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-behavioral-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jobe","sideBox":"Learn more about [Journal of Behavioral Education](http://link.springer.com/journal/10864)","snPcode":"10864","submissionUrl":"https://submission.springernature.com/new-submission/10864/3","title":"Journal of Behavioral Education","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"aafc5e0a-83f0-4ab4-8939-7d78d2ea273c","owner":[],"postedDate":"July 14th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-02-10T14:38:32+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-14 11:36:04","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7005035","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7005035","identity":"rs-7005035","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}