Concussion is a significant public health concern, which impacts more than 10% of Canadian school-aged children each year [1,2]. Sport is a leading cause, accounting for 30%‐40% of concussions among Canadian youth [3-5]. In Ontario, Canada, Rowan’s Law (Bill 193) guides concussion safety practices, including a mandate for medical clearance of youth returning to sport after concussion [6]. Similar policies exist in all US states [7] and are recommended across Canadian provinces [8,9]. Yet, evaluating concussion recovery and readiness for return-to-play (return-to-play) is challenging, as there is no single objective test and limited evidence-based standards to support decision-making [10-12]. Evidence suggests that current approaches are inadequate, as injury risk remains significantly elevated in the year following return-to-play postconcussion, indicating potential ongoing neurological dysfunction [13,14]. Many athletes also report feeling anxiety or fear upon return-to-play, highlighting the importance of psychological readiness [15,16]. Better tools are clearly needed to measure complete biopsychosocial recovery and guide safe return-to-play after concussion.

Emerging research shows that novel assessment paradigms simultaneously targeting multiple domains of function (eg, cognitive and motor performance) can help uncover lingering subclinical neurophysiological changes after concussion [12,17]. Dual task tests involving concurrent motor (eg, walking) and cognitive (eg, backward spelling) tasks elicit subtle neuromuscular and attentional impairments among individuals with concussion that persist beyond traditional single-domain measures for several months after injury and are linked to subsequent injury risk [17-21]. Deficits include changes in gait speed or stability, cognitive accuracy, and reaction time, which are quantified in dual task costs by the magnitude of performance decrements upon introduction of a second task [17,22-24]. These performance “costs” become more pronounced as cognitive task complexity increases [25]. It is thus problematic that standard dual task tests do not reflect the speed or complexity of skill integration in sport, which involves multiplanar movement and spontaneous decision-making in response to dynamic stimuli, within an unpredictable environment [26,27]. Emerging digital approaches such as virtual reality offer promising potential for creating more realistic testing scenarios but face challenges in clinical translation (eg, cost or availability, personalization, or scoring complexity) and have yet to be adapted for return-to-play (see the review by DuPlessis et al [26] of existing multidomain assessments). Thus, ecologically relevant and clinically feasible multidomain assessments that mimic the complex demands of sport may help guide rehabilitation and safe return-to-play following concussion [26].

Our team developed R2Play, a multidomain return-to-play assessment tool for youth with concussion, designed to help simulate the complex demands of sport within clinical settings [28]. R2Play was developed through a user-centered design approach involving clinicians, sport coaches, youth athletes, and families with lived experience of concussion. Through this process, R2Play was established as a system of low-cost tablets displaying numbers and letters that youth run between to connect an alphanumeric “trail” (1-A-2-B-3-C…) through levels with layered challenges to perception, cognition, and exertion. Heart rate (HR), perceived exertion, and concussion symptoms are monitored throughout R2Play, with performance measured via level completion times, errors, and multitask cost scores reflecting performance changes between levels with the addition of new challenges. Using low-cost and widely accessible technology, R2Play aims to provide a clinically feasible return-to-play assessment with greater ecological relevance through integration of physical skills (multiplanar exertion), cognition (complex decision-making), and psychosocial stress (performance pressure) within a dynamic, changing environment. A complete description of the R2Play development process and initial prototype can be found in DuPlessis et al [28]

Feasibility testing is an important step in complex intervention development that can help improve design, strengthen evaluation methods, and facilitate implementation [29,30]. For technology-based rehabilitation innovations, progressive small-scale studies involving end users are recommended to help identify and proactively address usability and feasibility issues before broader-scaled evaluation [31]. We conducted initial proof-of-concept testing for R2Play (version 1) with 5 clinicians and 10 youth sport participants without recent concussion history, demonstrating alignment with system design objectives, including excellent usability, moderate intensity physical exertion demands, and promising qualitative feedback regarding its potential value in clinical practice [32]. Based on this preliminary work, minor changes were made to improve R2Play (eg, enhancing the usability of interface screens, refining level structure, and shortening the assessment). In the present study, we sought to test the revised R2Play system (version 2) among youth previously cleared to return-to-play, a critical step to enable larger validation studies among youth recovering from concussion.

The purpose of this study was to evaluate the feasibility and face validity of R2Play (version 2) among youth with concussion history and prior clearance to return-to-play. We conceptualized face validity as the extent to which R2Play feels like sport, a valuable early indicator of how well it reflects multidomain demands of sport and the extent to which it may apply to real-world sport environments (ie, ecological validity) [28,33]. Specific objectives were:

Iterative design and testing are essential to ensure success in large-scale validation and implementation of health technologies [31]. Addressing these aims is thus intended to help optimize the design of R2Play, increasing the likelihood of achieving favorable outcomes in larger-scale psychometric validation, efficacy, and implementation trials.

This study followed a convergent parallel mixed methods design in which quantitative and qualitative data were collected simultaneously, analyzed separately, and merged upon interpretation with equal priority [34]. Specifically, we leveraged a descriptive observational design [35] for quantitative evaluation and used qualitative description [36] to explore participant perspectives. From a pragmatist perspective, this approach sought to obtain a comprehensive understanding of research questions through triangulation across methods and datasets [37]. It aimed to address not only whether R2Play could work, but also how and why it may work, with the goal of gathering data and feedback to optimize this novel assessment approach [38]. Broadly, we operationalized feasibility according to domains described by Bowen et al [39], which are reflected in both quantitative and qualitative components. The GRAMMS (Good Reporting of a Mixed Methods Study) checklist (Checklist 1) was used to guide study design and dissemination [40].

Identical samples were used for quantitative and qualitative strands [41]. Ten youth previously cleared to return-to-play after concussions were each paired with 1 of 5 trained clinicians to complete R2Play together. These sample sizes reflect the exploratory nature of the quantitative objectives and align with recommendations for qualitative interview studies [42]. In our proof-of-concept work, qualitative data saturation was reached with the same sample sizes and study design [32].

Youth inclusion criteria were age 10‐25 years (consistent with WHO definition of “young person” [43]), as younger children require unique developmentally appropriate assessments [12]; experience with return-to-play process after concussion within the previous 5 years (any mechanism of injury); clinician clearance for unrestricted RTP (per current standard of practice [12]); normal or corrected to normal vision and hearing; fluent in English. Youth were excluded if they had any pre-existing conditions that could impede safe participation (eg, musculoskeletal injury or neurological or cardiovascular condition) or a developmental or learning disability that interfered with their ability to do physical activities, hear noises, follow instructions, or communicate.

Clinicians were required to have ≥1 year of experience working with youth, experience working with concussion or another acquired brain injury population, English fluency, and ability to commit to study requirements (6 h total over 4 separate days).

Convenience sampling was used, in which youth were recruited internally through the Holland Bloorview Kids Rehabilitation Hospital (HBKRH) concussion services. Youth who accessed clinic services were either referred to this study by a care team member or contacted directly by the research team following discharge if they had consented to participate in the HBKRH research notification program. Additional youth were identified via snowball sampling, word of mouth, and institutional advertisements at HBKRH. Clinicians were recruited from the local community via social media, professional networks, and word of mouth. Diversity was sought in representation across disciplines and practice settings (ie, private vs public, acute vs chronic).

Figure 1 summarizes this study’s protocol. Clinicians first attended a 1-hour virtual training session to learn to administer R2Play. Training was led by the primary author (JS) and involved a brief presentation, demonstration of the system interface, and practicing key tasks in facilitating R2Play (Checklist 2). Following training, clinicians were paired with a youth participant to complete R2Play together in person at HBKRH. After finishing the assessment, the clinician and youth participants reviewed results together with a research team member and then completed separate semistructured interviews to provide feedback. To enable practice and reflection, clinicians were asked to complete 2 assessment sessions with different youth. Within 1 week after their second session, clinicians completed an in-depth follow-up interview (30‐60 min) to reflect further on their experience and perspective, considering both R2Play sessions.

Youth wore a Polar H10 chest strap monitor (Polar Electro) to capture their HR throughout R2Play. Average and peak HR (beats per minute) were calculated for each participant and level.

Youth reported their RPE during check-ins immediately after each R2Play level using the 10-point children’s OMNI scale, a validated tool for measuring RPE among exercising youth [44].

Immediately after finishing each R2Play assessment, clinicians completed the System Usability Scale (SUS), a validated measure of subjective usability [46]. The SUS contains 10 items rated on a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). Item responses are each assigned a score contribution from 0‐4, which are summed and multiplied by 2.5 to produce a total SUS score of 100. A total SUS score of 68 is considered average, 68‐80 is good, and ≥80 excellent [47].

Screen recordings of the R2Play clinician software interface and video recordings of the entire assessment sessions were captured to provide insight regarding participant interactions with the system and to enable further investigation of technical or usability issues.

A structured template was used by research team members during all R2Play assessment sessions to record field notes capturing any technical errors, usability issues, or support required by participants.

Interviews explored participants’ experiences with R2Play and perspectives on its feasibility, face validity, potential value, and challenges. Youth were asked to reflect on the acceptability of R2Play, usability, demand (ie, perceived value), practicality (eg, ability to learn rules), and face validity (ie, whether R2Play felt like playing sports). Clinician postsession interviews focused on initial reactions to R2Play in terms of usability, training requirements, and perceived value. Clinician follow-up interviews were structured around multiple domains of feasibility (acceptability, usability, demand, implementation, practicality, adaptability, expansion, and safety) [39] and face validity. Interview guides can be found in Multimedia Appendix 3. All interviews were audio recorded, transcribed verbatim, and checked for accuracy.

Ethics approval was obtained from HBKRH (REB #20‐099) and the University of Toronto (REB #40694). All participants provided voluntary informed written consent. Efforts were undertaken to maintain privacy and confidentiality of participant data and identities. Participants received compensation in the form of gift cards.

Five clinicians and 10 youth (ages 13‐20 y) with concussion history participated (Tables 2 and 3). Quantitative and qualitative results for each feasibility domain and face validity are presented within the subsections below, each followed by a brief description of key findings from mixed methods integration (ie, meta-inferences) presented via joint displays.

Youth reported feeling very safe while completing R2Play. They appreciated frequent check-ins and breaks, providing opportunities for rest and the option to stop if needed, which were recognized as important for youth recovering from a concussion. Some youth initially felt nervous due to the novelty or complexity of R2Play, which gradually subsided after the first few tasks, underscoring the importance of the introduction phase before core assessment levels. As youth became more comfortable, they also felt their performance improved: “I was anxious when coming in, but I think after the first set of stuff, I was pretty comfortable just because it was easy to grasp and follow … I felt like there was a very notable link between my anxiety going down and performance going up” (Y6). Three youth mentioned slipping and sliding on the floor as a potential safety risk or performance impediment.

To address our intended mixing purposes of comparing (triangulation across datasets), expanding (elucidating broader understandings), and enhancing (increase meaningfulness of interpretations), we constructed a side-by-side comparison joint display (Table 6) to illustrate integrated mixed methods findings (meta-inferences) within the feasibility domains of interest [58,59].

Elements of sport that clinicians thought were missing from R2Play were sport-specific skills, navigating other competitors (ie, collision avoidance), communicating with teammates or coaches, visually tracking objects (ie, smooth pursuits), and hand-eye coordination.

Clinicians offered the following suggestions to enhance face validity of R2Play: incorporating sport-specific skills (eg, stick handling or dribbling between tablets) and surfaces (eg, testing on grass, turf, or court), having the athlete visually track and respond to an external stimulus while completing R2Play (eg, catching a ball), varying the heights of tablet targets to match sport-specific movement patterns, and displaying participants’ scores between levels to increase performance pressure.

To address our mixing purposes of comparing, expanding, and enhancing, we constructed a cross-case comparison joint display [58,59] (Figure 6) linking youth participants’ physiological exertion (ie, HR), perceived exertion (RPE), and qualitative feedback. To better understand the relationship between these data sources and how well R2Play elicits a “sport-like” level of exertion, we stratified the joint display by whether participants met predefined exertion targets for HR and RPE (see Methods).

This study established initial feasibility and face validity of R2Play, a novel multidomain return-to-play assessment tool for concussion, at a single center through a mixed-methods approach involving a small sample of clinicians and youth with concussion history (Multimedia Appendix 6). Overall, most success criteria were met across all feasibility domains, with excellent usability (SUS=84.00±6.02), vigorous intensity exertion (peak HR=78%‐99% HR_max), and no adverse events. Usability and technical issues were identified and addressed through iterative design changes, some of which are highlighted below. Clinicians and youth feedback indicated that R2Play reflects elements of sport across physical, cognitive, and perceptual domains, making it a potentially valuable tool for assessing readiness to return-to-play and informing rehabilitation treatment planning postconcussion.

The mixed-methods approach and end user involvement in this study were particularly beneficial for understanding the usability and practicality of R2Play. Concordance across data sources indicated that R2Play is highly usable for both clinicians and youth (Table 6; meta-inference #1), which aligns with our proof-of-concept study findings [32]. This is critical given the importance of ease of use for successful adoption of rehabilitation technologies [61,62]. Importantly, we found that clinicians’ confidence in using R2Play was primarily limited by being unfamiliar with the assessment protocol and that support was needed to follow the protocol, deliver standardized instructions, and interpret and explain cost score results (meta-inference #2 and 4). These challenges have been addressed through upgrades to the clinician interface (map of assessment flow and standardized instruction scripts) and developing more extensive training resources. They also highlight an opportunity for participatory design and cocreation of knowledge mobilization materials to help clinicians communicate about R2Play in a manner that is meaningful for youth and families [63-65]. Consistent with broader usability literature [66,67], R2Play appeared to be used in an easier and faster manner with practice (meta-inference #3), reflecting calls to provide users with multiple opportunities to test a prototype [68]. Future work must consider changes in usability and R2Play results upon repeat administration (ie, test-retest reliability) due to familiarization with this novel approach. Overall, these findings reflect the value of mixed methods designs involving users in testing early prototypes of technology-based rehabilitation interventions and may be useful for other teams applying similar approaches [69].

The lack of adverse events and minimal symptom exacerbation during R2Play supports its safety, corroborated by youth and clinician feedback confirming their comfort with the protocol (meta-inference #6). Safety recommendations from clinicians, including preparticipation screening and monitoring tolerance, align with best practices for clinical exercise testing and postconcussion exertion assessment [50,51,70,71]. This feedback has informed an R2Play safety plan involving contraindication screening, standardized stopping criteria, and symptom management procedures. The absence of symptom provocation across most of this sample of recovered youth suggests that task requirements are appropriate and tolerable for youth beyond return-to-play clearance, providing a useful reference for future evaluation among youth with more recent concussion history. Importantly, 1 participant reported an increase in headache that may indicate incomplete physiological recovery [12,51], underscoring the potential for R2Play to detect unresolved changes that may be missed by traditional practices. Of note, this participant had a history of multiple concussions and was only 2 months postinjury at the time of testing. Further work is required to understand the safety of this approach among youth at the time of assessment for return-to-play clearance and establish the sensitivity of R2Play for detecting symptom exacerbation associated with ongoing concussion-related neurophysiological disturbances that may only manifest in complex testing environments involving integration across functional domains [72-74].

Qualitative feedback regarding acceptability and demand highlighted opportunities for R2Play to address gaps in the return-to-play process. Variability in operationalizing recovery and RTP noted by clinicians is well documented [11,75-77], reflecting fundamental challenges in this field [10,12]. Importantly, clinicians and youth expressed concerns about whether existing assessment and rehabilitation methods fully address the multidimensional aspects of return-to-play readiness, emphasizing the integration of physical and cognitive function as an element not adequately considered in practice, despite its ubiquity in sport [78] and growing attention in research [17,79]. The comprehensive nature of R2Play and its multidomain demands were thus appreciated by clinicians who viewed it as a potentially useful tool to assess tolerance to sport-like activity within a controlled setting before youth return to sport. As noted by clinicians, this assessment approach could involve monitoring physiological (HR) and symptom responses and capturing observations about movement quality and performance during prolonged dynamic exertion. Similarly, youth also saw R2Play as potentially useful to provide a clearer picture of readiness to return-to-play during advanced stages of recovery, which could help alleviate the anxiety and fear that are common when returning to sport [15,16]. In sum, clinicians and youth with concussion experience see value in novel multidomain assessments such as R2Play to help determine readiness for higher-risk sport activities (stages 4‐6 in the return-to-play protocol) [12] and to better capture the biopsychosocial factors that may contribute to reinjury [18,80] and limit psychological readiness to return-to-play [16,81]. These findings may be useful for other teams developing novel multidomain concussion assessments.

In this study and initial proof-of-concept work [32], some clinicians spontaneously identified the cost of R2Play as a potential implementation barrier. As cost estimates were not disclosed, these concerns may reflect preconceptions about technology, given the integration of multiple devices in R2Play. Nevertheless, cost is indeed an important factor that likely limits uptake of existing multidomain assessments, which often use sophisticated technologies to enhance ecological validity or enable scoring (eg, virtual reality and motion capture) [26]. Accessible pricing and equipment were therefore a priority in designing R2Play [28]. Equipment costs for the current R2Play prototype total approximately CAD $4250 (approximately US $3000) and are expected to decrease with further engineering refinement. This is markedly lower than many traditional neurocognitive testing batteries that face challenges in adoption due to high fees for software licensing, subscription, or interpretation [82]. Economic analyses are needed to establish the value of novel return-to-play tools such as R2Play by comparing the costs of assessment (in material and time) against potential savings in reducing reinjury and improving outcomes [83]. Commercialization models involving industry partnership could also help ensure sustainable pathways for widespread scalability of these tools.

In this small sample of youth previously recovered from concussion, R2Play elicited vigorous intensity physical exertion (peak HR=78%‐99% HR_max), with target exertion criteria met for 9/10 participants. This indicates that exertional demands are sufficient for assessing exercise tolerance, an important indicator of physiological recovery and readiness to return-to-play [51,84]. As expected, HR was highest in the exercise level, consistent with the physical demands. Interestingly, RPE was similar between the exercise and Stroop levels despite different physical demands. Possible explanations include (1) participants did not distinguish between physical exertion and cognitive effort in RPE ratings [85], (2) cognitive effort from the Stroop task caused higher perceptions of physical exertion [86], or (3) fatigue across the assessment led to higher-than-expected RPE during the final (Stroop) level [87]. Future work addressing these questions will obtain separate ratings of physical and cognitive effort (now built into the system) and explore perceptions of level demands using targeted interview questions.

Mixed method integration (Figure 6) revealed discordance wherein RPE and qualitative feedback often did not match participants’ peak HR. Here, we draw on mixed methods literature to address these discordant findings [88,89]. One approach is to seek explanations from existing research [90]. An exercise physiology lens may help understand the discrepancy between physiological (objective) and perceived (subjective) exertion responses. As R2Play involves short (~30‐60 s) intervals of high intensity exercise, it is possible that the brevity of these bouts and frequent breaks could limit perceptions of effort, despite substantially elevating HR. There is indeed evidence of RPE decoupling from HR during interval training, attributed to positive feelings of relief, achievement, and enjoyment [91,92]. This is particularly relevant because the youth described R2Play as enjoyable. Another strategy to manage discordance is comprehensive case analysis [88], a form of reconciliation (reanalysis) [89] that involves identifying patterns in individual case-level data. We speculate that sport history may influence participants’ effort perceptions such that those who are accustomed to explosive activities similar to R2Play (eg, hockey or soccer) may report lower effort perception. Given our limited sample size, initiation [89] of future work is warranted to explore this hypothesis by asking participants how the exertional demands of R2Play compare to their sports. Finally, an exclusion approach questions the validity of conflicting data [89]. In this case, temporal differences may have affected data quality. While HR was measured as participants actively completed R2Play levels, RPE and qualitative feedback were captured retrospectively while participants were at rest, during breaks, and upon assessment completion, respectively. Indeed, 2 participants mentioned that breaks affected their effort perception. In future work, care should be taken to ensure that RPE ratings pertain to active level completion.

This study lays the foundation for future work toward the goal of establishing R2Play as a clinically feasible, ecologically valid, and objective return-to-play assessment. Importantly, youth in this sample were previously cleared to RTP up to 5 years before enrollment, and clinicians did not administer R2Play in the context of real clinical decision-making for return-to-play clearance. While this work provided useful insights for the current stage of development, future validation studies should be conducted at the time of RTP clearance to determine the value of R2Play in supporting clinical decision-making (eg, increasing clinician confidence) and improving outcomes such as athletes’ psychological readiness to return-to-play and risk of reinjury. Additionally, youths’ concussion histories were established via self-report, without capturing how they received return-to-play clearance or what criteria were used to establish clearance. This information should be collected in future studies to contextualize participants’ R2Play results and symptom responses in relation to best practice guidelines and the established return-to-play strategy [12].

Future work is needed to validate the level structure, scoring, and ecological relevance of R2Play. This requires understanding the physiological, neurocognitive, and psychosocial task demands, which may be best accomplished through mixed-methods approaches that combine objective measurement with participant feedback [93]. Here, “sport-like” elements were primarily established through self-reported perceptions that may not translate to actual performance validity in real-world sport contexts. In addition to HR monitoring used here to capture physiological demands, objective measures of biomechanics (eg, accelerometry) and cognitive load (eg, electroencephalogram and functional near-infrared spectroscopy) could help assess whether movement and cognitive demands in R2Play reflect those in different sport environments. Validating a complex assessment such as R2Play is a complicated undertaking that is anticipated to span multiple years with ongoing iteration. This preliminary study was performed with a small convenience sample at a single site, supported by the original system design team, which limits generalizability. The primary author who conducted clinician training was also involved in data collection and interviews, which could influence participant responses (observer expectancy bias). Furthermore, we only considered a subset of relevant feasibility domains, some of which were only explored qualitatively. Future work will consider other aspects of feasibility (eg, implementation fidelity, adaptability, and integration) [39] across care settings, with quantitative progression criteria [59]. To address these questions, a larger multicenter pilot study is now underway evaluating the cross-site feasibility of R2Play and establishing its content validity by determining the physical, cognitive, and perceptual loading of the assessment leveling structure [94].

Findings from this preliminary single-center study support the potential feasibility and face validity of R2Play, a new multidomain assessment tool for youth with concussion, demonstrating excellent usability, vigorous physical exertion demands, and promising feedback regarding its potential to fill gaps in the return-to-play process. Implementation considerations and mitigation strategies were identified related to time, cost, space, and equipment set-up. A multicenter study is underway to assess cross-site feasibility of R2Play and evaluate its content validity by establishing the physical, cognitive, and perceptual loading of assessment levels [94].