This paper is in the following e-collection/theme issue:

Cognitive and Neurorehabilitation (236) Design and Usability of Medical Devices (86) Design and Usability of Consumer Health Tech and Home Monitoring Devices (188) Assistive Technologies (99) Stroke (148) Emerging Technologies for Rehabilitation (241) Neurorehabilitation and Technology (12)

Published on in Vol 10 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/49702, first published .
A Brain Computer Interface Neuromodulatory Device for Stroke Rehabilitation: Iterative User-Centered Design Approach

A Brain Computer Interface Neuromodulatory Device for Stroke Rehabilitation: Iterative User-Centered Design Approach

A Brain Computer Interface Neuromodulatory Device for Stroke Rehabilitation: Iterative User-Centered Design Approach

Authors of this article:

Gemma Alder1 Author Orcid Image ;   Denise Taylor1 Author Orcid Image ;   Usman Rashid1 Author Orcid Image ;   Sharon Olsen1 Author Orcid Image ;   Thonia Brooks1 Author Orcid Image ;   Gareth Terry1 Author Orcid Image ;   Imran Khan Niazi1, 2, 3 Author Orcid Image ;   Nada Signal1 Author Orcid Image
Article Authors Cited by Tweetations (3) Metrics

    Original Paper

    1Rehabilitation Innovation Centre, Health and Rehabilitation Research Institute, Auckland University of Technology, Auckland, New Zealand

    2Centre for Chiropractic Research, New Zealand College of Chiropractic, Auckland, New Zealand

    3Sensory Motor Integration, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark

    Corresponding Author:

    Gemma Alder, MSc

    Rehabilitation Innovation Centre, Health and Rehabilitation Research Institute

    Auckland University of Technology

    (A-11), Private Bag 92006

    Auckland, 1142

    New Zealand

    Phone: 64 9 921 9999 ext 7815

    Email: gemma.alder@aut.ac.nz


    Background: Rehabilitation technologies for people with stroke are rapidly evolving. These technologies have the potential to support higher volumes of rehabilitation to improve outcomes for people with stroke. Despite growing evidence of their efficacy, there is a lack of uptake and sustained use in stroke rehabilitation and a call for user-centered design approaches during technology design and development. This study focuses on a novel rehabilitation technology called exciteBCI, a complex neuromodulatory wearable technology in the prototype stage that augments locomotor rehabilitation for people with stroke. The exciteBCI consists of a brain computer interface, a muscle electrical stimulator, and a mobile app.

    Objective: This study presents the evaluation phase of an iterative user-centered design approach supported by a qualitative descriptive methodology that sought to (1) explore users’ perspectives and experiences of exciteBCI and how well it fits with rehabilitation, and (2) facilitate modifications to exciteBCI design features.

    Methods: The iterative usability evaluation of exciteBCI was conducted in 2 phases. Phase 1 consisted of 3 sprint cycles consisting of single usability sessions with people with stroke (n=4) and physiotherapists (n=4). During their interactions with exciteBCI, participants used a “think-aloud” approach, followed by a semistructured interview. At the end of each sprint cycle, device requirements were gathered and the device was modified in preparation for the next cycle. Phase 2 focused on a “near-live” approach in which 2 people with stroke and 1 physiotherapist participated in a 3-week program of rehabilitation augmented by exciteBCI (n=3). Participants completed a semistructured interview at the end of the program. Data were analyzed from both phases using conventional content analysis.

    Results: Overall, participants perceived and experienced exciteBCI positively, while providing guidance for iterative changes. Five interrelated themes were identified from the data: (1) “This is rehab” illustrated that participants viewed exciteBCI as having a good fit with rehabilitation practice; (2) “Getting the most out of rehab” highlighted that exciteBCI was perceived as a means to enhance rehabilitation through increased engagement and challenge; (3) “It is a tool not a therapist,” revealed views that the technology could either enhance or disrupt the therapeutic relationship; and (4) “Weighing up the benefits versus the burden” and (5) “Don’t make me look different” emphasized important design considerations related to device set-up, use, and social acceptability.

    Conclusions: This study offers several important findings that can inform the design and implementation of rehabilitation technologies. These include (1) the design of rehabilitation technology should support the therapeutic relationship between the patient and therapist, (2) social acceptability is a design priority in rehabilitation technology but its importance varies depending on the use context, and (3) there is value in using design research methods that support understanding usability in the context of sustained use.

    JMIR Rehabil Assist Technol 2023;10:e49702

    doi:10.2196/49702

    Keywords

    user-centered design; stroke; rehabilitation technology; wearable technology; brain computer interface; BCI; mobile app; think-aloud; near live; semistructured interviews 


    Background

    Stroke is a major health, socioeconomic, and financial burden that affects over 12 million people worldwide annually [1]. Despite advances in stroke prevention, the incidence of stroke is anticipated to rise due to population growth and aging [2]. Following a stroke, up to 80% of individuals experience difficulty with locomotion [3,4]. Locomotion refers to the ability to move from one place to another [5] and encompasses a wide range of activities such as getting on and off a chair; walking indoors; climbing stairs; and navigating obstacles, terrains, and environments. While most people with stroke regain some ability to walk unassisted, less than 20% achieve unrestricted community locomotion [6,7]. Rehabilitation can reduce locomotor disability following stroke, particularly when delivered in large volumes [8-14], yet observational studies confirm the amount of rehabilitation received is limited, translating into poorer outcomes for people with stroke and consequent lifelong disability [15-20]. Thus, innovative approaches for stroke rehabilitation are required.

    The last 2 decades have witnessed the rapid development of rehabilitation technologies, such as robotics, virtual reality, neuromodulation devices, activity monitors, and mobile apps designed to augment rehabilitation after stroke. While there is evidence that these technologies can increase the amount of rehabilitation a person with stroke receives and improve outcomes [13,14,21-26], user adoption and sustained use of such technologies remains low [27-33]. The disconnect between initial efficacy and clinical translation likely relates to the usability of these technologies and their acceptability to users [34,35]. As a result, there has been a call for increased application of user-centered design approaches in the development of rehabilitation technologies [35-37]. Adopting user-centered design approaches can support the development of usable and acceptable technologies by prioritizing user needs, involving users and relevant stakeholders throughout the project life cycle, and modifying the design of the technology based on iterative user-centered evaluation [38,39].

    Noninvasive neuromodulatory interventions are rehabilitation technologies with the potential to maximize rehabilitation outcomes and reduce physical disability. Typically, these interventions involve repeated magnetic or electrical stimulation of the central and peripheral nervous systems to induce neural plasticity [40]. Noninvasive neuromodulatory interventions that target movement control have the potential to accelerate stroke recovery when combined with traditional rehabilitation [41-44]. However, such interventions often rely on complex medical devices and user interfaces operated by expert operators, and lack usability and acceptability. To maximize the potential for successful implementation in rehabilitation practice, research and development of noninvasive neuromodulatory technologies must include a user-centered approach [45]. In this paper, we present a complex neuromodulatory rehabilitation technology (exciteBCI) and its evaluation in a user-centered design research process.

    exciteBCI

    exciteBCI is a prototype, portable, medical wearable device designed to deliver neuromodulation during locomotor rehabilitation for people with stroke. The device uses a brain computer interface in which a specific electroencephalography signal, which reflects the person’s intention to move, is extracted and paired with the afferent stimulus from peripheral electrical stimulation [46,47]. The electrical stimulation is timed to coincide precisely with the electroencephalography signal in the motor cortex to induce neural plasticity [46,47].

    exciteBCI evolved from an endeavor to translate a neuromodulatory intervention that had been tested in healthy and stroke populations in a clinical research laboratory setting [46-54] into a rehabilitation device suitable for stroke rehabilitation. Prior feasibility work found that the neuromodulatory intervention, when delivered during simple ankle movements while seated, was not acceptable to people with stroke and was not feasible for rehabilitation [55]. The equipment was deemed cumbersome and uncomfortable, the set-up time was excessive, and the movement tasks were considered meaningless and boring by people with stroke. Given that qualitative evidence indicates that rehabilitation should be centered on meaningful real-world activities that reflect a person’s aspirations and should be practiced at progressively higher intensities [56,57], these perspectives have important ramifications for the implementation of the intervention in clinical practice and for ensuring sustained use.

    The iterative user-centered design process for developing exciteBCI was guided by the International Organization for Standardization (ISO) 9241-210:2010 standard [38] and was driven by a transdisciplinary team comprising physiotherapists, biomedical engineers, product designers, user experience and user interface designers, and a lived experience researcher. Initial work involved the following three phases: (1) understanding and specifying the context of use, (2) identifying user requirements, and (3) iteratively developing design solutions [58]. This paper reports the fourth stage of the ISO 9241-210:2010 standard [38]: (4) evaluating the design. The aims of this research were to (1) explore users’ perspectives and experiences of exciteBCI and how well it fits with rehabilitation, and (2) facilitate modifications to exciteBCI design features.


    Study Design

    The evaluation phase of an iterative user-centered design approach supported by a qualitative descriptive methodology was used to address the aims of this study. In this study, users were people who had experienced a stroke and physiotherapists working in stroke rehabilitation. This study consisted of 2 phases. In phase 1, a series of usability testing sprint cycles were conducted [59]. In phase 2, a “near-live” [60] testing approach was used, in which 2 participants with stroke and a physiotherapist undertook a 3-week intervention of locomotor rehabilitation augmented by exciteBCI (Figure 1).

    ‎
    Figure 1. Overview of study design, data collection procedures, and data analysis. PwS: people with stroke.

    Ethical Considerations

    The study was approved by the New Zealand Health and Disability Ethics Committee (17/NTA/177), and locality authorization was endorsed by the Auckland University of Technology Ethics Committee (17/373). Before the study, all participants provided written informed consent. The privacy and confidentiality of participants was protected by secure storage of all data and deidentification of data where feasible. Participants received an NZ $40 (US $24) gift voucher for each session they attended in acknowledgment of their contributions.

    exciteBCI Prototype

    The exciteBCI prototype evaluated in this research is intended for clinical use in collaboration with a qualified physiotherapist in an inpatient, outpatient, or community setting. exciteBCI has 3 components: 2 wearable components, including an electroencephalography headset and a muscle stimulator, and a third component, a mobile app. The 3 components communicate wirelessly (Figure 2).

    ‎
    Figure 2. The exciteBCI consists of 3 components: an electroencephalography headset, a muscle stimulator, and a mobile app, each of which communicates wirelessly.

    The electroencephalography headset included 9 gel electrodes capable of recording brain activity that was used to predict when the person with stroke was going to move. The muscle stimulator (NeuroTrac Rehab) was housed within a neoprene sleeve and worn during rehabilitation tasks to deliver electrical stimulation to a lower limb muscle. The muscle stimulator delivered the afferent stimulus which was paired with electroencephalography brain activity to induce neural plasticity. The exciteBCI app was designed to support the delivery of the intervention. It included locomotor tasks cued with an audiovisual prompt. The locomotor tasks could be selected, and the task parameters such as number of repetitions, movement speed, and rest time manipulated to create an individualized locomotor rehabilitation program. See Figure 3 for example screenshots from the exciteBCI app interface prototype v3.3. This version of the app was presented to participants during the first sprint cycle of phase 1.

    ‎
    Figure 3. Example screenshots from the exciteBCI app interface prototype v3.3. (A) Registering a new patient, (B) the task selection suite where the patient and therapist select tasks that align with the patient’s goals, (C) the parameters that can be manipulated for each task by the therapist to ensure an optimal level of task difficulty, (D) checking the impedance levels of the electroencephalography headset, (E) the muscle stimulator is connected to the app and stimulation amplitude saved, (F) the patient watches a video on how to perform the task, (G-H) the timing signal (auditory-visual cue) to get ready and execute the task, and (I) the patient completes the task difficulty and confidence ratings at the end of the task set.

    Participants

    People diagnosed with a stroke at least 6 months prior who presented with some restriction of the foot and ankle movement limiting locomotor function were recruited. People with English language limitations, cognitive, perceptual, and communication impairments, who were unable to engage in the research process even with the support of a family member or a health professional, were excluded. New Zealand registered physiotherapists with at least 5 years of professional experience in the field of neurological rehabilitation were recruited. Networking with local health care and rehabilitation providers and community advertising were used to recruit a convenience sample of participants. All participants provided written informed consent before participating in the study.

    Procedures and Data Collection

    Phase 1: Think-Aloud Sprint Cycles

    People with stroke (n=4) and physiotherapists (n=4) participated in a single 1-hour usability testing session, where they interacted with the exciteBCI prototype using a “think-aloud” approach [61], followed by a semistructured interview [62] (refer to Figure 1). At the end of each sprint cycle, user device requirements were compiled, and changes were made to the user interface and device before the next sprint cycle began. Participants were asked to use a “think-aloud” process by verbalizing their thoughts, observations, and opinions while interacting with the exciteBCI prototype in a planned series of activities (Textbox 1). The researcher’s interactions were kept to a minimum to support participants to fully engage in the “think-aloud” process, but, when necessary, the researcher prompted the participant with phrases like “tell me what you’re thinking now.” Video and audio recordings, photographs, and researcher observations (TB SO, and UR) were used to capture the think-aloud process. Semistructured interviews were conducted by 2 experienced researchers (GA and NS). The interviews were audio-recorded and focused on participants’ experiences and opinions of the device, functionality, design features, and suggested improvements. See Multimedia Appendix 1 for phase 1 indicative interview questions. Consecutive participants participated in the sprint cycles until no new insights or changes to the exciteBCI device's design features were provided [63,64].

    Textbox 1. Examples of activities used in phase 1 sprint cycles to facilitate the “think-aloud” process.

    Physiotherapists

    • Use the tablet to complete the initial client registration (10-15 min).
    • Use the tablet to design a task-specific training program for a client (10-15 min).
    • Follow the instructions to set-up the headset (10-15 min).
    • Follow the instructions to set-up the electrical stimulation and place electrodes on the tibialis anterior muscle (10-15 min).
    • Calibrate the system—complete the task-specific training while the headset records your model’s brain signals (10-20 min).
    • Complete task-specific training while your model receives the paired intervention (5-10 min).
    • Remove the headset and electrical stimulation device (5-10 min).

    People with stroke

    • Now you are set-up, follow the cue on the tablet to perform the exercises and we will record your brain signals (20-25 min).
    • Follow the cue on the tablet screen to complete the exercises while receiving the neuromodulatory intervention (20-30 min).
    • Please rate how difficult it was to perform that task and how confident you felt performing the task (5-10 min).
    • Donning and doffing of equipment (10-15 min).

    As the time spent on each activity varied across participants, a time range (min) has been listed against each activity. Similarly, the number of task-specific training exercises and associated repetitions varied across participants (2-4 exercises 20-80 repetitions per exercise). For the physiotherapist session, the model was a member of the research team.

    Phase 2: Near-Live Program of Rehabilitation

    A “near-live” testing approach [60] was carried out in which 2 people with stroke engaged in a 3-week program of locomotor rehabilitation augmented by exciteBCI (Soft Headset v2, App v3.6, Electrical Stimulator v.3). Eight 1-hour rehabilitation sessions were conducted in an outpatient clinical setting supervised by a New Zealand registered physiotherapist with 10 years of clinical experience in stroke rehabilitation (GA). Before the rehabilitation sessions, participants attended an initial assessment and planning session to establish locomotor-related goals and completed clinical outcome measures. Clinical measures included the 30-second chair stand test, 10-m walk test; 6-minute walk test; four step square test, and lower limb muscle strength testing of the ankle dorsiflexors, ankle plantar flexors, knee extensors, and hip flexors using a handheld dynamometer [65]. Clinical measures were repeated at the end of the rehabilitation program.

    The rehabilitation program was based on current evidence-based practice as recommended in the National and International Clinical Guidelines for Stroke [66-68] and included goal-oriented task-specific training of locomotor-related skills that were deemed important to the participant. The physiotherapist prescribed 3 to 4 different tasks from the suite of tasks within the tablet-based exciteBCI app per session. Informed by the principles of motor learning [69], the rehabilitation tasks were progressed over the program based on the participant’s rating of perceived difficulty for each task using a numerical visual analog scale. Task parameters were manipulated or new tasks were prescribed to achieve a challenge point of 6 to 8 out of 10 on the task difficulty visual analog scale for each task [57]. Participants completed between 30 and 100 repetitions of each task during the 1-hour session. The participants used the exciteBCI throughout the rehabilitation program, which delivered electrical stimulation to the tibialis anterior muscle to coincide with the person’s intended movement. Approximately 10 minutes of the 1-hour session was attributed to the donning and doffing of the equipment.

    All rehabilitation sessions were video-recorded, researcher observation notes (NS and TB) of the participant interactions with the exciteBCI prototype were recorded, and photographs were taken at each session. Following completion of the rehabilitation program, the participants with stroke and the physiotherapist (GA) took part in separate semistructured interviews. Interviews were conducted by 2 experienced researchers (GT and SM) who were not involved in the development of the exciteBCI. The interview focused on the participants’ opinions and experiences of using the device within a rehabilitation context and included specific questions in response to video observations of the rehabilitation program. All interviews were audio-recorded. See the Multimedia Appendix 1 section for phase 2 indicative interview questions.

    Data Analysis

    “Think-aloud” and interview data were transcribed verbatim. The transcripts and written observation notes were imported into NVivo 12 (Lumivero) computer software package [70]. Data analysis was undertaken in 2 stages. In the first stage, transcripts and videos were descriptively analyzed by the primary researcher (GA) to identify user interface and device requirements within each sprint cycle. This analysis was then discussed with the team to inform the development of the device before the next cycle. This analysis also served as a familiarization process for the second stage of analysis.

    The second stage of analysis focused on addressing the aim, exploring users’ perspectives and experiences of exciteBCI, and how well it fits with rehabilitation. This stage used a modified version of conventional content analysis [71] to analyze the data from both phases of the study. Conventional content analysis allows the researcher to immerse themselves in the data to acquire an accurate description of what participants experienced and understood about the topic at hand [62]. The data were coded inductively by the primary researcher (GA) at the sentence or phrase level, and a semantic coding framework was iteratively developed during the data analysis process [72]. Several activities were used to enhance the understanding of code relationships, such as continuous comparisons within and between codes and data sources, as well as the practice of memoing to capture initial insights about the data and potential interactions among codes [73]. The coded data iteratively informed the development of categories. Categories and representative coded data were visually represented using a mind map in the MIRO Application to support the development of themes. The coded data, categories, and prototype themes were reviewed and discussed with 2 researchers (NS and GT) in a series of analysis meetings to ensure consistency of interpretation.

    Results

    Overall

    A total of 11 people participated in the study with no reported adverse events. Table 1 presents the demographic and clinical characteristics of the participants, and Figure 4 displays photographs of participants interacting with the device in phases 1 and 2 of usability testing. In the interest of intellectual property protection, the iterative device requirements for the user interface and the device are not presented in this paper.

    Table 1. Participant characteristics (n=11).
    Demographic and clinical characteristicsValue
    People with stroke (n=6)

    		Type of stroke, n (%)


    		Ischemic3 (50)


    		Hemorrhagic3 (50)

    		Lesion location, n (%)


    		Right hemisphere4 (67)


    		Left hemisphere2 (33)

    		Time since stroke (y), mean (SD; range)7 (6.7; 2-19)

    		Types of impairments, n (%)a


    		Motor6 (100)


    		Sensory4 (67)


    		Perceptual2 (33)


    		Cognition2 (33)


    		Communication1 (17)

    		Functional ambulation category scores [74], n (%)


    		0b0 (0)


    		1c0 (0)


    		2d1 (17)


    		3e1 (17)


    		4f3 (50)


    		5g1 (17)

    		Prior experience of technology components, n (%)


    		FESh



    		Clinical4 (67)



    		Research3 (50)



    		No experience1 (17)


    		BCIi



    		Clinical0 (0)



    		Research3 (50)



    		No experience3 (50)


    		Mobile app–based interventions



    		Clinical0 (0)



    		Research0 (0)



    		No experience6 (100)

    		Sex, n (%)


    		Female3 (50)


    		Male3 (50)

    		Ethnicity, n (%)


    		Asian1 (17)


    		European2 (33)


    		New Zealander3 (50)

    		Age (y), n (%)


    		<451 (17)


    		45-652 (33)


    		>653 (50)
    Physiotherapists (n=5), n (%)

    		Years qualified as a physiotherapist


    		5-101 (20)


    		10-203 (60)


    		>201 (20)

    		Highest qualification


    		Bachelor of Science4 (80)


    		Masters1 (20)

    		Years of clinical experience in stroke rehabilitation


    		5-103 (60)


    		10-201 (20)


    		>201 (20)

    		Prior experience of technology components


    		FESj



    		2-125 (100)


    		BCIk



    		3 (research context)1 (20)l


    		Mobile apps for rehabilitationm



    		3-105 (100)

    		Sex

    		Female5 (100)

    		Ethnicity


    		European2 (40)


    		New Zealander3 (60)

    an>6 due to some participants presenting with multiple impairments.

    bNonfunctional walker (unable to walk).

    cDependent walker requires continues manual contact.

    dDependent walker requires intermittent manual contact.

    eDependent walker requires verbal supervision or guiding.

    fIndependent walker on level surfaces only.

    gIndependent walker on any surface.

    hFES: functional electrical stimulation (n>6 due to experience in more than 1 category).

    iBCI: brain computer interface.

    jYears experience using functional electrical stimulation.

    kYears experience using brain computer interfaces.

    lPrimary author: GA.

    mYears experience using mobile apps for rehabilitation. Video mobile apps to capture patient performance and exercise provision mobile apps.

    ‎
    Figure 4. Photographs of participants from phase 1 sprint cycles and phase 2 “near-live” 3-week program interacting with the exciteBCI device. (A) A participant with stroke and physiotherapist work in partnership to select task-specific training exercises in the exciteBCI app, (B) a physiotherapist participant setting up the electroencephalography headset, (C) a physiotherapist setting up the muscle stimulator on a participant with stroke, (D-F) participants with stroke engaging in locomotor rehabilitation while wearing the exciteBCI device and receiving the neuromodulatory intervention with the physiotherapist, (G) a participant with stroke using the app rating scale of perceived rehabilitation task difficulty to inform the physiotherapist about the challenge-point of the task.

    Overall, the findings showed that participants with stroke and physiotherapists had positive perceptions and experiences of the exciteBCI intervention and could see it being used in a rehabilitation context. Five themes were generated from the data, as illustrated in Figure 5. Central was the theme (1) This is rehab, which interacted and was influenced by the themes (2) Getting the most out of rehab, (3) It is a tool not a therapist, (4) Weighing up the benefits versus the burden, and (5) Don’t make me look different. Illustrative quotes that corroborated the data have been selected for thematic representation, and pseudonyms have been used.

    ‎
    Figure 5. The relationship of themes associated with the users’ perceptions and experiences of exciteBCI, and how well it fits with rehabilitation.
    Theme 1: This Is Rehab

    At the core of the findings was the theme, “This is Rehab.” Despite being a novel technology incorporating a brain computer interface and app, both physiotherapists and people with stroke identified that the exciteBCI was clearly a rehabilitative tool suitable for supporting and augmenting rehabilitation practice:

    At first, I thought, it’s not a new treatment [...] we’re doing something that’s extra, but it’s basically just facilitating what we’re doing anyway.
    [Sarah, Physiotherapist, Sprint cycle 2]
    For clinical use, the intervention worked brilliantly.
    [Anni, Age 58, PwS, 3-wk program]

    Many expressed surprise at how congruent the technology was with their own clinical practice and understanding of rehabilitation principles. This congruence appeared to enhance perceptions of usability and acceptability. They readily identified ways in which the technology could be integrated into current rehabilitation practices:

    It doesn’t seem to be over complicated, and once you use it on a daily basis more or less it’s actually not that difficult.
    [Zoe Physiotherapist, Sprint cycle 2]

    Participants were easily able to understand the purpose and mechanism of the intervention. Physiotherapist participants drew on their understanding of functional electrical stimulation (FES), a modality commonly used in clinical practice, in understanding the physiological underpinnings of the paired neuromodulatory intervention. They identified that the exciteBCI device could offer benefits over FES, particularly in relation to the way the person’s own brain signals are used to drive the delivery of electrical simulation, and suggested that this would enhance outcomes from rehabilitation:

    With an external stimulus like the FES hand switch, I need to try and time it right. So, it’s not that it’s not inconvenient it’s just[...] makes more sense to take the relationship internally rather than externally. It’s the patients driving it[...] And I think that will facilitate learning more.
    [Sarah, Physiotherapist, Sprint cycle 2]

    Physiotherapists described how the exciteBCI app would support them in delivering effective rehabilitation by supporting their clinical reasoning process, enabling specificity in the design of rehabilitation programs, and promoting efficiency. They particularly valued how the exciteBCI app aided them in thinking about different aspects of task-specific training. However, they also consistently emphasized the importance of tailoring rehabilitation to individuals.

    This prompted them to describe additional app features which might support further personalization:

    What this shows [points to video playing task], it’s the goal of doing it perfectly well, but it might not look like that for them [the client]. Would be great if the actual client performing the task can be videotaped as well.
    [Sarah, Physiotherapist Sprint cycle 2]
    Shopping bags, washing basket so they are the things that we would [...] quite commonly do, but [...] when I choose secondary tasks, I often choose things that I know this patient is going to do and relates to their goals [...] so a customize option is essential
    [Zoe, Physiotherapist Sprint cycle 2]

    Attention to an efficient workflow, minimizing duplication of information being entered, and system interoperability were also priorities for physiotherapists. They particularly valued how the exciteBCI app could support efficient and comprehensive clinical record keeping and support handover between therapists:

    I would think about a lot of these different parameters, but I might not be as explicit about them in my notetaking [...] this is way more thorough[...] structured[...] easy to follow, easy to pick up on for next time[...] and so much quicker.
    [Megan, Physiotherapist, Sprint cycle 1]

    Physiotherapists identified the challenges associated with the use of clinical terminology and language, calling for app features such as icons and pop-up definitions, which would support users to have a clear understanding of key terms:

    Accuracy? so I’ve got large target, small target, wide path, small path [...] so path I presume means width? [...] I think photos or icons is a really good thing to just clarify stuff- probably more than words.
    [Megan, Physiotherapist, Sprint cycle 1]

    Most physiotherapists also described the need for prior hands-on training opportunities led by an experienced clinical expert, specifically addressing the rehabilitation approach, device set-up, and troubleshooting. This was deemed essential to support the successful uptake of the technology in rehabilitation. In addition, they recommended incorporating training and troubleshooting videos directly into the app.

    Theme 2: Getting the Most Out of Rehab

    People with stroke and physiotherapists described the ways in which the exciteBCI, and in particular the app, could support people with stroke to engage in more intensive and challenging rehabilitation at higher doses. Participants valued the way exciteBCI supported people with stroke to work hard. This was achieved in multiple ways. First, the app enabled physiotherapists to design specific and challenging rehabilitation programs for patients:

    It just was a really nice, structured way of [...] finding that sweet spot to maintain challenge, but for the participants to feel they are making progress and all that hard work is paying off.
    [Gemma, Physiotherapist, 3-wk program]

    Second, the task difficulty rating scales supported both people with stroke and physiotherapists to judge the challenge of each task and served as a prompt to work on more challenging tasks:

    [...]and the rating scale is really helpful too. You have a bit more about what’s challenging for them and what’s not. Gives you a better starting point.
    [Zoe, Physiotherapist, Sprint cycle 2]

    Third, the audiovisual cue that prompted the onset of movement compelled people with stroke to remain focused. By setting the time of task practice, limiting the amount of rest, and promoting attention on the task, large volumes of rehabilitation were achieved:

    [...]I think the way that the system has been constructed is quite focus oriented...It’s actually keeping the time [...] you get so much more in [the session].
    [Anni, Age 58, PwS, 3-wk program]

    Together, these factors supported people with stroke to work at intensities and volumes beyond what they would normally achieve during rehabilitation. This supported substantial gains in balance and walking and, in turn, built self-efficacy:

    I felt a sense of achievement you know, and that’s what’s important, I’m still making progress.
    [Jake, Age 44, PwS, 3-wk program]

    These findings illustrate that both people with stroke and physiotherapists discovered a number of features within the app that support fundamental rehabilitation principles.

    Theme 3: It Is a Tool, Not a Therapist

    Participants with stroke emphasized that developing a trusted relationship with their therapist was fundamental to their rehabilitative journey. They discussed ways in which the app might support or disrupt this relationship. Participants in phase 1 raised the possibility that the app could disrupt the therapeutic relationship:

    [...]Physically having the device and therapist there complicates the relationship...so the therapist really needs to make sure their cues show engagement and interest[...] there is a risk here.
    [Thonia, Lived experience researcher]

    While those who participated in the 3-week program did not highlight the same concern, all participants stressed that the app was a therapeutic tool that should not be viewed as a substitute for the therapist:

    [...] if you feel I handed over the therapy session to this [app] and I've cognitively left the building, then that is disastrous, to our relationship and their treatment. If I'm not engaged, why would they want to be
    [Jude, Physiotherapist, Sprint cycle 3]

    Most participants indicated that they would be happy to use the app independently; however, the majority indicated a preference for using the device with the physiotherapist present. They described how the physiotherapist offered guidance and feedback about their performance and progress:

    So, if I'm not doing a sit to stand correctly [points at app] then who is going to correct me? [...] I like the feedback from the therapist, because at some point the quality of the movement does matter.
    [Lilly, Age 64, PwS, Sprint cycle 1]

    Participants also described how the physiotherapist motivated them to work harder by encouraging them to do their best work, while also understanding their personal limits in a way that the app could not:

    The physiotherapist can push you further. They can see that you can be pushed extra. Which is important [...] people get tired, aagh and ready to give up, whereas the therapist goes another 10-minutes. Helps you squeeze out that last little bit.
    [Bob, Age 84, PwS, Sprint cycle 1]

    Consequently, some participants felt that the exciteBCI either should not or could not be used without the support of a physiotherapist. These findings illustrate that attention to the impact of technology on the therapeutic relationship during both the design and implementation of rehabilitation technologies is essential.

    Theme 4: Weighing up the Benefits Versus the Burden

    While both people with stroke and physiotherapist participants saw the benefits and potential of exciteBCI, they also called for “real-world evidence” of effect, research-based evidence, or endorsement from a trusted source:

    [...] If I knew it was going to bring about a speedier recovery, I would be more likely to use it.
    [James, Age 74, PwS, Sprint cycle 2]
    If my physio turned up with it, I wouldn’t mind at all.
    [Jenny, Age 67, PwS, Sprint cycle 2]

    Those who participated in the 3-week program drew on their own experiences of the intervention to generate real-world evidence:

    When we visited French festival, I was helped up onto the raised platform advertising electric cars [...] Renault Twizy. I don’t think I was pressing down on my husband’s hand as much, on ‘reaching’ up, and managed to squeeze myself into the car-space for a photo or two, [...] great! Extending my reach, even just a wee bit, was brilliant.
    [Anni, Age 58, PwS, 3-wk program]

    Physiotherapists described how they would consider cost, client suitability, set-up time, workflow, and clinical effectiveness when deciding whether to adopt the exciteBCI intervention:

    I would need to know it was going to make a really big difference to invest in purchasing it and have a fair amount of clients I could use it with. [...] it’s difficult without actually using it in clinical practice as to know sort of who would really benefit.
    [Megan, Physiotherapist, Sprint cycle 1]

    While participants with stroke called for the device to be integrated into their rehabilitation as early after stroke as possible, some physiotherapists saw a tension between the set-up time of both the headset and the app and the decision to implement the device in different clinical contexts:

    I don’t think[…] you’d use it on an acute ward[...] I think it takes too much time [...] based on how much time[...] physios have based in my experience[...] I just think time and all the equipment there and everything else that goes on. I don’t, can’t see it working. In a rehab ward maybe. Definitely outpatients.
    [Jude, Physiotherapist, Sprint cycle 3]

    When discussing use of the device in the home, participants with stroke and physiotherapists from phase 1 (single session) viewed the headset as a potential barrier to adopting the device. This was mainly due to the perceived difficulty of setting it up independently for those with upper limb disability and the need for gel to be inserted into the electroencephalography electrodes while wearing the headset. The set-up of the electrical stimulator and app was not viewed in the same light. While people with stroke who participated in the 3-week program expressed similar concerns, they were eager to offer suggestions to make it a viable option for independent use.

    Theme 5: Do Not Make Me Look Different

    Almost all participants emphasized how the social acceptability of the device would influence their desire to use it. The need for a socially acceptable device design was less important in a clinical setting or at home than if the device was being used in a public or social context:

    It’s [the headset] not for glamour it’s for results [...] it’s the job it’s doing, reading the brain. Glamour doesn't matter, does it? If there’s certain areas of the brain it can read, it’s got to work. It doesn’t matter what it looks like [...] not whilst you’re doing rehab. You’re there for the rehab.
    [Lilly, Age 64, PwS, Sprint cycle 1]

    Nevertheless, most participants called for a device which did not draw attention to themselves or their disabilities:

    Many people find it difficult to approach you, when you walk differently, or they treat you differently [...] So, you can understand why I would value a design that doesn't make me look even more different than I already do.
    [Jenny, 67 years; PwS, Sprint cycle 2]

    Participants posited that what constitutes a socially acceptable device might also vary depending on a person’s gender, age, and culture. The length of time since their stroke diagnosis and how much rehabilitation was prioritized in their daily lives also appeared to influence the participants’ perspectives on whether they would consider using the wearable device in their daily lives:

    Initially I would [during inpatient rehab] [...] you’re ready to take anything that you think will help but what shifts the balance of that is I think it’s about - I’m more progressed now and more - I want people to see me where I’m at now.
    [James, Age 74, PwS, Sprint cycle 2]

    Concerns about the social acceptability of the device largely pertained to the esthetics of the headset and electroencephalography gel. The current headset design was viewed as unacceptable for wearing-out in public, where social perceptions play a role. Minimizing and concealing the device with clothing or incorporating it into something that looked familiar and fits with everyday life, such as a hat or headphones, was seen as an important future design consideration. Participants from phase 1 perceived there to be no issue with the use of electroencephalography gel in a rehabilitation environment. While participants in the 3-week program described the inconvenience of repeated gel use and its impact outside of the rehabilitation context, they perceived that the benefits they experienced from the intervention outweighed this inconvenience. In contrast, participants perceived the electrical stimulator to be more acceptable. It was noted that the electrical stimulator and its neoprene housing resembled sports braces which were considered socially acceptable and in common use.

    Discussion

    Principal Findings

    This study applied a user-centered design approach to explore the perspectives and experiences of people with stroke and physiotherapists when engaging with exciteBCI rehabilitation technology and how well it fits within a stroke rehabilitation context. The results support the acceptability of the exciteBCI intervention and its “fit” with clinical practice and will inform the requirements for future device development. These findings also provide key insights that can inform the design and implementation of rehabilitation technologies more broadly.

    Technology and the Therapeutic Relationship

    This study highlights the importance of considering the impact of technology on the therapeutic relationship between patients and therapists. The therapeutic relationship refers to the relational process that takes place during clinical interactions and is considered a critical aspect of rehabilitation by both patients and therapists [75-77]. Qualitative research in neurorehabilitation indicates that positive therapeutic relationships are pivotal in supporting patient engagement, positive patient experiences, and in enhancing patient outcomes [78,79]. The “It is a tool, not a therapist” theme identified that both people with stroke and physiotherapists who participated in the phase 1 single usability sessions cautioned that the exciteBCI app could be disruptive to the therapeutic relationship. Similar concerns have been raised in other health technology domains, citing disruptions of the therapeutic relationship as a potential barrier to technology adoption [80-83]. In contrast, in this study, participants who used the exciteBCI app over multiple sessions in phase 2 highlighted how the technology could be successfully integrated into clinical interactions to support and enhance the therapeutic relationship. The participants described how using the rating scales within the exciteBCI app facilitated a shared understanding of the rehabilitative challenge, allowing the person with stroke to take control of their rehabilitation and identify opportunities for progressing rehabilitation challenge in collaboration with the physiotherapist. However, this required the physiotherapist and the person with stroke to be mindful of the role of the technology, its value, and limitations, and to use it purposefully and appropriately.

    The current findings combined with previous literature emphasize how crucial it is to take the patient, therapist, and technology triad into account when designing and implementing rehabilitation technology. While research in telehealth has attended to the influence of technology on the therapeutic relationship over time and its implications for usability [84-86], the same attention has not been given to the development of rehabilitation technology devices. Understanding the influence of technology on clinical interactions and workflow, and the ways in which these change over time, is critical. Being attuned to the impact rehabilitation technologies have on therapeutic relationships requires a deep understanding of the role of the therapist and patient and the rationale for the technology. This knowledge should be applied in two ways: (1) designers and developers should explicitly consider how the design of the technology can support and strengthen the therapeutic relationship, and (2) therapists and health care educators should consider how rehabilitation technologies can be used to support person-centered rehabilitation, ensuring that the therapeutic relationship is preserved and developed throughout the rehabilitation process.

    Considerations for Social Acceptability in Rehabilitation Technology Design

    The findings from the “Don’t make me look different” theme clearly articulate that people with stroke place weight on the esthetics and social acceptability of a rehabilitation technology device when considering whether to use it. Social acceptability may be particularly important in the design of head-mounted wearable devices [87-89]. This is an important finding given that social acceptability is often a lesser priority in the rehabilitation technology development process. Defining the target user population and understanding their device requirements early in the design and development process may resolve acceptability issues and minimize the need for significant design changes later in the process [90]. This approach has the potential to not only mitigate issues of acceptability and usability but also reduce the impact on development time and costs, while simultaneously increasing the chances of successful adoption and sustained use of the technology [45,91]. Importantly, our findings indicate that the need for a socially acceptable device design is dependent on the context in which the technology is to be used. Therefore, device developers should be cognizant of the use context when prioritizing social acceptability. Given the shift in rehabilitation services to the community [92] and supported self-management programs [93], social acceptability for both users and bystanders [87,89] is likely to be a pivotal consideration in the design of rehabilitation technologies in the future.

    User Testing in a Sustained Way

    User-centered design approaches are increasingly being encouraged to inform health technology design and development [45,94]. Yet much of the methodological literature describes single-session usability evaluation, where novice users’ perceptions and experiences of the technology inform the next iteration [95]. While this approach can highlight “entry-level” usability issues or novice user frustrations that can often be quick fixes, it is unlikely to identify fundamental usability and acceptability issues or the sources of frustrations that could be barriers to the sustained use of the technology [59,95]. This is an important limitation when using user-centered design methods for the development of rehabilitation technologies intended for sustained use. To ensure that we captured design requirements that support both the adoption and sustained use of the exciteBCI technology, in addition to the iterative single usability evaluation sessions, we conducted “near-live” usability testing over a 3-week period. This approach allowed us to investigate the technology’s “fit” with a program of rehabilitation. Users’ long-term experiences and the ways in which the device’s usability and acceptability evolved over time were elucidated.

    An important finding was that user perspectives and usability priorities shifted with sustained use of the exciteBCI technology. Long-term users were less concerned with the practicalities of the technology, such as the ease of setting up the exciteBCI app and headset. While these inconveniences were noted, they were apparently offset by the benefits users experienced from the intervention itself. It appeared that from engaging in sustained use, they obtained the “real-world evidence” in support of the technology which was desired by participants in phase 1. Long-term users also did not highlight that the technology might disrupt therapeutic relationships. Instead, they were more focused on the pleasure they had from “getting the most out of rehab,” the increase in self-efficacy they experienced, and the gains they made when the technology augmented the rehabilitation process. Similar findings have been previously reported in the literature. When meaningful connections are formed with technology over time, the pleasure derived from its use acquires more weight, and practical limitations become less significant [96]. A single-session usability assessment may not always foresee future satisfaction with the technology since it may be evaluating expectations rather than user experience [97]. In the context of this study, it is critical to remember that the user experience includes not only the technology itself, but also the experience gained from participating in the rehabilitation process, and that this experience occurs within the context of a therapeutic relationship. As a result, explicitly designing technology to support both meaningful rehabilitation and best practice from the physiotherapist to establish and nurture a therapeutic relationship is paramount.

    Our findings regarding the sustained use of exciteBCI technology also provide important guidance for its eventual implementation in clinical practice. Current models for implementing rehabilitation technologies often rely on instructional booklets and training and accreditation packages, usually provided by the technology manufacturer. While these strategies are likely to support the adoption of rehabilitation technology, they are less likely to support sustained use. Implementation science literature emphasizes the importance of strategies, such as identifying factors and barriers to sustained use, the use of clinical champions on site, and the use of behavior change strategies to ensure therapists are supported in practice change [98-100].

    Limitations of the Study

    This study adopted a rigorous approach to qualitative research; however, it is important to acknowledge that we used a convenience sampling method in this study; all participants were in the chronic stage of stroke (2-19 y since stroke onset), and the physiotherapists worked in an outpatient rehabilitation or community setting. While participants reflected on their prior lived experiences of inpatient rehabilitation closer to the onset of their stroke, when their stroke symptoms were more severe, or when physiotherapists worked in an inpatient setting, future user testing should include participants who are currently undergoing or providing inpatient rehabilitation. This will determine whether their experiences align with or differ from the device features described in this study and help to understand how well it fits in an inpatient rehabilitation setting. In phase 1, while the physiotherapists had extensive experience with the patient cohort and their associated clinical presentation, the model used during the usability evaluation was not an individual with stroke. This may have influenced the usability results. Only 1 physiotherapist (GA) participated in the “near-live” 3-week program. Future research should explore usability testing over periods of time with a range of different users. Another potential limitation of the study design was that it did not capture the initial expectations of phase 2 long-term users. Therefore, we were unable to interpret how their initial expectations may have shaped their experiences of using exciteBCI, and whether their expectations were confirmed or disregarded at the end of the program.

    Conclusions

    This study presented an iterative user-centered design approach supported by a qualitative descriptive methodology exploring users’ perspectives and experiences of exciteBCI, a complex neuromodulatory rehabilitative technology designed to augment locomotor rehabilitation for people with stroke. The 5 interrelated themes generated from the analysis revealed that overall exciteBCI was perceived and experienced positively by people with stroke and physiotherapists and viewed as technology that could be implemented in a rehabilitation context. These findings provide important insights pertinent to the broader field of rehabilitation technology design, implementation, and sustained use. Notably, these findings highlight that rehabilitation technology design should (1) consider ways to support and enhance the therapeutic relationship; (2) recognize that social acceptability is a design imperative, but its significance varies depending on the use context; and (3) that there is merit in using research design methods that explore device usability within the context of sustained use.

    Acknowledgments

    This research was partly funded by Callaghan Innovation (C Prize Challenge finalists 2017) and the Medical Technology Centre of Research Excellence.

    We gratefully acknowledge the time given to this study by the research participants. Without their willingness, this work would not have been possible. We thank Exsurgo Limited for their contributions to the technology hardware and software development of the exciteBCI device and for supplying exciteBCI illustrations. We thank Dr Suzie Mudge for interviewing GA in phase 2 of the data collection.

    Data Availability

    The data sets generated and analyzed during this study are not publicly available because of ethical approval requirements, which do not permit data sharing. Data may be obtained from the corresponding author upon reasonable request with the approval of the relevant ethics committee.

    Authors' Contributions

    GA, NS and DT conceptualized the study. GA, NS, DT, and GT designed the methodology, and DT and GA acquired ethical approval. NS, DT, UR, GA, SO, and IKN acquired funding, and GA performed project administration. GA, NS, TB, GT, UR, and SO contributed to data collection. UR and IKN were responsible for the technology. Data analysis was performed by GA and reviewed by NS and GT. Supervision was performed by NS and DT. The original draft was written by GA and NS, and edited by SO. All authors reviewed the submitted manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Indicative interview questions.

    DOCX File , 29 KB
    1. Feigin VL, Brainin M, Norrving B, Martins S, Sacco RL, Hacke W, et al. World stroke organization (WSO): global stroke fact sheet 2022. Int J Stroke. Jan 2022;17(1):18-29. [CrossRef] [Medline]
    2. Donkor ES. Stroke in the 21 century: a snapshot of the burden, epidemiology, and quality of life. Stroke Res Treat. Nov 27, 2018;2018:3238165. [FREE Full text] [CrossRef] [Medline]
    3. Duncan PW, Zorowitz R, Bates B, Choi JY, Glasberg JJ, Graham GD, et al. Management of adult stroke rehabilitation care: a clinical practice guideline. Stroke. Sep 2005;36(9):e100-e143. [CrossRef] [Medline]
    4. Veerbeek JM, Kwakkel G, van Wegen EE, Ket JC, Heymans MW. Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke. May 2011;42(5):1482-1488. [CrossRef] [Medline]
    5. Steigenberger J. Some theory towards a stringent definition of ‘locomotion’. Multibody Syst Dyn. Jan 28, 2011;26(1):81-90. [CrossRef]
    6. Perry J, Garrett M, Gronley JK, Mulroy SJ. Classification of walking handicap in the stroke population. Stroke. Jun 1995;26(6):982-989. [CrossRef] [Medline]
    7. Hill K, Ellis P, Bernhardt J, Maggs P, Hull S. Balance and mobility outcomes for stroke patients: a comprehensive audit. Aust J Physiother. 1997;43(3):173-180. [CrossRef] [Medline]
    8. Lohse KR, Lang CE, Boyd LA. Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke. Jul 2014;45(7):2053-2058. [FREE Full text] [CrossRef] [Medline]
    9. Veerbeek JM, van Wegen E, van Peppen R, van der Wees PJ, Hendriks E, Rietberg M, et al. What is the evidence for physical therapy poststroke? A systematic review and meta-analysis. PLoS One. Feb 4, 2014;9(2):e87987. [FREE Full text] [CrossRef] [Medline]
    10. Schneider EJ, Lannin NA, Ada L, Schmidt J. Increasing the amount of usual rehabilitation improves activity after stroke: a systematic review. J Physiother. Oct 2016;62(4):182-187. [FREE Full text] [CrossRef] [Medline]
    11. French B, Thomas LH, Coupe J, McMahon NE, Connell L, Harrison J, et al. Repetitive task training for improving functional ability after stroke. Cochrane Database Syst Rev. Nov 14, 2016;11(11):CD006073. [FREE Full text] [CrossRef] [Medline]
    12. English C, Hillier SL, Lynch EA. Circuit class therapy for improving mobility after stroke. Cochrane Database Syst Rev. Jun 02, 2017;6(6):CD007513. [FREE Full text] [CrossRef] [Medline]
    13. Mehrholz J, Thomas S, Elsner B. Treadmill training and body weight support for walking after stroke. Cochrane Database Syst Rev. Aug 17, 2017;8(8):CD002840. [FREE Full text] [CrossRef] [Medline]
    14. Nascimento LR, Boening A, Galli A, Polese JC, Ada L. Treadmill walking improves walking speed and distance in ambulatory people after stroke and is not inferior to overground walking: a systematic review. J Physiother. Apr 2021;67(2):95-104. [FREE Full text] [CrossRef] [Medline]
    15. Bernhardt J, Chan J, Nicola I, Collier JM. Little therapy, little physical activity: rehabilitation within the first 14 days of organized stroke unit care. J Rehabil Med. Jan 2007;39(1):43-48. [FREE Full text] [CrossRef] [Medline]
    16. Bernhardt J, Dewey H, Thrift A, Donnan G. Inactive and alone: physical activity within the first 14 days of acute stroke unit care. Stroke. Apr 2004;35(4):1005-1009. [CrossRef] [Medline]
    17. Hayward KS, Brauer SG. Dose of arm activity training during acute and subacute rehabilitation post stroke: a systematic review of the literature. Clin Rehabil. Dec 2015;29(12):1234-1243. [CrossRef] [Medline]
    18. Kaur G, English C, Hillier S. How physically active are people with stroke in physiotherapy sessions aimed at improving motor function? A systematic review. Stroke Res Treat. ;2012:820673. [FREE Full text] [CrossRef] [Medline]
    19. McNaughton H, DeJong G, Smout RJ, Melvin JL, Brandstater M. A comparison of stroke rehabilitation practice and outcomes between New Zealand and United States facilities. Arch Phys Med Rehabil. Dec 2005;86(12 Suppl 2):S115-S120. [CrossRef] [Medline]
    20. McLaren R, Signal N, Lord S, Taylor S, Henderson J, Taylor D. The volume and timing of upper limb movement in acute stroke rehabilitation: still room for improvement. Disabil Rehabil. Nov 2020;42(22):3237-3242. [CrossRef] [Medline]
    21. Pereira S, Mehta S, McIntyre A, Lobo L, Teasell RW. Functional electrical stimulation for improving gait in persons with chronic stroke. Top Stroke Rehabil. 2012;19(6):491-498. [CrossRef] [Medline]
    22. Howlett OA, Lannin NA, Ada L, McKinstry C. Functional electrical stimulation improves activity after stroke: a systematic review with meta-analysis. Arch Phys Med Rehabil. May 2015;96(5):934-943. [CrossRef] [Medline]
    23. Mehrholz J, Thomas S, Werner C, Kugler J, Pohl M, Elsner B. Electromechanical-assisted training for walking after stroke. Cochrane Database Syst Rev. May 10, 2017;5(5):CD006185. [FREE Full text] [CrossRef] [Medline]
    24. Rodrigues-Baroni JM, Nascimento LR, Ada L, Teixeira-Salmela LF. Walking training associated with virtual reality-based training increases walking speed of individuals with chronic stroke: systematic review with meta-analysis. Braz J Phys Ther. 2014;18(6):502-512. [FREE Full text] [CrossRef] [Medline]
    25. Corbetta D, Imeri F, Gatti R. Rehabilitation that incorporates virtual reality is more effective than standard rehabilitation for improving walking speed, balance and mobility after stroke: a systematic review. J Physiother. Jul 2015;61(3):117-124. [FREE Full text] [CrossRef] [Medline]
    26. Lynch EA, Jones TM, Simpson DB, Fini NA, Kuys S, Borschmann K, et al. Activity monitors for increasing physical activity in adult stroke survivors. Stroke. Jan 2019;50(1):e4-e5. [CrossRef] [Medline]
    27. Turchetti G, Vitiello N, Trieste L, Romiti S, Geisler E, Micera S. Why effectiveness of robot-mediated neurorehabilitation does not necessarily influence its adoption. IEEE Rev Biomed Eng. 2014;7:143-153. [CrossRef] [Medline]
    28. Langan J, Subryan H, Nwogu I, Cavuoto L. Reported use of technology in stroke rehabilitation by physical and occupational therapists. Disabil Rehabil Assist Technol. Oct 2018;13(7):641-647. [CrossRef] [Medline]
    29. Liu L, Miguel Cruz A, Rios Rincon A, Buttar V, Ranson Q, Goertzen D. What factors determine therapists' acceptance of new technologies for rehabilitation – a study using the Unified Theory of Acceptance and Use of Technology (UTAUT). Disabil Rehabil. 2015;37(5):447-455. [CrossRef] [Medline]
    30. Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S, et al. Designing, implementing, and evaluating mobile health technologies for managing chronic conditions in older adults: a scoping review. JMIR Mhealth Uhealth. Jun 09, 2016;4(2):e29. [FREE Full text] [CrossRef] [Medline]
    31. Riek LD. Healthcare robotics. Commun ACM. Oct 24, 2017;60(11):68-78. [CrossRef]
    32. Chua KS, Kuah CW. Innovating with rehabilitation technology in the real world: promises, potentials, and perspectives. Am J Phys Med Rehabil. Oct 2017;96(10 Suppl 1):S150-S156. [FREE Full text] [CrossRef] [Medline]
    33. Cano Porras D, Sharon H, Inzelberg R, Ziv-Ner Y, Zeilig G, Plotnik M. Advanced virtual reality-based rehabilitation of balance and gait in clinical practice. Ther Adv Chronic Dis. Aug 23, 2019;10:2040622319868379. [FREE Full text] [CrossRef] [Medline]
    34. Holloway C, Dawes H. Disrupting the world of disability: the next generation of assistive technologies and rehabilitation practices. Healthc Technol Lett. Dec 07, 2016;3(4):254-256. [FREE Full text] [CrossRef] [Medline]
    35. Kerr A, Smith M, Reid L, Baillie L. Adoption of stroke rehabilitation technologies by the user community: qualitative study. JMIR Rehabil Assist Technol. Aug 17, 2018;5(2):e15. [FREE Full text] [CrossRef] [Medline]
    36. Mawson S, Nasr N, Parker J, Zheng H, Davies R, Mountain G. Developing a personalised self-management system for post stroke rehabilitation; utilising a user-centred design methodology. Disabil Rehabil Assist Technol. Nov 2014;9(6):521-528. [CrossRef] [Medline]
    37. Lobo EH, Frølich A, Rasmussen LJ, Livingston PM, Grundy J, Abdelrazek M, et al. Understanding the methodological issues and solutions in the research design of stroke caregiving technology. Front Public Health. Apr 16, 2021;9:647249. [FREE Full text] [CrossRef] [Medline]
    38. ISO 9241-210:2019 Ergonomics of human-system interaction Part 210: human-centred design for interactive systems. International Organization for Standardization. 2019. URL: https://www.iso.org/standard/77520.html [accessed 2022-05-12]
    39. De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs. 2009;27(3):175-183. [FREE Full text] [CrossRef] [Medline]
    40. Rossini PM, Burke D, Chen R, Cohen LG, Daskalakis Z, Di Iorio R, et al. Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: basic principles and procedures for routine clinical and research application. An updated report from an I.F.C.N. Committee. Clin Neurophysiol. Jun 2015;126(6):1071-1107. [FREE Full text] [CrossRef] [Medline]
    41. Wessel MJ, Zimerman M, Hummel FC. Non-invasive brain stimulation: an interventional tool for enhancing behavioral training after stroke. Front Hum Neurosci. May 15, 2015;9:265. [FREE Full text] [CrossRef] [Medline]
    42. Ayache SS, Farhat WH, Zouari HG, Hosseini H, Mylius V, Lefaucheur JP. Stroke rehabilitation using noninvasive cortical stimulation: motor deficit. Expert Rev Neurother. Aug 09, 2012;12(8):949-972. [CrossRef] [Medline]
    43. Vaz PG, Salazar AP, Stein C, Marchese RR, Lukrafka JL, Plentz RD, et al. Noninvasive brain stimulation combined with other therapies improves gait speed after stroke: a systematic review and meta-analysis. Top Stroke Rehabil. Apr 08, 2019;26(3):201-213. [CrossRef] [Medline]
    44. Tien HH, Liu WY, Chen YL, Wu YC, Lien HY. Transcranial direct current stimulation for improving ambulation after stroke: a systematic review and meta-analysis. Int J Rehabil Res. Dec 2020;43(4):299-309. [FREE Full text] [CrossRef] [Medline]
    45. Jankowski N, Schönijahn L, Salchow C, Ivanova E, Wahl M. User-centred design as an important component of technological development. Curr Dir Biomed Eng. 2017;3(1):69-73. [CrossRef]
    46. Mrachacz-Kersting N, Kristensen SR, Niazi IK, Farina D. Precise temporal association between cortical potentials evoked by motor imagination and afference induces cortical plasticity. J Physiol. Apr 01, 2012;590(7):1669-1682. [FREE Full text] [CrossRef] [Medline]
    47. Niazi IK, Mrachacz-Kersting N, Jiang N, Dremstrup K, Farina D. Peripheral electrical stimulation triggered by self-paced detection of motor intention enhances motor evoked potentials. IEEE Trans Neural Syst Rehabil Eng. Jul 2012;20(4):595-604. [CrossRef] [Medline]
    48. Olsen S, Signal N, Niazi IK, Christensen T, Jochumsen M, Taylor D. Paired associative stimulation delivered by pairing movement-related cortical potentials with peripheral electrical stimulation: an investigation of the duration of neuromodulatory effects. Neuromodulation. Jun 2018;21(4):362-367. [CrossRef] [Medline]
    49. Olsen S, Signal N, Niazi IK, Rashid U, Alder G, Mawston G, et al. Peripheral electrical stimulation paired with movement-related cortical potentials improves isometric muscle strength and voluntary activation following stroke. Front Hum Neurosci. May 15, 2020;14:156. [FREE Full text] [CrossRef] [Medline]
    50. Jochumsen M, Cremoux S, Robinault L, Lauber J, Arceo JC, Navid MS, et al. Investigation of optimal afferent feedback modality for inducing neural plasticity with a self-paced brain-computer interface. Sensors (Basel). Nov 03, 2018;18(11):3761. [FREE Full text] [CrossRef] [Medline]
    51. Jochumsen M, Navid MS, Nedergaard RW, Signal N, Rashid U, Hassan A, et al. Self-paced online vs. cue-based offline brain-computer interfaces for inducing neural plasticity. Brain Sci. Jun 01, 2019;9(6):127. [FREE Full text] [CrossRef] [Medline]
    52. Jochumsen M, Navid MS, Rashid U, Haavik H, Niazi IK. EMG- versus EEG-triggered electrical stimulation for inducing corticospinal plasticity. IEEE Trans Neural Syst Rehabil Eng. Sep 2019;27(9):1901-1908. [CrossRef]
    53. Alder G, Signal N, Vandal AC, Olsen S, Jochumsen M, Niazi IK, et al. Investigating the intervention parameters of endogenous paired associative stimulation (ePAS). Brain Sci. Feb 12, 2021;11(2):224. [FREE Full text] [CrossRef] [Medline]
    54. Niazi IK, Navid MS, Rashid U, Amjad I, Olsen S, Haavik H, et al. Associative cued asynchronous BCI induces cortical plasticity in stroke patients. Ann Clin Transl Neurol. May 30, 2022;9(5):722-733. [FREE Full text] [CrossRef] [Medline]
    55. Olsen SA. Rewiring the brain after stroke: a novel neuromodulatory intervention to improve neuromuscular control. Auckland University of Technology. 2019. URL: http:///C:/Users/HP/Downloads/OlsenS.pdf [accessed 2023-11-16]
    56. Luker J, Lynch E, Bernhardsson S, Bennett L, Bernhardt J. Stroke survivors' experiences of physical rehabilitation: a systematic review of qualitative studies. Arch Phys Med Rehabil. Sep 2015;96(9):1698-1708.e10. [CrossRef] [Medline]
    57. Signal N, McPherson K, Lewis G, Kayes N, Saywell N, Mudge S, et al. What influences acceptability and engagement with a high intensity exercise programme for people with stroke? A qualitative descriptive study. NeuroRehabilitation. Oct 14, 2016;39(4):507-517. [CrossRef]
    58. C Prize finalists' journal. CallaghanInnovation. 2017. URL: https://www.callaghaninnovation.govt.nz/sites/all/files/cprize-2017-team-notebooks-web.pdf [accessed 2022-11-01]
    59. Hornbæk K. Current practice in measuring usability: challenges to usability studies and research. Int J Hum Comput Stud. Feb 2006;64(2):79-102. [CrossRef]
    60. Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, et al. Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support. Int J Med Inform. Nov 2012;81(11):761-772. [CrossRef] [Medline]
    61. Arsand E, Demiris G. User-centered methods for designing patient-centric self-help tools. Inform Health Soc Care. Sep 2008;33(3):158-169. [CrossRef] [Medline]
    62. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. Aug 2000;23(4):334-340. [CrossRef] [Medline]
    63. Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud. Sep 1994;41(3):385-397. [CrossRef]
    64. Virzi RA. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors. Nov 23, 2016;34(4):457-468. [CrossRef]
    65. Hill K, Denisenko S, Miller K, Clements T, Batchelor F, Morgan P. Clinical Outcome Measurement in Adult Neurological Physiotherapy. Melbourne, Australia. Australian Physiotherapy Association, National Neurology Group; 2010.
    66. Living clinical guidelines for stroke management. Stroke Foundation. URL: https://informme.org.au/Guidelines/Clinical-Guidelines-for-Stroke-Management [accessed 2023-01-15]
    67. Hebert D, Lindsay MP, McIntyre A, Kirton A, Rumney PG, Bagg S, et al. Canadian stroke best practice recommendations: stroke rehabilitation practice guidelines, update 2015. Int J Stroke. Jun 2016;11(4):459-484. [CrossRef] [Medline]
    68. Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, et al. Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. Jun 2016;47(6):e98-169. [CrossRef] [Medline]
    69. Maier M, Ballester BR, Verschure PF. Principles of neurorehabilitation after stroke based on motor learning and brain plasticity mechanisms. Front Syst Neurosci. Dec 17, 2019;13:74. [FREE Full text] [CrossRef] [Medline]
    70. NVivo (Version 12). Lumivero. URL: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home [accessed 2023-01-10]
    71. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. Nov 2005;15(9):1277-1288. [CrossRef] [Medline]
    72. Kondracki NL, Wellman NS, Amundson DR. Content analysis: review of methods and their applications in nutrition education. J Nutr Educ Behav. Jul 2002;34(4):224-230. [CrossRef] [Medline]
    73. Braun V. Successful Qualitative Research: A Practical Guide for Beginners. Thousand Oaks, CA. SAGE Publications Ltd; 2013.
    74. Mehrholz J, Wagner K, Rutte K, Meissner D, Pohl M. Predictive validity and responsiveness of the functional ambulation category in hemiparetic patients after stroke. Arch Phys Med Rehabil. Oct 2007;88(10):1314-1319. [CrossRef] [Medline]
    75. Rowlands L, Coetzer R, Turnbull OH. Building the bond: predictors of the alliance in neurorehabilitation. NeuroRehabilitation. May 21, 2020;46(3):271-285. [CrossRef]
    76. Lawton M, Haddock G, Conroy P, Sage K. Therapeutic alliances in stroke rehabilitation: a meta-ethnography. Arch Phys Med Rehabil. Nov 2016;97(11):1979-1993. [CrossRef] [Medline]
    77. Martín-Sanz MB, Salazar-de-la-Guerra RM, Cuenca-Zaldivar JN, Salcedo-Perez-Juana M, Garcia-Bravo C, Palacios-Ceña D. Person-centred care in individuals with stroke: a qualitative study using in-depth interviews. Ann Med. Dec 05, 2022;54(1):2167-2180. [FREE Full text] [CrossRef] [Medline]
    78. Terry G, Kayes N. Person centered care in neurorehabilitation: a secondary analysis. Disabil Rehabil. Aug 29, 2020;42(16):2334-2343. [CrossRef] [Medline]
    79. Kayes NM, Mcpherson K, Kersten P. Therapeutic connection in neurorehabilitation: theory, evidence and practice. In: Evidence-Based Neurology: Management of Neurological Disorders. Hoboken, NJ. John Wiley & Sons; 2015.
    80. Giunti G, Kool J, Rivera Romero O, Dorronzoro Zubiete E. Exploring the specific needs of persons with multiple sclerosis for mHealth solutions for physical activity: mixed-methods study. JMIR Mhealth Uhealth. Feb 09, 2018;6(2):e37. [FREE Full text] [CrossRef] [Medline]
    81. Pearce C, Dwan K, Arnold M, Phillips C, Trumble S. Doctor, patient and computer--a framework for the new consultation. Int J Med Inform. Jan 2009;78(1):32-38. [CrossRef] [Medline]
    82. Booth N, Robinson P, Kohannejad J. Identification of high-quality consultation practice in primary care: the effects of computer use on doctor-patient rapport. Inform Prim Care. 2004;12(2):75-83. [FREE Full text] [CrossRef] [Medline]
    83. Rortvedt D, Jacobs K. Perspectives on the use of a telehealth service-delivery model as a component of school-based occupational therapy practice: designing a user-experience. Work. Feb 07, 2019;62(1):125-131. [CrossRef]
    84. Rietdijk R, Power E, Attard M, Togher L. Acceptability of telehealth-delivered rehabilitation: experiences and perspectives of people with traumatic brain injury and their carers. J Telemed Telecare. May 28, 2020;28(2):122-134. [CrossRef]
    85. Sucala M, Schnur JB, Constantino MJ, Miller SJ, Brackman EH, Montgomery GH. The therapeutic relationship in e-therapy for mental health: a systematic review. J Med Internet Res. Aug 02, 2012;14(4):e110. [FREE Full text] [CrossRef] [Medline]
    86. Aafjes-van Doorn K, Békés V, Prout TA. Grappling with our therapeutic relationship and professional self-doubt during COVID-19: will we use video therapy again? Counsel Psychol Q. Jun 03, 2020;34(3-4):473-484. [CrossRef]
    87. Schwind V, Reinhardt R, Rzayev R, Henze N, Wolf K. Virtual reality on the go?: a study on social acceptance of VR glasses. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. Presented at: MobileHCI '18; September 3-6, 2018, 2018; Barcelona, Spain. [CrossRef]
    88. Sehrt J, Braams B, Henze N, Schwind V. Social acceptability in context: stereotypical perception of shape, body location, and usage of wearable devices. Big Data Cogn Comput. Sep 23, 2022;6(4):100. [CrossRef]
    89. Gugenheimer J, Mai C, McGill M, Williamson J, Steinicke F, Perlin K. Challenges using head-mounted displays in shared and social spaces. In: Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Presented at: CHI EA '19; May 4-9, 2019, 2019; Glasgow, Scotland. [CrossRef]
    90. Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. Feb 2005;38(1):75-87. [FREE Full text] [CrossRef] [Medline]
    91. Jackson AE, Holt RJ, Culmer PR, Makower SG, Levesley MC, Richardson RC, et al. Dual robot system for upper limb rehabilitation after stroke: the design process. Proc Inst Mech Eng C J Mech Eng Sci. Aug 10, 2016;221(7):845-857. [CrossRef]
    92. Sun X, Ding J, Dong Y, Ma X, Wang R, Jin K, et al. A survey of technologies facilitating home and community-based stroke rehabilitation. Int J Hum Comput Interact. Apr 20, 2022;39(5):1016-1042. [CrossRef]
    93. Fryer CE, Luker JA, McDonnell MN, Hillier SL. Self management programmes for quality of life in people with stroke. Cochrane Database Syst Rev. Aug 22, 2016;2016(8):CD010442. [FREE Full text] [CrossRef] [Medline]
    94. Chan K. A design thinking mindset beyond the public health model. World Med Health Policy. Mar 12, 2018;10(1):111-119. [CrossRef]
    95. Mendoza V, Novick DG. Usability over time. In: Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information. Presented at: SIGDOC '05; September 21-23, 2005, 2005; Coventry, UK. [CrossRef]
    96. Biduski D, Bellei EA, Rodriguez JP, Zaina LA, De Marchi AC. Assessing long-term user experience on a mobile health application through an in-app embedded conversation-based questionnaire. Comput Hum Behav. Mar 2020;104:106169. [CrossRef]
    97. Kujala S, Miron-Shatz T. The evolving role of expectations in long-term user experience. In: Proceedings of the 19th International Academic Mindtrek Conference. Presented at: AcademicMindTrek '15; September 22-24, 2015, 2015; Tampere, Finland. [CrossRef]
    98. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. Feb 12, 2015;10:21. [FREE Full text] [CrossRef] [Medline]
    99. Cahill LS, Carey LM, Lannin NA, Turville M, Neilson CL, Lynch EA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. Oct 15, 2020;10(10):CD012575. [FREE Full text] [CrossRef] [Medline]
    100. Murrell JE, Pisegna JL, Juckett LA. Implementation strategies and outcomes for occupational therapy in adult stroke rehabilitation: a scoping review. Implement Sci. Dec 18, 2021;16(1):105. [FREE Full text] [CrossRef] [Medline]

    ‎
    FES: functional electrical stimulation
    ISO: International Organization for Standardization

    Edited by A Mavragani; submitted 07.06.23; peer-reviewed by K Clanchy, G Nicora; comments to author 31.07.23; revised version received 03.09.23; accepted 27.09.23; published 11.12.23.

    Copyright

    ©Gemma Alder, Denise Taylor, Usman Rashid, Sharon Olsen, Thonia Brooks, Gareth Terry, Imran Khan Niazi, Nada Signal. Originally published in JMIR Rehabilitation and Assistive Technology (https://rehab.jmir.org), 11.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Rehabilitation and Assistive Technology, is properly cited. The complete bibliographic information, a link to the original publication on https://rehab.jmir.org/, as well as this copyright and license information must be included.