Physical rehabilitation is a therapeutic approach aimed at improving the quality of life for individuals who have lost mobility in one or more body parts. Rehabilitation typically involves a series of repetitive exercises using the affected limb to gradually regain mobility. However, the repetitive nature of these exercises often becomes monotonous, leading to patient boredom and, in many cases, therapy discontinuation [1].

To address these challenges, alternative interventions such as serious games have been developed. Serious games are video games designed to be both entertaining and educational [2]. They have great potential for training, motivation, and entertainment. One of their key features is the ability to simulate reality in a virtual environment, enhancing user immersion and concentration. These types of games have been used in various fields, including health, education, politics, and engineering. Specifically, serious games designed for physical rehabilitation aim to provide a more motivating environment, enhance treatment adherence, and make repetitive exercises more engaging. They provide a different context within therapeutic interventions to help patients reacquire, recover, and maintain physical capabilities, thereby improving quality of life [3]. Applications include motor recovery following stroke, gait rehabilitation, Parkinson disease, multiple sclerosis, orthopedic recovery, and physical stimulation for individuals with cerebral palsy [4,5]. These interventions have gained considerable attention among researchers and health care professionals due to their potential to address complex rehabilitation needs through innovative technologies [6].

A wide range of serious games has been developed for physical rehabilitation [7,8], which have shown positive results compared to conventional rehabilitation therapies. For instance, these games enhance the quality, enjoyment, and effectiveness of rehabilitation by providing an engaging environment and varying levels of difficulty based on injury severity. They can also overcome logistical limitations by enabling patients to engage in therapy remotely, attending clinical sessions only when necessary [5,9]. However, literature reviews show that most existing platforms focus on motor metrics, such as range of motion and kinematics, and often overlook the patients’ affective state, including frustration from pain and feelings of sadness, depression, or anxiety [9,10]. Consequently, there are significant gaps in achieving an optimal balance between game challenge and emotional engagement.

One approach to improving interaction between patients and serious games is affective computing, a type of human-computer interaction that enables systems to recognize and adapt to users’ affective states [11]. Evidence suggests that dynamic adaptation based on affective states in serious games can enhance rehabilitation outcomes by improving motivation, performance, and treatment adherence [12]. Additionally, affective computing offers real-time diagnostic capabilities that allow for personalized adjustments in gameplay in response to detected emotional cues [13].

Recent advances in affective computing focus on developing techniques to detect affective states and integrating real-time affective feedback within adaptive systems. This is especially relevant in digital health, human-computer interaction, and intelligent rehabilitation. For instance, one study shows that emotion-sensitive adaptive approaches can promote task persistence and enhance in-game performance [33]. Another review finds that machine learning models support real-time personalization of psychological treatments by including behavioral and affective signals [34]. A systematic review by Leong et al [35] focused on facial emotion recognition, evaluating classification techniques and technologies used, while emphasizing the increasing use of deep learning. Despite its contributions, the review is limited to 2 emotion recognition techniques and does not examine adaptive mechanisms within the software.

These studies do not focus specifically on physical rehabilitation, where emotional engagement is essential for treatment adherence. Therefore, there is a critical need to explore how affective computing can support empathetic interactions in rehabilitation contexts, through either the adaptation of gameplay elements or the application of novel emotion detection techniques.

This paper presents a scoping review of affective computing technologies, techniques, and methods used in serious games for physical rehabilitation, following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. It offers a comprehensive overview of the current research and highlights key gaps for future developments in this field.

To assess the current landscape of research in affective computing within serious games for physical rehabilitation, the following research questions were formulated:

RQ1: What theoretical approaches to emotion were applied in the studies, and which emotions were recognized?

RQ2: What techniques, including sensors and algorithms, were used for emotion recognition?

RQ3: Which game elements were adapted in response to affective state detection?

RQ4: What specific hardware was used to support the rehabilitation process?

Given the objective of providing a broad overview of the literature and the diverse nature of the research questions, a scoping review was determined to be the most appropriate methodological approach. Scoping reviews are particularly suitable when the volume of publications or the heterogeneity of the topic precludes more focused methodologies such as systematic reviews [36].

This study adhered to the reporting standards outlined in the PRISMA protocols extension for scoping reviews [37] (Checklist 1).

Due to the limited available literature on this particular topic and to ensure the inclusion of all relevant contributions, studies with varied designs and outcome measures were considered. The inclusion criteria were as follows: papers published in English, regardless of publication date; studies that used affective computing within a serious game for physical rehabilitation; and studies that involved at least 1 patient in the evaluation process. Participants in the included studies were required to have a condition necessitating physical rehabilitation.

Review papers (eg, systematic reviews, scoping reviews, and literature reviews) were excluded to focus on primary empirical evidence.

A comprehensive literature search was carried out in March 2024 and updated in February 2025 across the following databases: Google Scholar, PubMed, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and PEDro (Physiotherapy Evidence Database). The search strategy used the following keywords: “(affective OR emotional) AND serious (game OR games) AND rehabilitation.” The number of results retrieved from each database is presented in Table 1.

Three authors independently reviewed the records obtained through the search. Duplicate entries were removed using Mendeley Desktop (version 2.122.0) Reference Manager. Studies that met the inclusion criteria based on the title and abstract were retrieved in full text for further evaluation. Any disagreements among the reviewers were resolved by a fourth reviewer, who made the final decision.

Once the primary papers were identified, each was analyzed by at least 2 reviewers. These reviewers were responsible for extracting essential information from each study. Subsequently, the reviewers met to reach a consensus on the extracted data. The following data items were identified per paper: study, citation, affective computing technique, sensor or device used to detect affective states, population characteristics (age and sex), type of disease, type of rehabilitation, game elements adapted in response to affective states, affective states detected, machine learning algorithms or software used for affective state detection, and theoretical approaches to affective states.

The methodological quality of the selected studies was evaluated using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) tool [38], which assesses 5 domains: sample selection, index test, reference standard, flow, and timing (Multimedia Appendix 1). Risk of bias and overall study quality were reviewed by 2 authors. Any disagreements in the assessment process were resolved through discussion and consensus within the research team. Question 1 in domain 3 was modified to require at least 2 tests to validate the target condition. The question was modified because, although there is no gold standard for validating emotions, Gonçalves et al [25] suggest that 5 components should be considered: cognitive appraisals, behavioral tendencies, motor expression, physiological reactions, and subjective feelings. Although validating the emotion using the 5 components would be more effective, it is also more complex. For this reason, it was expected that at least 2 tests would be used.

A formal narrative synthesis was conducted to evaluate the influence of affective computing on the rehabilitation process.

A total of 5293 records were identified through database searching. After removing 12 duplicate records, 5281 records were screened based on title and abstract, of which 5161 were excluded. As a result of this screening process, the full texts of 120 reports were sought for retrieval, all of which were successfully obtained and assessed for eligibility. Of these, 111 reports were excluded for various reasons, including interventions involving healthy individuals, games not involving physical rehabilitation, or review papers. Finally, 9 studies were included in the systematic review (Figure 2).

The affective states detected across the studies included engagement, valence, tension, pain, satisfaction, happiness, sadness, anger, surprise, boredom, excitement, frustration, and meditation. The most frequently recognized emotions were engagement, pain, tiredness, and anxiety. Only 3 studies used theoretical frameworks of emotion. The most commonly used affective computing techniques were facial expression analysis and gesture recognition, while user behavior analysis was the least used technique. Merely 2 studies adapted serious game elements based on the affective states of users. The adaptive elements identified included modifications to background music, visual interface parameters, game difficulty, and avatar facial expressions, movements, and gestures.

Quality assessment using the QUADAS-2 tool indicated that 88% (8/9) of the included studies exhibited a high risk of bias in the patient selection category, mainly due to insufficient descriptions of recruitment procedures. Neither random nor consecutive recruitment was reported, which substantially increases the risk of selection bias. In the index test category, all studies (n=9, 100%) clearly defined the algorithms or techniques used for detecting affective states. For the reference standard, although no gold standard exists in affective computing, it is recommended to use multiple emotion recognition methods to improve accuracy [49]. However, 88% (8/9) of the studies did not use appropriate methods to assess reliability or validate their findings. In the flow and timing domain, 33% (3/9) of the studies demonstrated a high risk of bias due to the lack of explanation of the number of sessions and the failure to report participant dropout rates.

Regarding applicability, 88% (8/9) of the studies showed a high risk of bias in patient selection due to small sample sizes, limiting the generalizability of the findings. Additionally, patient recruitment from a single center further restricts the representativeness of the results for the wider patient population with varying severity levels. All studies presented low applicability concerns for the index test, as the techniques or algorithms used were clearly described. For the reference standard, 88% (8/9) of the studies exhibited low applicability concerns because they relied on clinical experts to validate emotions or used at least 2 emotion identification techniques, thereby ensuring that the system measures clinically relevant outcomes. Figure 3 illustrates the quality assessment outcomes.

Given the critical role that affective states play in the rehabilitation process, this scoping review provides a comprehensive examination of the application of affective computing techniques within serious games designed for physical rehabilitation.

Studies involving healthy participants were excluded to ensure the analysis was relevant to clinical contexts. Patients frequently experience emotions such as anxiety, depression, and fear during rehabilitation. Thus, it is crucial to assess whether recognizing and adapting to these affective states can improve therapeutic outcomes. As a result, only 9 studies were included in the final synthesis. This highlights an important opportunity for future research to design studies involving patient populations where emotional state monitoring is integrated into the rehabilitation process.

Analysis of the 9 studies shows clear differences among techniques, each with specific implications for rehabilitation: (1) gestures and motion analysis, used by Rivas et al [42-46] and Funabashi et al [47], represent the predominant technique. The main advantage is its nonintrusive nature, which uses sensors already required for therapy to measure variables such as hand position and finger pressure. This characteristic makes it suitable for routine clinical practice, as it does not increase the sensory burden on patients, although its inference capability may be less direct than that of other techniques; (2) facial expression analysis, applied in multimodal approaches by Rivas et al [42-46], helps detect states such as satisfaction or pain. However, its efficacy can be compromised when patient expressions are altered, for example, throughout intense physical exercise, which may lead to misinterpretation by monitoring algorithms; (3) EEG, as implemented by Hou and Sourina [41] and Rodriguez et al [40], directly measures brain activity and allows precise detection of affective states such as emotional valence. However, EEG used in serious games for physical rehabilitation is limited because electrodes can be intrusive for patients, and the technique is highly sensitive to noise from muscle movements. This sensitivity may limit its use during dynamic exercise in clinical settings.

A common limitation across the reviewed literature is the small sample size, which raises concerns regarding the generalizability and reliability of the reported findings. Future studies should aim for larger, more representative samples. Methodological guidelines, such as those proposed by Fritz et al [50], should be used for sample size calculation. Additionally, the reasons for participant dropout in physical rehabilitation interventions should be thoroughly documented, as this information can help inform and improve the design of future studies.

Research involving human participants is one of the most highly regulated areas in scientific inquiry, particularly regarding ethical standards. Data obtained from affective computing studies may be used for various secondary purposes, including video labeling, database inclusion, or training AI models. Consequently, adherence to ethical frameworks, such as those established by the American Psychological Association [51], is essential. Among the studies reviewed, only Rodriguez et al [40] explicitly reported compliance with the Declaration of Helsinki, received ethics committee approval, ensured data protection, and obtained explicit consent for video and photographic recordings. In contrast, several studies [42-44,46] merely acknowledged consent for video recordings, and others did not mention compliance with ethical guidelines at all, raising serious concerns regarding the ethical implications of affective computing applications. Clinical data privacy, especially in pediatric populations, must be a priority. Ensuring data confidentiality in storage, transmission, and usage is essential.

Regarding emotion theory, there is a lack of standardization, which makes direct comparisons between systems difficult; most studies did not justify their choice of emotion model. Among those that did [39-41], the dimensional approach to emotion was the most frequently adopted. This approach appears sufficient for many serious game applications, as determining whether a patient’s affective state is generally positive or negative may be adequate for making gameplay adaptations such as altering difficulty or modifying environmental elements.

Another issue identified is the classification of pain as an emotion, which was reported in 50% (4/8) of the studies [42-46]. However, existing literature [52,53] suggests that emotions like depression, anxiety, and anger often emerge as psychological responses to pain rather than being emotions themselves. Therefore, classifying pain as an emotion may be conceptually inaccurate, highlighting the need for a more nuanced and consistent emotional taxonomy in future research.

However, detecting pain and fatigue (physical and psychological) is essential in rehabilitation, as these are major barriers to motor recovery. Pain may cause anxiety and fear of movement [43,46]. Nonintrusive pain detection allows systems to adjust challenges in real time, prevent secondary injuries, and maintain patient safety, which can improve engagement.

By detecting patient fatigue, the system can recommend breaks, adjust exercises, or lower the difficulty to maintain high motor quality and enhance the rehabilitation process.

Clinical applicability is a further consideration when translating research into practice. When selecting data acquisition devices, factors such as cost, precision, portability, patient comfort, and mobility should be considered. Some devices, such as standard webcams [42-47], are easily accessible and cost-effective, while others, like the Emotiv Insight [40,41], require a more substantial investment.

Patient characteristics should also inform the choice of emotion recognition techniques. For instance, in pediatric rehabilitation, noninvasive methods should be prioritized to accommodate children’s natural restlessness and their need for freedom of movement. In studies using facial expression analysis, factors such as ambient lighting, camera specifications, and the distance between the patient and the device can influence outcomes. However, these conditions were not reported in the included studies, and such factors must be considered when selecting or developing software libraries or AI algorithms.

An emerging trend identified is the integration of machine learning techniques. Findings suggest that combining multiple machine learning methods or modifying existing classifiers, such as the one proposed by Rivas et al [45], can enhance emotion recognition accuracy in virtual rehabilitation settings. These findings point to promising future directions for the integration of advanced AI methods in affective computing research.

Due to the variability in study designs and small sample sizes, the findings of this scoping review must be interpreted with caution. While affective computing appears to be a feasible strategy for enabling emotion-sensitive adaptations in serious games, the evidence for its clinical efficacy remains inconclusive. Accordingly, future research should adopt more rigorous methodologies, including larger patient samples, standardized protocols, and validated outcome measures, such as the Fugl-Meyer Assessment or Berg Balance Scale, to better assess the therapeutic impact.

Zhao et al [49] suggest that using multiple affective state recognition techniques can improve classification accuracy and the effectiveness of affective adaptations. However, only 4 of the studies [42-45] used more than 1 recognition method, with gesture and facial expression analysis being the most common combination. It is therefore recommended that future studies use at least 2 complementary techniques, selected based on their noninvasiveness and their ability to preserve patient mobility during rehabilitation.

User behavior analysis was the least explored approach, despite its potential for nonintrusive emotion detection. In this review, only Tironi et al [39] incorporated this technique, and its effectiveness remains to be validated. Future research should consider the application and validation of this approach in clinical rehabilitation settings.

The field remains underdeveloped, with most studies focused on technical validation or proof-of-concept, rather than rigorous experimental designs. The lack of randomized controlled trials is a significant limitation, preventing affective computing from being established as an evidence-based intervention. As a result, these technologies are still considered experimental. Future research should use multicenter designs with statistically significant samples and control groups to validate the long-term clinical value of affective computing.

Analysis of the 9 studies highlights a major limitation in translating affective state recognition into functional application: adaptive approaches are infrequently implemented, with only 2 out of 9 studies [39,41] using a functional closed-loop system that detects emotion and adapts therapy accordingly. Three primary barriers account for this limitation: (1) clinical safety, as incorrect adaptations based on false positives may delay patient progress or compromise treatment effectiveness, for example, if physical effort is misinterpreted as pain and exercise intensity is reduced; (2) technical complexity, since real-time integration of patient signal processing with game engine logic requires considerable coordination among components such as game mechanics and hardware; and (3) the absence of theoretical intervention models, as current psychological models define affective states but do not provide protocols for automatic responses in affect-based rehabilitation systems [13]. As a result, researchers frequently use open-loop systems, in which software monitors the patient, but a human expert, such as the therapist, ultimately adjusts the therapy.

The application of the QUADAS-2 quality assessment tool revealed a high risk of bias in study participant selection, as most studies have small sample sizes. A high risk of bias was also found in the reference standard, since emotions were labeled through expert observation of video recordings. Subjectivity raises concerns that the emotion-detection algorithm, or index test, may simply replicate the biases of human observers. A moderate risk was observed regarding flow and timing, as several studies did not specify the testing interval. Considerable concerns also exist regarding the applicability of patient selection, since studies are often conducted in controlled research laboratories that may not accurately reflect real clinical practice, given patient variability as well as environmental factors such as movement and external noise.

The combination of serious games with affective computing facilitates the identification of states that may hinder rehabilitation, such as pain, fatigue, or anxiety. Failure to deal with these barriers can result in treatment abandonment or injuries due to overexertion in the short or medium term. Nonintrusive detection of pain or fatigue has direct implications for patient safety. In conventional clinical practice, therapists rely on patient self-reports, which may be biased by a desire to please health care professionals, communication difficulties, or embarrassment. Automated detection systems can support health care professionals in making more informed rehabilitation decisions. Affective serious games may serve as objective safety monitors, enabling the system to reduce challenge intensity before patients reach risk thresholds. Additionally, the inclusion of empathetic virtual agents offers a clinical strategy to address loneliness or depression associated with disability. Furthermore, integrating affective computing improves the feasibility of home-based therapy systems by delivering health care providers and family members with alerts regarding patient progress and emotional health.

This scoping review presents several limitations. Although systematic search strategies were used to identify relevant studies across major electronic databases using carefully selected keywords, terminological heterogeneity within the literature may have resulted in the inadvertent omission of pertinent studies.

A major limitation of this review is the small number of included studies (n=9), which limits the ability to draw robust generalizations about the implementation of affective computing in serious games for physical rehabilitation. As a result, the identified trends should be viewed as preliminary, indicating that the integration of clinical rehabilitation and affective computing is still in an early stage of development.

The exclusion of gray literature further limits the scope of this review. By focusing exclusively on peer-reviewed studies, the review ensures a consistent standard of quality, coherence, and reliability. In contrast, gray literature often lacks external validation and standardized reporting structures, which increases the risk of methodological errors and interferes with the assessment of bias, sample representativeness, and the robustness of conclusions.

Another limitation is the absence of searches in languages other than English. All included studies are published in English in high-impact journals or specialized conferences, such as IEEE. This approach may exclude relevant research in other languages and obscure relevant regional developments.

Finally, the findings of this review should be interpreted in light of the methodological limitations of the included studies. First, most studies use small sample sizes, typically 1 to 11 participants. Second, there is also significant selection bias, as many studies rely on samples from specific institutions and often do not clearly describe the patient selection process. Third, most studies focus only on inferring and classifying affective states without implementing adaptive mechanisms to complete the functional loop.

This scoping review presents an updated overview of the state of research on the application of affective computing in serious games for physical rehabilitation. The studies analyzed identify various emotional states in patients undergoing rehabilitation through serious games. These papers use machine learning algorithms or commercial software development kits for emotion detection through at least 1 modality, such as facial expressions, EEG, gesture recognition, or user behavior.

The study shows that applying affective computing techniques to serious games for physical rehabilitation offers a promising complement to traditional rehabilitation approaches for various types of disabilities, incorporating patients’ affective states into the design and adaptation of these games. While affective classification methods have advanced, closing the affective loop through automated interventions remains rare in serious game-based rehabilitation systems. Moreover, the limited availability of primary sources weakens the conclusions regarding adherence and functional performance. Consequently, there is a strong need for multicenter studies with rigorous methodologies to develop systems with a greater impact on rehabilitation.

Therefore, the current evidence base for affective computing in rehabilitation through serious games is technically advanced, yet methodologically fragile. Existing studies remain in a model exploration phase, demonstrating the feasibility of emotion detection but lacking reliability at scale due to selection biases and uncertainties regarding real-life applicability.

Future research should consider the following key aspects: (1) methodological design, including a clear description of the theoretical approaches to emotion, patient selection criteria and characteristics, and a detailed description of the index tests and reference tests used; (2) the adaptation of serious game elements based on the patient’s affective state; (3) use of validated motor function assessment scales to evaluate improvements in the rehabilitation treatment; and (4) promotion of multidisciplinary collaboration among experts in affective computing, physical therapists, and mental health professionals such as psychologists or psychiatrists.

We hope this study serves as a foundation for future investigations seeking to advance physical rehabilitation by integrating affective computing technologies.