The development process of the U-ART-Q involved several distinct phases, including (1) literature review, (2) domain definition and item generation, (3) expert consultation, (4) linguistic validation, and (5) content validation. The detailed development process of this questionnaire is presented in Figure 1.

The first step in developing the U-ART-Q was to conduct a literature review to identify existing questionnaires assessing user satisfaction with assistive, rehabilitation, and training technologies. The search was conducted in the PubMed/MEDLINE, Scopus, and Web of Science databases on September 10, 2025. Database-specific search strategies were developed using combinations of terms related to satisfaction, user experience, questionnaires, development, rehabilitation, assistive technology, robotic rehabilitation, exoskeletons, prostheses, orthoses, and training technologies. The complete search strategies for each database are presented in Multimedia Appendix 1.

All retrieved records were exported to a Mendeley Reference Manager (Elsevier), and duplicate records were removed. Titles and abstracts were independently screened according to the predefined eligibility criteria. Potentially relevant studies underwent full-text review to determine final eligibility. Reasons for exclusion during the full-text assessment were documented in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‑Analyses) recommendations [24].

Studies were considered eligible if they reported the development, validation, adaptation, or application of questionnaires intended to evaluate user satisfaction with relevant technologies. Furthermore, studies were excluded if they focused on technologies outside the scope of rehabilitation, such as industrial exoskeletons, surgical robotics, dental technologies, and other unrelated medical devices. As our search was based only on hardware solutions, publications evaluating software or mobile apps without other hardware components were also excluded. Similarly, user interfaces without accompanying hardware components and virtual reality systems were excluded. Additionally, studies that evaluated other constructs without measuring satisfaction (eg, quality of life, functional performance, technology acceptance without satisfaction outcomes, or clinical effectiveness alone) were excluded. Conference abstracts without full text available, editorials, letters, study protocols, dissertations, non-English publications, and studies with insufficient methodological information were also excluded.

Following full-text screening, studies were excluded for the following reasons: published study not in English language (n=9); technology outside the scope of this review (n=16); no satisfaction-related outcome measures reported (n=64); and study that does not use a questionnaire (n=3).

The complete study selection process is illustrated in the PRISMA flow diagram. All included studies are presented in Multimedia Appendix 2 (Microsoft Excel worksheet titled Studies for Extraction).

After searching the literature, we noted the existence of 117 questionnaires used to measure user experience, usability, and satisfaction with various technologies and systems. The most frequently used questionnaires were general questionnaires, such as the Quebec User Evaluation of Satisfaction With Assistive Technology (QUEST 2.0), USE, and SUS, as well as questionnaires specific to certain technologies, such as the Client Satisfaction With Device Module of the Orthotics and Prosthetics Users’ Survey (CSD-OPUS), the Questionnaire for Usability Evaluation of orthopaedic shoes (QUE), the Trinity Amputation and Prosthesis Experience Scale (TAPES), the prosthesis-satisfaction subscale, and others [23,25-31]. The list of all identified questionnaires used in the evaluation of satisfaction with technologies is available in Multimedia Appendix 2 (Excel worksheet titled Identified Questionnaires).

Although a considerable number of questionnaires were identified (Multimedia Appendix 2), several limitations were noted. The identified questionnaires were either designed for a specific type of technology or failed to cover all important aspects necessary to provide detailed information on user satisfaction with assistive, rehabilitation, and training technologies. Forty-six out of 117 identified questionnaires were completely custom questionnaires, intended for the specific device or population in the study, limiting their broader applicability. Eighty-nine out of 117 questionnaires were developed specifically for the needs of the study, including custom questionnaires, modified instruments for usability testing of specific devices, newly developed questionnaires, and translations of existing questionnaires. Only 23 out of 117 questionnaires were reported in more than a single publication. Among the identified questionnaires, 58 primarily evaluated usability, user experience, technology acceptance, or overall system performance rather than user satisfaction, with satisfaction represented by only a small number of items instead of a comprehensive multidimensional construct. Additionally, none of the questionnaires appeared to cover all identified satisfaction domains. Item-level analysis demonstrated that domains with similar labels (eg, design) frequently assessed different underlying constructs and therefore did not capture equivalent aspects of user satisfaction. The identified questionnaires were also developed and used for different technology categories: 84 for assistive, 23 for rehabilitation, 9 for assistive and rehabilitation, 1 for nontechnology-specific questionnaire, and none for training technologies (Multimedia Appendix 2). Although several questionnaires have subsequently been applied beyond their original context, they were not specifically developed for these broader applications. In addition, widely used generic instruments, such as the SUS, were designed as technology-independent usability measures rather than for evaluating hardware-based technologies that directly interact with the human body and therefore do not capture several device-specific aspects identified as important in this review. Overall, while the identified questionnaires provide valuable information on specific aspects of user evaluation, no single instrument was found to simultaneously (1) assess user satisfaction as its primary outcome; (2) comprehensively cover all identified satisfaction domains at the item level; and (3) be specifically developed for application across assistive, rehabilitation, and training technologies.

Therefore, we aimed to develop a questionnaire to comprehensively assess user satisfaction across different types of assistive, rehabilitation, and training technologies. The questionnaire was specifically designed to capture the perspectives of direct end users, including both individuals using or training with the technology and professionals such as physiotherapists, kinesiologists, personal trainers, and other health care providers involved in its operation and delivery.

The second phase of the development process focused on identifying key domains and generating items to evaluate user experience and satisfaction with assistive, rehabilitation, and training technologies. The initial version of the questionnaire consisted of 72 items divided into 7 thematic domains. Item generation was guided by the preceding literature review and further informed by the research team’s prior experience in developing and evaluating similar technologies. The list of all identified items and related domains during the literature review is available in Multimedia Appendix 2 (Excel worksheet titled Items and Domains). Items were generated with the intention that each item should apply to at least one type of existing assistive, rehabilitation, or training technology. However, given that such technologies can differ significantly in their fundamental characteristics, not all items are relevant for every device. To address this, a set of initial screening questions was developed to be completed by the researcher or developer prior to administering the questionnaire. The responses to these screening questions are used to generate a tailored set of questionnaire items, selected according to the characteristics of the technology and the role of the end user (eg, patient, client, therapist, or trainer). Furthermore, the researcher or developer may manually adjust the recommended item set by adding or removing questions to ensure optimal relevance for the specific technology.

After the domains were identified and the items generated, a round of expert revision was conducted. Six experts with extensive experience in the design and evaluation of assistive, rehabilitation, and training technologies and 8 master’s students in physiotherapy were recruited to evaluate each item independently as either “relevant” or “not relevant.” Additional information about the experts included in the expert consultation is presented in Table 1. For any item marked as not relevant, the recruited experts were asked to provide a brief explanation for their decision. Based on the results obtained, the questionnaire was revised, and items marked as not relevant were either removed or modified. Following the revision process, the questionnaire was evaluated by 2 English-language experts to ensure linguistic accuracy, clarity, and overall readability.

The final part of the development process was the evaluation of content validity. Content validity relates to the degree to which a sample of items constitutes an adequate operational definition of a concept [32]. To assess item content validity, another 8 experts in physiotherapy, kinesiology, and ergonomics were recruited. These experts were independent of the 6 experts involved in the previous stage of questionnaire development and had at least 5 years of experience in the development or use of assistive, rehabilitation, or training technologies. The participating experts were informed of the purpose of the questionnaire and its intended use to evaluate user experience and satisfaction with the assistive, rehabilitation, and training technologies. Each expert independently evaluated all items of the questionnaire on a 4-point Likert-type scale (1=“the item is not relevant to the measured domain,” 2=“the item is somewhat relevant to the measured domain,” 3=“the item is quite relevant to the measured domain,” and 4=“the item is highly relevant to the measured domain”). The content validity ratio (CVR) was calculated for each item alongside the content validity index for each item (I-CVI) using equations 1 and 2, where n₄ represents the number of experts grading an item as highly relevant (grade 4), n_3,4 represents the number of experts grading an item as quite or highly relevant (grade 3 or 4), and N represents the total number of experts. Additionally, the average content validity index (Ave-CVI) was calculated as the average of the I-CVIs, representing the content validity of the overall instrument [33].

Items with a CVR below 0.75 and an I-CVI below 0.78 were reviewed and either revised or removed in accordance with the content validity criteria described by Lawshe [34] and Lynn [35].

All participants received oral and written information about the study, were informed that participation was voluntary, and provided informed consent prior to participation. Participant confidentiality was ensured throughout the study. The study was approved by the University of Primorska Commission for Ethics in Research Involving Human Subjects (approval number: 4264-19-6/23) and was conducted in accordance with the principles of the Declaration of Helsinki.

HCD is a vital component of the product development process, ensuring that technologies are designed around the actual needs, abilities, and expectations of end users. In assistive, rehabilitation, and training technologies, HCD promotes usability, accessibility, and satisfaction by actively involving end users throughout the design, testing, and implementation phases. A key component of HCD is usability testing, which provides critical insight into how users interact with a product and identifies barriers to effective use. Failing to include usability testing at different stages of development can result in technically advanced products that do not meet user expectations, leading to poor acceptance, limited adoption, and ultimately unsuccessful implementation.

Although several relevant questionnaires were identified in the literature, none was considered fully suited to the specific purpose and scope of this study. Therefore, this study introduces the newly developed U-ART-Q and evaluates its content validity. It serves as a complementary tool for evaluating user satisfaction with assistive, rehabilitation, and training technologies. The items and domains for the initial version of the questionnaire were identified through a comprehensive literature review. The preliminary set of items and domains underwent expert validation, which resulted in the removal or merging of 20 items to reduce redundancy and improve clarity. To further enhance linguistic accuracy, clarity, and overall readability, the revised questionnaire was reviewed and refined by 2 English-language experts. Finally, 8 additional experts with backgrounds in physiotherapy, kinesiology, and ergonomics participated in the content validation process. Out of 52 items, 44 met the predefined CVR and I-CVI thresholds and therefore met the predefined criteria for content validity. The remaining 8 items with CVR and I-CVI values below the recommended thresholds were reviewed further, resulting in the removal of 2 items from the item bank. Finally, the remaining 50-item questionnaire achieved a high Ave-CVI of 0.915, demonstrating strong evidence of content validity based on expert evaluation.

The proposed U-ART-Q represents a newly developed questionnaire with evidence of content validity for assessing user satisfaction within the domains of assistive, rehabilitation, and training technologies. To ensure its relevance across a broad range of technologies, the researcher or developer administering the questionnaire first completes a set of initial screening questions that guide item selection. This approach allows the instrument to be tailored to the specific characteristics of the technology being evaluated, making it both concise and practical for use across diverse assistive, rehabilitation, and training technologies.

The proposed questionnaire offers several important contributions to both research and practice. For researchers and clinicians, it is intended to serve as a structured and efficient evaluation tool for existing technologies, allowing user satisfaction to be compared across different devices, applications, and user groups. Its use can contribute to a more comprehensive understanding of user experience, leading to more informed decisions regarding device selection, clinical implementation, and future product improvement. For developers, the questionnaire provides a structured and standardized tool for collecting user feedback throughout the design process. It helps identify usability issues early and supports evidence-based design decisions. By systematically capturing satisfaction across multiple domains, the instrument can help optimize the development process by reducing redesign costs and improving the functionality of technologies under development. As user satisfaction and experience are closely linked to successful technology implementation, the results may also help clinical managers and other stakeholders compare available market options and support the decision-making, selection, and purchasing of technologies.

Future work will focus on further validation and practical application of the questionnaire. Pilot testing will be conducted to evaluate reliability, internal consistency, and test-retest reliability, and validated translations of the questionnaire into Serbian and Slovenian will be completed. Although a macro-enabled Excel tool (.xlsm) has been developed to support automated item selection, which is provided in Multimedia Appendix 5, the development of a mobile app or a web-based version of the questionnaire is planned to facilitate simple, time-efficient, and cost-effective administration in clinical and research settings.

The development of the U-ART-Q has several limitations. First, the existing literature did not provide a comprehensive conceptual framework that specifically describes the multidimensional structure of user satisfaction with assistive, rehabilitation, and training technologies. Consequently, the domains and items of the U-ART-Q were informed by findings from the literature, existing questionnaires, expert consultations, and the research team’s experience in developing similar technologies. Although the experts involved had substantial experience in relevant fields, expert-based evaluation is inherently subject to some degree of subjectivity. Different expert panels could have reached different conclusions regarding the relevance, wording, or importance of individual items.

Additionally, it is important to point out that the U-ART-Q is a newly developed questionnaire with initial evidence of content validity. No actual end users were included in the development or content-validation process, which represents an important limitation, particularly because the questionnaire is intended to assess user satisfaction. As such, the U-ART-Q should undergo further evaluation before it can be considered a fully validated and standardized instrument for assessing user satisfaction with assistive, rehabilitation, and training technologies. In line with recommended questionnaire-development procedures, important next steps include face validity assessment with intended users, pilot testing, evaluation of construct validity (through exploratory and confirmatory factor analysis), reliability testing, and establishment of a clear scoring system and interpretation of the results [46].

In conclusion, the U-ART-Q is a newly developed questionnaire with evidence of content validity. Its main objective is to support researchers and developers applying HCD principles in the evaluation and development of assistive, rehabilitation, and training technologies. The questionnaire is intended to complement existing methods of assessing user experience, including interviews, focus groups, and observational approaches. By integrating this quantitative measure with qualitative methods, the U-ART-Q can contribute to more informed, iterative, and end user–driven design processes. Future work will focus on further psychometric evaluation needed to determine whether the U-ART-Q can be established as a validated and standardized questionnaire for assessing user satisfaction with assistive, rehabilitation, and training technologies.