Review
Abstract
Background: The field of rehabilitation has seen a recent rise in technologies to support shared decision-making (SDM). Usability testing during the design process of SDM technologies is needed to optimize adoption and realize potential benefits. There is variability in how usability is defined and measured. Given the complexity of usability, a thorough examination of the methodologies used to measure usability to develop the SDM technologies used in rehabilitation care is needed.
Objective: This scoping review aims to answer the following research questions: which methods and measures have been used to produce knowledge about the usability of rehabilitation technologies aimed at supporting SDM at the different phases of development and implementation? Which parameters of usability have been measured and reported?
Methods: This review followed the Arksey and O’Malley framework. An electronic search was performed in the Ovid MEDLINE, Embase, CINAHL, and PsycINFO databases from January 2005 up to November 2020. In total, 2 independent reviewers screened all retrieved titles, abstracts, and full texts according to the inclusion criteria and extracted the data. The International Organization for Standardization framework was used to define the scope of usability (effectiveness, efficiency, and satisfaction). The characteristics of the studies were outlined in a descriptive summary. Findings were categorized based on usability parameters, technology interventions, and measures of usability.
Results: A total of 38 articles were included. The most common SDM technologies were web-based aids (15/33, 46%). The usability of SDM technologies was assessed during development, preimplementation, or implementation, using 14 different methods. The most frequent methods were questionnaires (24/38, 63%) and semistructured interviews (16/38, 42%). Satisfaction (27/38, 71%) was the most common usability parameter mapped to types of SDM technologies and usability evaluation methods. User-centered design (9/15, 60%) was the most frequently used technology design framework.
Conclusions: The results from this scoping review highlight the importance and the complexity of usability evaluation. Although various methods and measures were shown to be used to evaluate the usability of technologies to support SDM in rehabilitation, very few evaluations used in the included studies were found to adequately span the selected usability domains. This review identified gaps in usability evaluation, as most studies (24/38, 63%) relied solely on questionnaires rather than multiple methods, and most questionnaires simply focused on the usability parameter of satisfaction. The consideration of end users (such as patients and clinicians) is of particular importance for the development of technologies to support SDM, as the process of SDM itself aims to improve patient-centered care and integrate both patient and clinician voices into their rehabilitation care.
doi:10.2196/41359
Keywords
Introduction
Background
Shared decision-making (SDM), the collaborative process involving active participation from both patients and providers in health care treatment decisions, reflects an important paradigm shift in medicine toward patient-centered care [
, ]. SDM facilitates information exchange and discussion of treatment options that involve the best scientific evidence and consider patient preferences [ , ]. The readiness for using SDM may be enhanced through its accessibility to individuals with limited health literacy or those with disabilities [ ]. In the context of rehabilitation, SDM typically occurs during goal setting by selecting and agreeing upon behavioral objectives that patients, caregivers, and the rehabilitation team work together to achieve [ ]. The development of mutual trust, 2-way communication, and sharing of power are conditions that influence patients’ capacity and confidence to participate in SDM in musculoskeletal physiotherapy [ ] and in the treatment of depression [ ]. As a result, SDM assists patients in making individualized care decisions, and health care providers can feel confident in the presented and prescribed options [ , ]. SDM is important to increase satisfaction with care among both patients and providers, may improve individuals’ quality of life and clinical outcomes, and fosters a better patient-provider relationship [ ]. Furthermore, SDM encourages patient participation in their rehabilitation, supporting self-efficacy, empowerment, and ownership over the decisions [ ].Despite the listed benefits, it has been difficult to implement SDM in clinical practice because of barriers such as time constraints, accessibility to information and effective SDM tools, and limited technical and organizational resources [
]. It has been reported that only 10% of face-to-face clinical consultations involve SDM [ , ]. Advances in digital health technologies (eHealth) have resulted in tools that can bridge this SDM gap by allowing increased access to shared information and support for patient-provider communication [ ]. Accessible, cost-effective, web-based decision-making is supported by use across various platforms such as the internet, tablets, or smartphone apps [ , ]. Such SDM technologies include patient decision aids that clarify options and values for personalized decision support, leading to reduced decisional conflict and increased participation in treatment choices that are consistent with the patient’s values [ ]. Patient portals reflect another technology that can support SDM, providing patients with secure access to their health information profile and communication with their care provider [ - ].Although studies have been conducted to introduce and investigate the acceptance of rehabilitation technologies, research into the usability of technology systems is limited [
, ]. A technology system in rehabilitation is defined as an environmental factor that incorporates aspects of the physical and social environments that may affect communicative participation [ ]. Technology systems need to be evaluated in terms of their usability to maximize their acceptance and benefits. The International Organization for Standardization (ISO) 9241 defined usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [ ]. Evaluation of usability is key to guiding the development of efficient and effective technologies that end users will readily adopt by providing information about how a user uses the technology system and the challenges they find while interacting with a system’s interface [ ]. Different usability models have been proposed for evaluating software usability. Gupta et al [ ] proposed a comprehensive hierarchal usability model with a detailed taxonomy, including 7 usability parameters: efficiency, effectiveness, satisfaction, memorability, security, universality, and productivity. Evaluating these usability parameters throughout the design process can allow for continuous improvement of ease of use and can predict the user’s acceptance or rejection of the product [ ]. Therefore, including input from individuals who will use the technology (in the case of SDM technologies, clinicians, patients, and caregivers) through usability testing is a necessary component in designing relevant, understandable, and usable technologies.Objectives
The field of rehabilitation science is defined as a multidimensional person-centered process targeting body functions, activities and participation, and the interaction with the environment aiming at optimizing functioning among persons with health conditions experiencing disability [
]. It has seen a recent rise in the development and implementation of technologies aimed at supporting SDM between clinicians, patients, and their caregivers [ ]. However, it is unclear how user input or usability testing is integrated into the design process of these rehabilitation health technologies, including how usability is conceptualized, what measures are used, and at what stage of design usability is evaluated. To date, few studies, and no systematic or scoping reviews that we are aware of, have addressed how usability is measured among rehabilitation technologies supporting SDM. Given the complexity of usability, a thorough examination of the methodologies used to measure usability in this context is required to comprehensively map what has been done and inform future research efforts. A greater understanding of how the parameters of usability are measured will guide future usability testing to inform further development of SDM technologies designed to enhance patient-centered care in rehabilitation. Therefore, this scoping review was conducted to provide knowledge about the methods and measures used to determine the usability of rehabilitation technologies aimed at supporting SDM at different phases of technology development and implementation.Methods
This scoping review followed the methodology described by Arksey and O’Malley [
] and was reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines [ ] ( ).Identifying the Research Questions
This scoping review aimed to answer the following research questions: (1) which methods and measures have been used to produce knowledge about the usability of rehabilitation technologies aimed at supporting SDM at the different phases of development and implementation? (2) Which parameters of usability have been measured and reported in studies focusing on rehabilitation technologies aimed at supporting SDM?
Eligibility Criteria
The eligibility criteria for this scoping review are outlined in
.Inclusion criteria
- Articles published in peer-reviewed journals, including quantitative (randomized controlled trials or nonrandomized controlled trials), qualitative, and mixed methods studies
- Articles including different groups of people, such as health care practitioners and individuals seeking rehabilitation services (ie, patients and their caregivers) or case managers
- Articles that focused on the usability of technology in making decisions
- Articles reporting a clear objective to evaluate the usability of shared decision-making (SDM) technologies in rehabilitation
Exclusion criteria
- Nonstructured reviews, protocols, descriptive reviews, nonhuman studies, and gray literature
- Articles not focusing on or measuring the usability of technologies in SDM and groups not related to the health care sector (ie, students)
Search Strategy
The search strategy was developed in collaboration with a health science librarian. As health system issues often change with models of care delivery, the economic climate, and the environment [
], we decided to narrow the scope of the search (2005 to 2020). The following electronic databases were searched in both English and French: Ovid MEDLINE, Embase, CINAHL, and PsycINFO. A combination of Medical Subject Heading terms, subject headings, and keywords was used and covered five concepts: (1) usability OR user* friendl* OR eas* to use OR useful* OR user* perspective* OR patient* perspective* OR client* perspective* OR user* experience* AND (2) rehabilitation OR telerehabilitation OR tele rehabilitation OR disabled OR disabilit* OR physical limitation* OR mental limitation* OR psycho* limitation* OR adaptation* OR mobility OR occupational therap* OR physiotherap* OR physical therap* OR speech languag* pathol* OR speech therap* OR language therap* OR communication disorder* AND (3) think* aloud OR focus group* OR interview* OR Wizard* OR Empathy map* OR Persona* OR Questionnaire* OR instrument* OR scale* OR tool OR tools OR measurement* OR survey* OR drama OR deliberation* OR evaluation* OR assessment* OR video confrontation* OR photo voice* AND (4) technolog* OR gerontotechnolog* OR smart* OR intelligen* OR ambient assisted living OR virtual reality OR virtual rehabilitation OR telemonitoring OR telehealth OR telemedicine OR telerehabilitation OR ehealth OR tele monitoring OR tele health OR tele medicine OR tele rehabilitation OR e health or sensor* OR biosensor* OR mobile app* OR product* OR internet OR web OR computer* OR software* OR device* OR self-help OR wheelchair* OR wheelchair* OR communication aid* AND (5) shared decision making OR Decision-Making OR patient-provider communication OR decision aid OR decision support. This was followed by hand searches of the reference lists of the included studies (the search strategy for Ovid MEDLINE is presented in ).Study Selection
All identified studies were uploaded into EndNote X9.1 (Clarivate Analytics), and duplicates were removed. In total, 2 independent reviewers conducted the selection of abstracts starting with a pilot phase involving the examination of the first 10 titles and abstracts to screen and decide on retention of the abstract based on the inclusion criteria. Interrater agreements were assessed using the κ statistic [
]. Interrater agreement of <75% resulted in a clarification of the eligibility criteria and a revision if needed. The process was repeated twice between the reviewers until an agreement of 75% was reached, which is evidence of excellent agreement [ ]. Finally, all eligible studies and those classified as unclear (ie, requiring further information to make a final decision regarding their retention) were independently reviewed as full-text articles. Disagreements at this stage were resolved through consensus. The PRISMA-ScR flow diagram [ ] was used to guide the selection process.Data Extraction
In total, 2 reviewers independently extracted data from the included articles to avoid missing relevant information. The data extracted included information corresponding to study design, rehabilitation technology intervention used (ie, setting, content, and detail of the type of user interface), population studied (participant demographics and target conditions), characteristics of the measures, and the development stage.
Data Synthesis
Descriptive statistics were used to describe the characteristics of the included studies, study design, characteristics of the study population, and geographical location. Findings were categorized based on study designs, parameters of usability, types of technologies, stage of development of the technology, and usability evaluation methodologies.
Types of SDM technologies and usability evaluations were mapped to parameters of usability based on a comprehensive hierarchal usability model presented by Gupta et al [
]. The usability parameters include efficiency, defined as “enables user to produce desired results with respect to investment of resources”; effectiveness, defined as “a measure of software product with which user can accomplish specified tasks and desired results with completeness and certainty”; satisfaction, defined as “a measure of responses, feelings of user when users are using the software i.e., freedom from discomfort, likeability”; memorability, defined as “the property of software product that enables the user to remember the elements and the functionality of the system product”; security, defined as “the degree to which risks and damages to people or other resources i.e. hardware and software can be avoided”; universality, defined as “the accommodation of different cultural backgrounds of diverse users with software product and practical utility of software product”; and productivity, defined as “the amount of useful output with the software product” [ ] ( ).The usability evaluation methodologies were mapped based on the framework by Jacobsen [
]. The categories of the usability evaluation methods included (1) empirical methods, based on users’ experience with the technology in a systematic way; (2) inspection methods, conducted by experts who examined usability-related aspects of a user interface without involving any users; and (3) inquiry methods, based on the information about users’ needs, likes, and understanding of the technology through interviews or focus groups, observation, and verbal or written questions [ ].Efficiency
- Resources
- Time
- User effort
- Economic
- Cost
Effectiveness
- Task accomplishment
- Operability
- Extensibility
- Reusability
- Scalability
Satisfaction
- Likability
- Convenience
- Esthetics
Memorability
- Learnability
- Memorability of structure
- Comprehensibility
- Consistency of structure
Security
- Safety
- Error tolerance
Universality
- Approachability
- Utility
- Faithfulness
- Cultural universality
Productivity
- Useful user task output
Consulting and Translating Knowledge
This scoping review is part of an initiative (Réseau provincial de recherche en adaptation-réadaptation–RS6 Technologies de readaptation [Quebec Rehabilitation Research Network]; [
]) to create an interactive directory of methodological tools for measures of the usability of rehabilitation technologies. Stakeholder consultations with members of the Réseau provincial de recherche en adaptation-réadaptation–RS6 group were held at the beginning of the process (requesting feedback to refine the research question for data extraction and synthesis), during the study (validating the data extraction and deciding on the best way to align the information with stakeholders’ needs), and when the final results were available (knowledge mobilization).Results
Study Selection
A total of 430 studies were identified from electronic searches, and a total of 19 were identified through hand sorting reference lists. We excluded 57.2% (257/449) of the studies at the title and abstract stage, resulting in 192 full-text articles. Of these 192 studies, 154 (80.2%) were excluded at the full-text stage, resulting in 38 (19.8%) studies. The search strategy was updated in November 2020 and followed the PRISMA-ScR flowchart of the selection process. Reasons for exclusion of studies are provided in
. Interrater agreement reached ≥75%, which is evidence of excellent agreement. Disagreements were resolved through consensus.Characteristics of the Included Studies
The characteristics of the included studies are presented in
[ - ]. Overall, the 38 included studies were published between 2008 and 2020 as peer-reviewed studies. Studies were published in the United States (17/38, 43%), Europe (14/38, 37%), Canada (5/38, 13%), and Asia (2/38, 5%). The study designs of the included studies were mixed methods (20/38, 53%), qualitative (12/38, 31%), and quantitative (6/38, 16%).Characteristics of the Included Participants
presents the characteristics of the included participants. The number of participants across all the included studies was 2138, with age ranging between 18 and 86 years. Participants of usability evaluations included patients (38/38, 100%); clinicians (32/38, 84%); caregivers or family (12/38, 32%); and others (6/38, 16%), including case managers, drug advisory committees, computer scientists, behavioral scientists, communication scientists, clinical administrators, service providers, and social service providers. The target end users of the developed SDM technologies were mainly patients and clinicians (24/38, 63%). The recruitment methods and settings varied across the included studies, including hospitals (24/38, 63%), the community (10/38, 26%), and universities (4/38, 11%).
Usability Definitions and Parameters
presents usability definitions and parameters provided by the authors across the included studies. Notably, only 50% (19/38) of the included studies provided an a priori definition of usability or listed parameters of usability. Usability parameters were categorized as effectiveness (9/38, 23%), efficiency (8/38, 21%), memorability (11/38, 29%), satisfaction (14/38, 37%), security (5/38, 13%), universality (4/38, 10%), and productivity (10/38, 26%) based on Gupta et al [ ].
Study | Definition of usabilitya | Usability parametersa | Gupta et al [ | ] framework
Bauerle Bass et al [ | ], 2018User testing was completed to assess the extent to which the tool was understandable, how easily it could be navigated, and its relevance to patients taking HCVb+methadone. |
|
|
Berry et al [ | ], 2015Usability testing is the evaluation of information systems through testing by representative users, enabling evaluation of social acceptability, practicality, and usability of a technology. |
|
|
Bogza et al [ | ], 2020NRc |
|
|
Chrimes et al [ | ], 2014Refers to commentary on the perceived effectiveness, efficiency, and ease of use, or lack thereof, of the ADAPTd Toolkit. |
|
|
Cox et al [ | ], 2015Usability describes the quality of a user’s experience with software or an IT considering their own needs, values, abilities, and limitations. |
|
|
Cuypers et al [ | ], 2019NR |
|
|
Danial-Saad et al [ | ], 2016Usability is defined by the ISOe 9241 as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” |
|
|
De Vito Dabbs et al [ | ], 2009The measure of the ease with which a system can be learned and used, including its safety, effectiveness, and efficiency. |
|
|
Fleisher et al [ | ], 2008Whether patients found the tools easy to use and navigate, as well as the readability and usefulness of the physician report. Usability protocol based on NCIf guidelines. |
|
|
Fu et al [ | ], 2020Usability is defined by the ISO 9241-11 as the extent to which a product can be used by a specific person in a specific context to achieve realistic goals of effectiveness, efficiency, and satisfaction. |
|
|
Goud et al [ | ], 2008NR |
|
|
Grim et al [ | ], 2017Usability refers to commentary. Understandability and usefulness are 2 major constructs when talking about usability. Understandability refers to the extent to which the descriptive texts and items are comprehensible. Usefulness refers to commentary on the extent to which the features in the decision aid are perceived as supporting decision-making processes on the perceived effectiveness, efficiency, and ease of use, or lack thereof, of the decision aid. |
|
|
Kallen et al [ | ], 2012Usability was considered an incorporation of system effectiveness, efficiency, and user satisfaction. Usability was defined in the context of the assessment and review of tasks assigned to study participants. |
|
|
Li et al [ | ], 2013A usability issue was defined as (1) when a participant was not able to advance to the next step because of the decision aid design or a programming error or (2) when a participant was distracted by a particular design or content of the web tool. |
|
|
Rochette et al [ | ], 2008The term “usability” is defined as the effectiveness, efficiency, and satisfaction with which users can achieve tasks in a particular environment. High usability means that a system is easy to learn and remember, efficient, visually pleasing, and fun to use and enables quick recovery from errors. |
|
|
Span et al [ | ], 2014NR |
|
|
Støme et al [ | ], 2019NR |
|
|
Van Maurik et al [ | ], 2019Clinicians were asked to complete the SUSg after using the tool. |
|
|
Williams et al [ | ], 2016NR |
|
|
Zafeiridi et al [ | ], 2020Usability is measured as the user-friendliness (eg, ease to learn) and perceived usefulness in addressing users’ needs. |
|
|
aAs defined by the authors.
bHCV: hepatitis C virus.
cNR: not reported.
dADAPT: Avoiding Diabetes Through Action Plan Targeting.
eISO: International Organization for Standardization.
fNCI: National Cancer Institute.
gSUS: System Usability Scale.
Technology for SDM
presents the type of SDM technologies that were used across the included studies. Technologies for SDM included clinical decision support systems (9/33, 27%), mobile health apps (9/33, 27%), and web-based aids (15/33, 46%). The SDM context was mainly between clinicians and patients (32/36, 89%). The types of technology for SDM were mapped to usability parameters, including effectiveness (10/38, 26%), efficiency (11/38, 29%), memorability (20/38, 53%), satisfaction (27/38, 71%), security (5/38, 13%), universality (4/38, 10%), and productivity (16/38, 42%) based on Gupta et al [ ]. The most common SDM technologies evaluated for usability were web-based aids. Satisfaction was the most common usability parameter mapped to types of SDM technologies.
Study | Title of developed technology | Technology overview | Stage of development of technology intervention | Framework followed or guidelines by the authors | Description of SDM context or type of decision-making | Usability parameters measured | Gupta et al [ | ] framework
Anderson et al [ | ], 2014STOPa Tool | Web-based user interface for adaptive clinical decision support integrated into electronic health record | Preimplementation | Framework based on usability engineering | SDM between patients and clinicians for self-management and secondary stroke prevention |
|
|
Barrio et al [ | ], 2017SIDEALb | Mobile app | Developmental laboratory | MIc was the main source of guidance throughout the development process. | SDM between patients and clinicians related to self-management of alcohol dependence |
|
|
Bauerle Bass et al [ | ], 2018“Take Charge, Get Cured” | mHealthd decision support tool | Development | Model of illness self-regulation, information-communication theory, and formative evaluation framework | SDM between patients and clinicians related to initiating hepatitis C treatment |
|
|
Berry et al [ | ], 2015P3Pe | Web-based decision aid | Preimplementation | NRf | SDM between patients and clinicians about prostate cancer management options |
|
|
Bogza et al [ | ], 2020Web-based decision aids | —g | Development | User-centered approach; Center for eHealth and Wellbeing Research guidelines | SDM between patients and clinicians |
|
|
Burns and Pickens [ | ], 2017NR | Technology-based CDSSh for app-based assessments | Preimplementation | NR | SDM between providers, client, and family for home evaluation and modifications |
|
|
Canally et al [ | ], 2015NR | GUIsi | Developmental laboratory | NR | Shared decision support system that integrated biophysiological information obtained through multiple nonintrusive monitoring for home care |
|
|
Chrimes et al [ | ], 2014ADAPTj | Clinical decision support tool integrating evidence-based shared goal-setting components into electronic health record | Developmental laboratory | ADAPT framework | SDM between patients and clinicians for behavior changes to manage prediabetes |
|
|
Cox et al [ | ], 2015eCODESk | Web-based decision aid integrated into data entry and management system | Developmental laboratory | NR | SDM between clinicians and surrogate decision makers of patients receiving prolonged mechanical ventilation |
|
|
Cuypers et al [ | ], 2019Web-based decision aid system | — | Development | On the basis of existing evidence-based Canadian decision aid, developed by Feldman-Stewart et al [ | - ]SDM between patients and clinicians | NR | NR |
Danial-Saad et al [ | ], 2016OSCARl | Interactive CDSS | Development laboratory | LUCIDm framework | Server-client system to recommend and select optimal pointing device |
|
|
De Vito Dabbs et al [ | ], 2009Pocket PATHn | IHTo through handheld computer device | Preimplementation | User-centered design | SDM between patients of lung transplant and their transplant team about self-monitoring of critical values |
|
|
Fleisher et al [ | ], 2008CONNECTp | Interactive web-based communication aid | Preimplementation | C-SHIPq model | SDM between patients and clinicians about treatment decisions supported through communication skill development modules |
|
|
Flynn et al [ | ], 2015COMPASSr prototype | User interface with decision analytical model developed on iPad mobile device | Developmental laboratory | Decision analytic model predictions developed from S-TPIs | SDM between clinicians and patients about patient-specific treatment options for acute ischemic stroke and personalized information to patients |
|
|
Fu et al [ | ], 2020Mobile apps | — | Testing | Nielsen heuristics | Unclear |
|
|
Goud et al [ | ], 2008CARDSSt | Guideline-based computerized decision support systems | Implementation | Clinical guidelines | SDM between clinicians and patients for patient-specific care for cardiac rehabilitation and patient management |
|
|
Grim et al [ | ], 2017NR | Interactive web-based software | Preimplementation | The team followed published evidence on the consensus guidelines for development of decision aids and SDM. | SDM between patients and clinicians about care in psychiatric services |
|
|
Holch et al [ | ], 2017e-RAPIDu | Integrated electronic platform for patient self-report | Preimplementation | Translational research model | SDM between patients and clinicians for management of events during cancer treatment |
|
|
Jameie et al [ | ], 2019Cardiac telerehabilitation platform | — | Development | BACPRv | SDM between patients and clinicians | NR | NR |
Jessop et al [ | ], 2020“Take Charge, Get Cured” | mHealth treatment decision support tool embedded in Articulate 360 app | Preimplementation | NR | SDM between patients and physicians about hepatitis C treatment | NR | — |
Kallen et al [ | ], 2012PROw-based Palliative and Hospice Care Management System—prototype | Electronic PRO system | Implementation | User-centered design approach | SDM between patients in palliative care and treating physician or nurse |
|
|
Li et al [ | ], 2013ANSWERx | Web-based decision aid with educational modules | Preimplementation | The International Patient Decision Aid Standards and the Jabaja-Weiss edutainment decision aid model | SDM between patients and clinicians about using methotrexate |
|
|
Murphy et al [ | ], 2020CP-PDAy | Web-based algorithmic intervention | Development | International Patient Decision Aid Standards criteria checklist, SUNDAEz checklist, and the EQUATORaa CONSORTab checklist | SDM between patients and clinicians about postprostatectomy care regarding continence product choice |
|
|
Rochette et al [ | ], 2008StrokEngine-Family | Stroke rehabilitation layperson website | Implementation | NR | SDM between patients and clinicians |
|
|
Setiawan et al [ | ], 2019iMHereac 2.0 | Adaptive mHealth system with mobile app modules (client app, caregiver app, web-based clinician portal, back-end server, and 2-way communication protocol) | Development | User-centered design | Monitoring and support of self-management for people with chronic conditions and disabilities and allowing for personalized and adaptive treatment strategies |
|
|
Schön et al [ | ], 2018Digital interactive decision support tool | — | Development | The decision support tool is based on the theoretical framework of SDM. | SDM between patients and clinicians |
|
|
Snyder et al [ | ], 2009PatientViewpoint prototype | Web-based system to collect PROs linked with electronic medical record | Preimplementation | NR | SDM between patients and clinicians for cancer management |
|
|
Span et al [ | ], 2014DecideGuide | Interactive web tool | Developmental laboratory | NR | SDM in dementia care networks between patients, care managers, and informal caregivers |
|
|
Span et al [ | ], 2018DecideGuide | Interactive web tool | Preimplementation | The 5 phases of the CeHResad road map | SDM made by care network of people with dementia (patients, care managers, and informal caregivers) |
|
|
Støme et al [ | ], 2019Vett interactive mobile app | — | Implementation | NR | Unclear |
|
|
Tony et al [ | ], 2011EVIDEMae decision support framework | MCDAaf and HTAag | Developmental laboratory | EVIDEM framework | SDM between patients and clinicians to appraise health care interventions |
|
|
Toth-Pal et al [ | ], 2008EviBase | CDSS through internet-based application | Implementation | Clinical guidelines (1 Swedish and 2 European) | SDM between clinicians and patients through integration of individual patient data with guidelines for management of chronic heart failure |
|
|
Tsai et al [ | ], 2019MagicPlan | Mobile app with laser distance measurer | Preimplementation | NR | Clinical home evaluations with virtual floor plan for DMEah recommendations |
|
|
Van Maurik et al [ | ], 2019Web-based diagnostic support tool named ADappt | — | Development | NR | SDM between patients and clinicians |
|
|
Welch et al [ | ], 2015MedMinder | Cellular pillbox monitoring device | Implementation | NR | SDM between clinicians and patients related to treatment and adherence support |
|
|
Williams et al [ | ], 2016NR | Clinical decision support on mHealth app | Developmental laboratory | User-centered design approach (user interface and user experience design) | SDM between clinicians and patients for patient-specific recommendations for cardiovascular disease |
|
|
Zafeiridi et al [ | ], 2018CAREGIVERSPRO-MMDai | Web-based platform | Development | User-centered design | Social network for sharing information, tips, and support across peers and health professionals |
|
|
Zheng et al [ | ], 2017NR | mHealth app with PROs | Preimplementation | User-centered design principles | SDM between patients and clinicians for knee arthritis treatment |
|
|
aSTOP: Self-Management to Prevent Stroke.
bSIDEAL: Soporte Innovador al paciente con Dependencia del Alcohol, Innovative Support to the Alcohol Dependent Patient.
cMI: motivational interviewing.
dmHealth: mobile health.
eP3P: The Personal Patient Profile-Prostate.
fNR: not reported.
gData not available.
hCDSS: clinical decision support system.
iGUI: graphical user interface.
jADAPT: Avoiding Diabetes Through Action Plan Targeting.
keCODES: Electronic Collaborative Decision Support.
lOSCAR: Ontology-Supported Computerized Assistive Technology Recommender.
mLUCID: logical user-centered interaction design.
nPATH: Personal Assistant for Tracking Health.
oIHT: interactive health technology.
pCONNECT: web-based communication aid.
qC-SHIP: Cognitive-Social Health Information Processing.
rCOMPASS: Computerized Decision Aid for Stroke Thrombolysis.
sS-TPI: Stroke-Thrombolytic Predictive Instrument.
tCARDSS: Cardiac Rehabilitation Decision Support System.
ue-RAPID: Electronic Patient Self-Reporting of Adverse-Events: Patient Information and Advice.
vBACPR: British Association for Cardiovascular Prevention and Rehabilitation.
wPRO: patient-reported outcome.
xANSWER: Animated, Self-Serve, Web-Based Research Tool.
yCP-PDA: Continence Product Patient Decision Aid.
zSUNDAE: Standards for Universal Reporting of Patient Decision Aid Evaluations.
aaEQUATOR: Enhancing the Quality and Transparency of Health Research.
abCONSORT: Consolidated Standards of Reporting Trials.
aciMHere: Interactive Mobile Health and Rehabilitation.
adCeHRes: Center for eHealth Research and Disease Management.
aeEVIDEM: Evidence and Value: Impact on Decision-Making.
afMCDA: multicriteria decision analysis.
agHTA: health technology assessment.
ahDME: durable medical equipment.
aiCAREGIVERSPRO-MMD: Caregivers Patient-Reported Outcome-Mild Mental Disorder.
Usability Evaluation Methods
The usability evaluation methods were categorized, based on the framework by Jacobsen [
], into (1) empirical (think-aloud protocol, 14/38, 36%; user tracking, 3/38, 8%; performance measures, 4/38, 10%; field test, 2/38, 5%; video recording, 1/38, 2%; and screen capture, 2/38, 5%), (2) inspection (cognitive walk-through, 1/38, 2% and Near live clinical situation, 1/38, 2%), and (3) inquiry (focus groups, 3/38, 8%; workshops, 2/38, 5%; semistructured interviews, 16/38, 42%; structured interviews, 1/38, 2%; questionnaires, 24/38, 63%; observations, 5/38, 13%; and comments, 3/38, 8%; ). An important point to emphasize is the frequency with which researchers used 1 (13/38, 34%), 2 (15/38, 39%), 3 (7/38, 18%), 4 (2/38, 5%), and 6 (1/38, 2%) methods from the framework by Jacobsen [ ], presented in [ - ]. Most (28/38, 73%) used 1 or 2 methods of evaluation. Usability was assessed during development (18/38, 47%), preimplementation (13/38, 34%), or implementation (7/38, 18%) through a variety of measures, including usability questionnaires (15/38, 39%), tailored tools developed by the authors (17/38, 45%), and acceptance and satisfaction questionnaires (6/38, 16%). The usability evaluation parameters identified by the authors were mapped to the usability parameters explained by Gupta et al [ ], including effectiveness (13/38, 34%), efficiency (12/38, 31%), memorability (13/38, 34%), productivity (2/38, 5%), security (2/38, 5%), and satisfaction (32/38, 84% and ).Study | Method | Jacobsen [ | ] frameworkDetails |
Anderson et al [ | ], 2014
|
|
|
Barrio et al [ | ], 2017
|
|
|
Bauerle Bass et al [ | ], 2018
|
|
|
Berry et al [ | ], 2015
|
|
|
Bogza et al [ | ], 2020
|
|
|
Burns and Pickens [ | ], 2017
|
|
|
Canally et al [ | ], 2015
|
|
|
Chrimes et al [ | ], 2014
|
|
|
Cox et al [ | ], 2015
|
|
|
Cuypers et al [ | ], 2019
|
|
|
De Vito Dabbs et al [ | ], 2009
|
|
|
Danial-Saad et al [ | ], 2016
|
|
|
Fleisher et al [ | ], 2008
|
|
|
Flynn et al [ | ], 2015
|
|
|
Fu et al [ | ], 2020
|
|
|
Goud et al [ | ], 2008
|
|
|
Grim et al [ | ], 2017
|
|
|
Holch et al [ | ], 2017
|
|
|
Jameie et al [ | ], 2019
|
|
|
Jessop et al [ | ], 2020
|
|
|
Kallen et al [ | ], 2012
|
|
|
Li et al [ | ], 2013
|
|
|
Murphy et al [ | ], 2020
|
|
|
Rochette et al [ | ], 2008
|
|
|
Schön et al [ | ], 2018
|
|
|
Setiawan et al [ | ], 2019
|
|
|
Snyder et al [ | ], 2009
|
|
|
Span et al [ | ], 2014
|
|
|
Span et al [ | ], 2018
|
|
|
Støme et al [ | ], 2019
|
|
|
Tony et al [ | ], 2011
|
|
|
Toth-Pal et al [ | ], 2008
|
|
|
Tsai et al [ | ], 2019
|
|
|
Van Maurik et al [ | ], 2019
|
|
|
Welch et al [ | ], 2015
|
|
|
Williams et al [ | ], 2016
|
|
|
Zafeiridi et al [ | ], 2018
|
|
|
Zheng et al [ | ], 2017
|
|
|
aUSE: Usefulness, Satisfaction and Ease of Use.
bSUS: System Usability Scale.
cASQ: After-Scenario Questionnaire.
dPSSUQ: Post-Study System Usability Questionnaire.
eCSUQ: Computer System Usability Questionnaire.
fPrepDM: Preparation for Decision Making.
gPRO: patient-reported outcome.
hNR: not reported.
Usability measures | Type of scale | Items | Usability evaluation parameters identified by the authors | Gupta et al [ | ] framework
Acceptability questionnaire [ | ]5-point Likert scale |
|
|
|
ASQa [ | ]7-point Likert scale |
|
|
|
Clinical decision support report questionnaire [ | ]5-point Likert scale |
|
|
|
Feasibility of encodes [ | ]—e |
|
|
|
IBM CSUQf [ | ]7-point Likert scale |
|
|
|
LQg [ | ]6-point Likert scale |
|
|
|
Measures of accessibility and satisfaction [ | ]5-point Likert scale |
|
|
|
Mobile app usability concept questions [ | ]5-point Likert scale |
|
|
|
Nielsen heuristic checklist [ | ]Likert scale |
|
|
|
Open-ended questionnaire for usability [ | ]— |
|
|
|
Patient satisfaction with diabetes telehealth program questionnaire [ | ]5-point Likert scale |
|
|
|
Perceived general helpfulness and value [ | ]3-point Likert scale |
|
|
|
Perceived helpfulness [ | ]10-point Likert scale |
|
|
|
Perceived usefulness [ | ]10-point Likert scale |
|
|
|
PSSUQk [ | , ]7-point Likert scale |
|
|
|
PrepDMl scale [ | ]5-point Likert scale |
|
|
|
Remote home monitoring device usability questionnaire [ | ]5-point Likert scale |
|
|
|
Survey for perception of easiness and usability of the 6 interfaces [ | ]5-point Likert scale |
|
|
|
Survey satisfaction questionnaire [ | ]Likert scale |
|
|
|
Telehealth nurse satisfaction questionnaire [ | ]5-point Likert scale |
|
|
|
Tool developed by the authors focused on description [ | ]5-point Likert scale |
|
|
|
Usability questionnaire [ | ]100-point Likert scale |
|
|
|
Usability questionnaire [ | ]10-point Likert scale |
|
|
|
Usability scale (SUS) questionnaire [ | , , , - , , , , ]5-point Likert scale |
|
|
|
USEp questionnaire [ | ]7-point Likert scale |
|
|
|
Usefulness and relevance survey [ | ]7-point Likert scale |
|
|
|
User acceptance and satisfaction scale [ | ]5-point Likert scale |
|
|
|
User-friendliness measured with an instrument based on the CeHResq assessment of design quality [ | ]— |
|
|
|
User-specific evaluation questionnaire for clinicians [ | ]— |
|
|
|
User-specific evaluation questionnaire for patients and caregivers [ | ]— |
|
|
|
aASQ: After-Scenario Questionnaire.
bBG: blood glucose.
cBP: blood pressure.
dEMR: electronic medical record.
eData not available.
fCSUQ: Computer System Usability Questionnaire.
gLQ: learnability questionnaire.
hRHM: remote home monitoring.
iT2D: type 2 diabetes.
jHCV: hepatitis C virus.
kPSSUQ: Post-Study System Usability Questionnaire.
lPrepDM: Preparation for Decision-Making.
mDSM: diabetes self-management.
nPCP: primary care provider.
oCDS: clinical decision support.
pUSE: Usefulness, Satisfaction, and Ease of Use.
qCeHRes: Center for eHealth Research and Disease Management.
Frameworks and Theoretical Models
The frameworks and theoretical models reported by the authors during the development, implementation, and evaluation of the technologies to support SDM reflected 5 categories: technology design (15/38, 39%), behavior change (21/38, 21%), analysis (9/38, 24%), SDM framework (8/38, 21%), and not reported (9/38, 24%;
). Notably, 24% (9/38) of the studies did not report using a framework or model during any stage of their research. Authors most commonly reported using a model or framework as a foundation to inform the design of their respective SDM technologies. User-centered design (9/15, 60%) was the most frequently used technology design framework.Discussion
Principal Findings
This scoping review was conducted to provide knowledge about how usability is evaluated when developing or implementing rehabilitation technologies aimed at supporting SDM. The first research question examined the methods and measures used in the context of SDM at different phases of technology development and implementation. Our findings revealed 14 reported methods that can help in evaluating the overall functionalities of the system and whether it fulfills the users’ requirements [
] and can be effective for identifying issues with a system [ ]. The most frequent reported methods included think-aloud protocols (14/38, 36%), semistructured interviews (16/38, 42%), and questionnaires (24/38, 63%; ). There was a total of 30 usability measures reported ( ), with the System Usability Scale being the most frequently used among the included studies. We operationalized the different types of methods used through the model by Jacobsen [ ], reflecting empirical methods (based on users’ experience with the technology in a systematic way), inspection methods (conducted by experts who examine usability-related aspects of a user interface without involving any users), and inquiry methods (based on the information about users’ needs, likes, and understanding of the technology through interviews or focus groups, observation, or comments). Notably, the reported methods were predominantly classified as inquiry and empirical ( ).The second research question examined the parameters of usability that were measured and reported. We found that the methods used to evaluate different parameters of usability varied according to the a priori framing of usability, demonstrated by the variations in the definitions of usability described by the authors (
). There was an evolution in the definition of usability across the included studies, with more recent studies (published since 2016) using the unified definition proposed by the ISO [ , , , , , , , ]. The usability parameters of the definitions were categorized based on the proposed comprehensive hierarchal model by Gupta et al [ ] as effectiveness (9/38, 23%), efficiency (8/38, 21%), memorability (11/38, 29%), satisfaction (14/38, 37%), security (5/38, 13%), universality (4/38, 10%), and productivity (10/38, 26%). These are consistent with the 3 constructs of the ISO standards, which are effectiveness, efficiency, and satisfaction, and allows for a more detailed categorization of usability parameters.Although the ISO standards [
] and the usability model by Gupta et al [ ] provide dimensions that could be considered as primary usability parameters, there remain challenges with measuring usability that emerged in this review. On the surface, usability is a simple concept. In fact, simplicity is at the heart of usability; however, measuring usability is not simple. Paradoxically, the ISO definition of usability is complex. Usability is about the person’s experience; however, that experience is influenced by many aspects, such as a person’s behavior and social network and the complexity of the technological functionalities. Usability may be viewed as a feature of the technology or an emergent property of the interaction between the user, the system, and contextual factors. Evaluating usability through these lens leads to using inspection, empirical, or inquiry methods [ ]. These can be applied at different stages of development of a technology (ie, in a developmental laboratory, in preimplementation, or during implementation), as described by the included studies ( ).This review revealed that evaluating usability requires a comprehensive approach with several methods to cover multiple usability parameters. Most articles included in this review (36/38, 95%) focused on inquiry methods, relying heavily on questionnaires and semistructured interviews to evaluate usability, and the most frequent empirical method was think-aloud protocols (
). Although a comprehensive approach is suggested for accurate usability evaluation, this was largely not shown in the included articles. Rather, 73% (28/38) of the included studies only used 1 or 2 methods in total to evaluate usability. Only 2% (1/38) of the studies, conducted by Span et al [ ], incorporated multiple methods that covered all 3 dimensions—inquiry, inspection, and empirical [ ]. However, some of the included studies (2/38, 5%) described different usability evaluations for the same technology at different stages of development in separate articles (eg, “Take Charge, Get Cured” in the developmental [ ] and preimplementation [ ] stages). It is believed that the combination of inspection, empirical, and inquiry methods can provide more accurate and complete results in finding usability problems as there is no exact method considered to be the best for usability evaluation [ ]. Matera et al [ ] developed a systematic usability evaluation framework to address this challenge. They posited that usability can be reliably evaluated by systematically combining evaluation methods [ ]. Recent reviews of usability not specific to SDM in software [ ], mobile health [ ], eHealth [ ], user experience [ ], and web development [ ] mirrored the results of this review in that few studies used a combination of evaluation methods.However, the lack of reported inspection methods demonstrated in this review may partially be explained by the inherent nature of SDM technologies for rehabilitation rather than a lack of comprehensive evaluation. Very few examples of inspection methods were demonstrated across the included studies, with only 2% (1/38) using cognitive walk-throughs and an additional 2% (1/38) using “near live” clinical situations. Critically, inspection methods refer to evaluations conducted by specific usability experts [
], not by the end users of the technology (eg, patients and clinicians). As the purpose of technology to support SDM in rehabilitation is to improve patient-centered care, the consideration of end users in the development—and, consequently, the usability evaluations—is crucial to ensure that the technology will be understood and adopted by the target population. Therefore, we propose that a comprehensive approach for evaluating the usability of rehabilitation technologies aimed at supporting SDM could focus on empirical and inquiry methods to prioritize the input of the patient and clinician end users.Although questionnaires were found to be the most common method used overall, the identified measures of usability in the included studies demonstrated limitations in comprehensiveness, largely mapping to the parameters of satisfaction and memorability (
). The emphasis on the parameter of satisfaction (demonstrated in 32/38, 84% of measures) may reflect the importance of this parameter when developing technologies for SDM in rehabilitation (eg, the importance of evaluating the usefulness, user-friendliness, and ease of use). However, this may also reflect key missing areas in usability evaluation. Critically, the parameters of usability described by the authors in their a priori definitions of usability were not found to be consistent with the parameters of the measures that were used. Therefore, although individuals may be conceptualizing usability in a comprehensive manner, the measurement itself was not comprehensive. For example, there was a demonstrated lack of measurement of the parameters of effectiveness and efficiency, which were both described in the definition of usability in 34% (13/38) of the included studies, although both were only found to be used in 23% (9/38) of usability measures.This review uncovered the need for inclusion of theoretical models or frameworks during various stages of SDM usability studies to guide which usability parameter to measure. Theoretical models and frameworks were infrequently reported (
). Most studies in this review (27/38, 71%) reported using 1 model or framework, whereas some (10/38, 26%) integrated 2. Only 2% (1/38) of the studies, carried out by Bauerle Bass et al [ ], exhibited an in-depth application of models and frameworks as underpinnings to their research. The most common (9/38, 24%) and perhaps the most beneficial framework, user-centered design, served as the foundation for designing an SDM technology [ , , ].The importance of using theoretical models and frameworks during the development, implementation, and analysis of technologies and evaluation of usability is demonstrated through the implications of poor usability [
, , ], which discourages users from using the technology systems. Moreover, if the technology systems are not user-friendly, then they can increase the problems experienced by users. Solutions to systems failing to meet the users’ needs include understanding user feedback [ ], usability evaluations [ ], involving users in the early stages of development [ ], and including professionals such as providers [ ]. There is a need for flexibility and for friendly, simple, and self-explanatory interfaces that allow users to interact with the system [ ]. For the systems to be effective, it is important to assess a system that is easy to use on a daily basis. This would increase the ability of the patients to control their diseases and allow their daily lives to be more satisfying [ ]. The technology systems need to be designed for a particular type of user and need to be easy to use to create acceptance. The usability of the technology system is vital as it has a high degree of influence over the success of the system. Thus, the system needs to be designed to provide a friendly environment for the user to develop a positive attitude toward using it and lead to its successful adoption.It is envisioned that the involvement of end users in the development of SDM technologies will continue to grow and that more applications of existing technology, such as mobile phones, websites, or applications, will be used to benefit individuals with disabilities. We also anticipate that more companies may show an interest in this market, potentially promoting frequent use of SDM technologies in rehabilitation care. However, there are challenges in the development of SDM technologies, such as tailoring to individuals’ capabilities and properly addressing the emotional state of individuals with disabilities or cognitive impairments during everyday tasks. It will be critical to develop these technologies in a way that meets individual variations in needs and abilities of individuals with disabilities so that they really help maintain autonomy, provide meaningful activities, and promote decision-making [
, , ].An important area for this growing field will be how to effectively integrate end-user input throughout all stages of development of such SDM technologies, including effective usability testing. An additional challenge for the field of rehabilitation care in supporting SDM technologies would be in integrating the technology into the built environment, such as a client-server system, and into routine care [
]. There is a clear need for new methods of rapid SDM technology appraisal and evaluation to inform deployment to overcome the barriers that will be faced because of the expected further integration of SDM technologies within the built environment.Limitations
We did not assess the quality of the included articles, consistent with the scoping review methodology [
, ]. Therefore, we included studies with different designs and different quality levels, which allowed for a broad exploration of measures and methods used to evaluate the usability of SDM technologies. In our results, we focused mainly on general usability measures and did not report the psychometric properties and clinical utility of these measures. Future work needs to evaluate the psychometric properties and clinical utility of usability measures through a systematic review methodology with a quality assessment of the included articles. Another limitation was that we did not include gray literature as this scoping review aimed to examine the reported measures and methods used in peer-reviewed rehabilitation literature on SDM technologies. It could be an area of interest for future work to examine what methods and measures are used in gray literature.Conclusions
The results of this scoping review highlight the importance as well as the complexity of usability evaluation. Although various methods and measures were shown to be used to evaluate the usability of technologies to support SDM in rehabilitation, very few evaluations used in the included studies adequately spanned the selected usability parameters. This review identified gaps in usability evaluation as most studies relied solely on questionnaires rather than a combination of inspection and empirical methods and most questionnaires simply focused on the usability parameter of satisfaction. We recommend for individuals to adopt a comprehensive approach to usability evaluation of SDM technologies, starting with a clear definition of how usability is conceptualized to guide the structure of the evaluation. In addition, we recommend the use of multiple usability evaluation methods categorized as inspection (eg, questionnaires, focus groups, and interviews) or empirical (eg, think-aloud protocols) to capture a more complete picture of end-user needs and interpretations. The selected methods should span a variety of parameters of usability, not just satisfaction (eg, effectiveness, efficiency, memorability, security, universality, and productivity). The consideration of end users (such as patients and clinicians) is of particular importance for the development of technologies to support SDM as the process of SDM itself aims to improve patient-centered care and integrate both patient and clinician voices into their rehabilitation care.
Acknowledgments
RA is supported by Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia. SA and CA are supported by career awards from the Fonds de recherche du Québec–Santé. The funding agency had no role in the design of the study, in the collection and analysis of the data, or in the decision to publish the results. The authors would also like to acknowledge the network investigators from Réseau provincial de recherche en adaptation-réadaptation that provided guidance for this work, particularly Geneviève Lessard, PhD. This study was funded by the Réseau provincial de recherche en adaptation-réadaptation.
Authors' Contributions
All authors contributed to the design of this study, provided critical insights, and contributed to the final written manuscript.
Conflicts of Interest
None declared.
Ovid MEDLINE search strategy.
DOCX File , 15 KBCharacteristics of the included studies and participants.
DOCX File , 24 KBReferences
- Barry MJ, Edgman-Levitan S. Shared decision making--pinnacle of patient-centered care. N Engl J Med. Mar 01, 2012;366(9):780-781. [CrossRef] [Medline]
- Sandman L, Munthe C. Shared decision making, paternalism and patient choice. Health Care Anal. Mar 2010;18(1):60-84. [CrossRef] [Medline]
- Légaré F, Witteman HO. Shared decision making: examining key elements and barriers to adoption into routine clinical practice. Health Aff (Millwood). Feb 2013;32(2):276-284. [CrossRef] [Medline]
- Davis S, MacKay L. Moving beyond the rhetoric of shared decision-making: designing personal health record technology with young adults with type 1 diabetes. Can J Diabetes. Jul 2020;44(5):434-441. [CrossRef] [Medline]
- Zisman-Ilani Y, Gorbenko KO, Shern D, Elwyn G. Comparing digital vs paper decision aids about the use of antipsychotic medication: client, clinician, caregiver and administrator perspectives. Int J Pers Cent Med. Jul 13, 2017;7(1):21-30. [FREE Full text] [CrossRef]
- Rose A, Rosewilliam S, Soundy A. Shared decision making within goal setting in rehabilitation settings: a systematic review. Patient Educ Couns. Jan 2017;100(1):65-75. [CrossRef] [Medline]
- Grenfell J, Soundy A. People's Experience of Shared Decision Making in Musculoskeletal Physiotherapy: A Systematic Review and Thematic Synthesis. Behav Sci (Basel). Jan 12, 2022;12(1):12. [FREE Full text] [CrossRef] [Medline]
- Matthews EB, Savoy M, Paranjape A, Washington D, Hackney T, Galis D, et al. Acceptability of health information exchange and patient portal use in depression care among underrepresented patients. J Gen Intern Med. Nov 2022;37(15):3947-3955. [FREE Full text] [CrossRef] [Medline]
- Zisman-Ilani Y, Roe D, Elwyn G, Kupermintz H, Patya N, Peleg I, et al. Shared decision making for psychiatric rehabilitation services before discharge from psychiatric hospitals. Health Commun. May 2019;34(6):631-637. [CrossRef] [Medline]
- Shay LA, Lafata JE. Where is the evidence? A systematic review of shared decision making and patient outcomes. Med Decis Making. Jan 2015;35(1):114-131. [FREE Full text] [CrossRef] [Medline]
- Wilson SR, Strub P, Buist AS, Knowles SB, Lavori PW, Lapidus J, et al. Better Outcomes of Asthma Treatment (BOAT) Study Group. Shared treatment decision making improves adherence and outcomes in poorly controlled asthma. Am J Respir Crit Care Med. Mar 15, 2010;181(6):566-577. [FREE Full text] [CrossRef] [Medline]
- Hartasanchez SA, Heen AF, Kunneman M, García-Bautista A, Hargraves IG, Prokop LJ, et al. Remote shared decision making through telemedicine: a systematic review of the literature. Patient Educ Couns. Feb 2022;105(2):356-365. [CrossRef] [Medline]
- Safran Naimark J, Madar Z, Shahar DR. The impact of a web-based app (eBalance) in promoting healthy lifestyles: randomized controlled trial. J Med Internet Res. Mar 02, 2015;17(3):e56. [FREE Full text] [CrossRef] [Medline]
- Solomon M, Wagner SL, Goes J. Effects of a web-based intervention for adults with chronic conditions on patient activation: online randomized controlled trial. J Med Internet Res. Feb 21, 2012;14(1):e32. [FREE Full text] [CrossRef] [Medline]
- Antonio MG, Petrovskaya O, Lau F. The state of evidence in patient portals: umbrella review. J Med Internet Res. Nov 11, 2020;22(11):e23851. [FREE Full text] [CrossRef] [Medline]
- Seljelid B, Varsi C, Solberg Nes L, Øystese KA, Børøsund E. Feasibility of a digital patient-provider communication intervention to support shared decision-making in chronic health care, involveme: pilot study. JMIR Form Res. Apr 07, 2022;6(4):e34738. [FREE Full text] [CrossRef] [Medline]
- Davis S, Roudsari A, Raworth R, Courtney KL, MacKay L. Shared decision-making using personal health record technology: a scoping review at the crossroads. J Am Med Inform Assoc. Jul 01, 2017;24(4):857-866. [FREE Full text] [CrossRef] [Medline]
- Kao H, Wei C, Yu M, Liang T, Wu W, Wu YJ. Integrating a mobile health applications for self-management to enhance telecare system. Telemat Inform. Jul 2018;35(4):815-825. [FREE Full text] [CrossRef]
- Taylor L, Capling H, Portnoy JM. Administering a telemedicine program. Curr Allergy Asthma Rep. Sep 15, 2018;18(11):57. [CrossRef] [Medline]
- Howe TL, Worrall LE, Hickson LM. What is an aphasia-friendly environment? Aphasiology. Aug 18, 2010;18(11):1015-1037. [FREE Full text] [CrossRef]
- Jokela T, Iivari N, Matero J, Karukka M. The standard of user-centered design and the standard definition of usability: analyzing ISO 13407 against ISO 9241-11. In: Proceedings of the Latin American Conference on Human-Computer Interaction. Presented at: CLIHC '03; August 17-20, 2003, 2003;53-60; Rio de Janeiro, Brazil. URL: https://dl.acm.org/doi/abs/10.1145/944519.944525 [CrossRef]
- Holzinger A. Usability engineering methods for software developers. Commun ACM. Jan 2005;48(1):71-74. [FREE Full text] [CrossRef]
- Gupta D, Ahlawat AK, Sagar K. Usability prediction and ranking of SDLC models using fuzzy hierarchical usability model. Open Eng. Jun 24, 2017;7(1):161-168. [FREE Full text] [CrossRef]
- Schultheis MT, Rebimbas J, Mourant R, Millis SR. Examining the usability of a virtual reality driving simulator. Assist Technol. 2007;19(1):1-10. [CrossRef] [Medline]
- Negrini S, Meyer T, Arienti C, Kiekens C, Pollock A, Selb M, et al. 3rd Cochrane Rehabilitation Methodology Meeting participants. The 3rd Cochrane rehabilitation methodology meeting: "rehabilitation definition for scientific research purposes". Eur J Phys Rehabil Med. Oct 2020;56(5):658-660. [CrossRef] [Medline]
- Damman OC, Jani A, de Jong BA, Becker A, Metz MJ, de Bruijne MC, et al. The use of PROMs and shared decision-making in medical encounters with patients: an opportunity to deliver value-based health care to patients. J Eval Clin Pract. Apr 2020;26(2):524-540. [FREE Full text] [CrossRef] [Medline]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32. [FREE Full text] [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [FREE Full text] [CrossRef] [Medline]
- Gammon D, Berntsen GK, Koricho AT, Sygna K, Ruland C. The chronic care model and technological research and innovation: a scoping review at the crossroads. J Med Internet Res. Feb 06, 2015;17(2):e25. [FREE Full text] [CrossRef] [Medline]
- Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Hoboken, NJ, USA. Wiley-Blackwell; 2008.
- Jacobsen NE. Usability evaluation methods: the reliability and usage of cognitive walkthrough and usability test. Department of Psychology, University of Copenhagen. Oct 05, 1999. URL: https://rauterberg.employee.id.tue.nl/lecturenotes/0H420/uem%5b1999%5d.pdf [accessed 2023-06-21]
- Anderson JA, Godwin KM, Saleem JJ, Russell S, Robinson JJ, Kimmel B. Accessibility, usability, and usefulness of a web-based clinical decision support tool to enhance provider-patient communication around self-management TO prevent (STOP) stroke. Health Informatics J. Dec 2014;20(4):261-274. [FREE Full text] [CrossRef] [Medline]
- Barrio P, Ortega L, López H, Gual A. Self-management and shared decision-making in alcohol dependence via a mobile app: a pilot study. Int J Behav Med. Oct 2017;24(5):722-727. [CrossRef] [Medline]
- Bauerle Bass S, Jessop A, Gashat M, Maurer L, Alhajji M, Forry J. Take charge, get cured: the development and user testing of a culturally targeted mHealth decision tool on HCV treatment initiation for methadone patients. Patient Educ Couns. Nov 2018;101(11):1995-2004. [CrossRef] [Medline]
- Berry DL, Halpenny B, Bosco JL, Bruyere Jr J, Sanda MG. Usability evaluation and adaptation of the e-health personal patient profile-prostate decision aid for Spanish-speaking Latino men. BMC Med Inform Decis Mak. Jul 24, 2015;15:56. [FREE Full text] [CrossRef] [Medline]
- Bogza LM, Patry-Lebeau C, Farmanova E, Witteman HO, Elliott J, Stolee P, et al. User-centered design and evaluation of a web-based decision aid for older adults living with mild cognitive impairment and their health care providers: mixed methods study. J Med Internet Res. Aug 19, 2020;22(8):e17406. [FREE Full text] [CrossRef] [Medline]
- Burns SP, Pickens ND. Embedding technology into inter-professional best practices in home safety evaluation. Disabil Rehabil Assist Technol. Aug 2017;12(6):585-591. [CrossRef] [Medline]
- Canally C, Doherty S, Doran DM, Goubran RA. Using integrated bio-physiotherapy informatics in home health-care settings: a qualitative analysis of a point-of-care decision support system. Health Informatics J. Jun 2015;21(2):149-158. [FREE Full text] [CrossRef] [Medline]
- Chrimes D, Kitos NR, Kushniruk A, Mann DM. Usability testing of avoiding diabetes thru action plan targeting (ADAPT) decision support for integrating care-based counseling of pre-diabetes in an electronic health record. Int J Med Inform. Sep 2014;83(9):636-647. [FREE Full text] [CrossRef] [Medline]
- Cox CE, Wysham NG, Walton B, Jones D, Cass B, Tobin M, et al. Development and usability testing of a web-based decision aid for families of patients receiving prolonged mechanical ventilation. Ann Intensive Care. Mar 25, 2015;5:6. [FREE Full text] [CrossRef] [Medline]
- Cuypers M, Lamers RE, Kil PJ, The R, Karssen K, van de Poll-Franse LV, et al. A global, incremental development method for a web-based prostate cancer treatment decision aid and usability testing in a Dutch clinical setting. Health Informatics J. Sep 2019;25(3):701-714. [FREE Full text] [CrossRef] [Medline]
- De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs. May 2009;27(3):175-183. [FREE Full text] [CrossRef] [Medline]
- Danial-Saad A, Kuflik T, Weiss PL, Schreuer N. Usability of clinical decision support system as a facilitator for learning the assistive technology adaptation process. Disabil Rehabil Assist Technol. 2016;11(3):188-194. [CrossRef] [Medline]
- Fleisher L, Buzaglo J, Collins M, Millard J, Miller SM, Egleston BL, et al. Using health communication best practices to develop a web-based provider-patient communication aid: the CONNECT study. Patient Educ Couns. Jun 2008;71(3):378-387. [FREE Full text] [CrossRef] [Medline]
- Flynn AJ, Friedman CP, Boisvert P, Landis-Lewis Z, Lagoze C. The knowledge object reference ontology (KORO): a formalism to support management and sharing of computable biomedical knowledge for learning health systems. Learn Health Syst. Apr 16, 2018;2(2):e10054. [FREE Full text] [CrossRef] [Medline]
- Fu H, Rizvi RF, Wyman JF, Adam TJ. Usability evaluation of four top-rated commercially available diabetes apps for adults with type 2 diabetes. Comput Inform Nurs. Jun 2020;38(6):274-280. [FREE Full text] [CrossRef] [Medline]
- Goud R, Jaspers MW, Hasman A, Peek N. Subjective usability of the CARDSS guideline-based decision support system. Stud Health Technol Inform. 2008;136:193-198. [Medline]
- Grim K, Rosenberg D, Svedberg P, Schön UK. Development and usability testing of a web-based decision support for users and health professionals in psychiatric services. Psychiatr Rehabil J. Sep 2017;40(3):293-302. [CrossRef] [Medline]
- Holch P, Warrington L, Bamforth LC, Keding A, Ziegler LE, Absolom K, et al. Development of an integrated electronic platform for patient self-report and management of adverse events during cancer treatment. Ann Oncol. Sep 01, 2017;28(9):2305-2311. [FREE Full text] [CrossRef] [Medline]
- Jameie S, Haybar H, Aslani A, Saadat M. Development and usability evaluation of web-based telerehabilitation platform for patients after myocardial infarction. Stud Health Technol Inform. 2019;261:68-74. [Medline]
- Jessop AB, Bass SB, Brajuha J, Alhajji M, Burke M, Gashat MT, et al. "Take charge, get cured": pilot testing a targeted mHealth treatment decision support tool for methadone patients with hepatitis C virus for acceptability and promise of efficacy. J Subst Abuse Treat. Feb 2020;109:23-33. [CrossRef] [Medline]
- Kallen MA, Yang D, Haas N. A technical solution to improving palliative and hospice care. Support Care Cancer. Jan 2012;20(1):167-174. [CrossRef] [Medline]
- Li LC, Adam PM, Townsend AF, Lacaille D, Yousefi C, Stacey D, et al. Usability testing of ANSWER: a web-based methotrexate decision aid for patients with rheumatoid arthritis. BMC Med Inform Decis Mak. Dec 01, 2013;13:131. [FREE Full text] [CrossRef] [Medline]
- Murphy C, de Laine C, Macaulay M, Fader M. Development and randomised controlled trial of a continence product patient decision aid for men postradical prostatectomy. J Clin Nurs. Jul 2020;29(13-14):2251-2259. [CrossRef] [Medline]
- Rochette A, Korner-Bitensky N, Tremblay V, Kloda L. Stroke rehabilitation information for clients and families: assessing the quality of the strokengine-family website. Disabil Rehabil. 2008;30(19):1506-1512. [CrossRef] [Medline]
- Schön UK, Grim K, Wallin L, Rosenberg D, Svedberg P. Psychiatric service staff perceptions of implementing a shared decision-making tool: a process evaluation study. Int J Qual Stud Health Well-being. Dec 2018;13(1):1421352. [FREE Full text] [CrossRef] [Medline]
- Setiawan IM, Zhou L, Alfikri Z, Saptono A, Fairman AD, Dicianno BE, et al. An adaptive mobile health system to support self-management for persons with chronic conditions and disabilities: usability and feasibility studies. JMIR Form Res. Apr 25, 2019;3(2):e12982. [FREE Full text] [CrossRef] [Medline]
- Snyder CF, Jensen R, Courtin SO, Wu AW, Website for Outpatient QOL Assessment Research Network. PatientViewpoint: a website for patient-reported outcomes assessment. Qual Life Res. Sep 2009;18(7):793-800. [FREE Full text] [CrossRef] [Medline]
- Span M, Hettinga M, Groen-van de Ven L, Jukema J, Janssen R, Vernooij-Dassen M, et al. Involving people with dementia in developing an interactive web tool for shared decision-making: experiences with a participatory design approach. Disabil Rehabil. Jun 2018;40(12):1410-1420. [CrossRef] [Medline]
- Span M, Smits C, Jukema J, Groen-van de Ven L, Janssen R, Vernooij-Dassen M, et al. An interactive web tool to facilitate shared decision-making in dementia: design issues perceived by caregivers and patients. Int J Adv Life Sci. 2014;6(3-4):107-121. [FREE Full text] [CrossRef]
- Støme LN, Pripp AH, Kværner JS, Kvaerner KJ. Acceptability, usability and utility of a personalised application in promoting behavioural change in patients with osteoarthritis: a feasibility study in Norway. BMJ Open. Jan 28, 2019;9(1):e021608. [FREE Full text] [CrossRef] [Medline]
- Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. Nov 30, 2011;11:329. [FREE Full text] [CrossRef] [Medline]
- Toth-Pal E, Wårdh I, Strender LE, Nilsson G. Implementing a clinical decision-support system in practice: a qualitative analysis of influencing attitudes and characteristics among general practitioners. Inform Health Soc Care. Mar 2008;33(1):39-54. [CrossRef] [Medline]
- Tsai YL, Huang JJ, Pu SW, Chen HP, Hsu SC, Chang JY, et al. Usability assessment of a cable-driven exoskeletal robot for hand rehabilitation. Front Neurorobot. Feb 13, 2019;13:3. [FREE Full text] [CrossRef] [Medline]
- van Maurik IS, Visser LN, Pel-Littel RE, van Buchem MM, Zwan MD, Kunneman M, et al. Development and usability of ADappt: web-based tool to support clinicians, patients, and caregivers in the diagnosis of mild cognitive impairment and Alzheimer disease. JMIR Form Res. Jul 08, 2019;3(3):e13417. [FREE Full text] [CrossRef] [Medline]
- Welch G, Balder A, Zagarins S. Telehealth program for type 2 diabetes: usability, satisfaction, and clinical usefulness in an urban community health center. Telemed J E Health. May 2015;21(5):395-403. [CrossRef] [Medline]
- Williams PA, Furberg RD, Bagwell JE, LaBresh KA. Usability testing and adaptation of the pediatric cardiovascular risk reduction clinical decision support tool. JMIR Hum Factors. Jun 21, 2016;3(1):e17. [FREE Full text] [CrossRef] [Medline]
- Zafeiridi P, Paulson K, Dunn R, Wolverson E, White C, Thorpe JA, et al. A web-based platform for people with memory problems and their caregivers (CAREGIVERSPRO-MMD): mixed-methods evaluation of usability. JMIR Form Res. Mar 12, 2018;2(1):e4. [FREE Full text] [CrossRef] [Medline]
- Zheng H, Tulu B, Choi W, Franklin P. Using mHealth app to support treatment decision-making for knee arthritis: patient perspective. EGEMS (Wash DC). Apr 20, 2017;5(2):7. [FREE Full text] [CrossRef] [Medline]
- Feldman-Stewart D, Brundage MD, Nickel JC, MacKillop WJ. The information required by patients with early-stage prostate cancer in choosing their treatment. BJU Int. Feb 2001;87(3):218-223. [CrossRef] [Medline]
- Feldman-Stewart D, Brundage MD, Van Manen L, Svenson O. Patient-focussed decision-making in early-stage prostate cancer: insights from a cognitively based decision aid. Health Expect. Jun 2004;7(2):126-141. [FREE Full text] [CrossRef] [Medline]
- Feldman-Stewart D, Brennenstuhl S, Brundage MD, Roques T. An explicit values clarification task: development and validation. Patient Educ Couns. Nov 2006;63(3):350-356. [CrossRef] [Medline]
- Feldman-Stewart D, Brundage MD, Zotov V. Further insight into the perception of quantitative information: judgments of gist in treatment decisions. Med Decis Making. Jan 2007;27(1):34-43. [CrossRef] [Medline]
- Feldman-Stewart D, Tong C, Siemens R, Alibhai S, Pickles T, Robinson J, et al. The impact of explicit values clarification exercises in a patient decision aid emerges after the decision is actually made: evidence from a randomized controlled trial. Med Decis Making. Jul 2012;32(4):616-626. [CrossRef] [Medline]
- van Osch M, Rövekamp A, Bergman-Agteres SN, Wijsman LW, Ooms SJ, Mooijaart SP, et al. User preferences and usability of iVitality: optimizing an innovative online research platform for home-based health monitoring. Patient Prefer Adherence. Jun 30, 2015;9:857-867. [FREE Full text] [CrossRef] [Medline]
- Lilholt PH, Jensen MH, Hejlesen OK. Heuristic evaluation of a telehealth system from the Danish TeleCare north trial. Int J Med Inform. May 2015;84(5):319-326. [CrossRef] [Medline]
- Paz F, Paz FA, Villanueva D, Pow-Sang JA. Heuristic evaluation as a complement to usability testing: a case study in web domain. In: Proceedings of the 12th International Conference on Information Technology-New Generations. Presented at: ITNG '15; April 13-15, 2015, 2015;546-551; Las Vegas, NV, USA. URL: https://ieeexplore.ieee.org/document/7113530 [CrossRef]
- Matera M, Costabile MF, Garzotto F, Paolini P. SUE inspection: an effective method for systematic usability evaluation of hypermedia. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans. Jan 2002;32(1):93-103. [FREE Full text] [CrossRef]
- Sagar K, Saha A. A systematic review of software usability studies. Int J Inf Technol. 2017:1-24. [FREE Full text] [CrossRef]
- Al-Saadi TA, Aljarrah TM, Alhashemi AM, Hussain A. A systematic review of usability challenges and testing in mobile health. Int J Account Financ Report. Jul 12, 2015;5(2):1. [FREE Full text] [CrossRef]
- Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inform. Jun 2019;126:95-104. [CrossRef] [Medline]
- Maia CL, Furtado ES. A systematic review about user experience evaluation. In: Proceedings of the 5th International Conference on Design, User Experience, and Usability: Design Thinking and Methods. Presented at: DUXU '16; July 17–22, 2016, 2016;445-455; Toronto, Canada. URL: https://link.springer.com/chapter/10.1007/978-3-319-40409-7_42 [CrossRef]
- Insfran E, Fernandez A. A systematic review of usability evaluation in web development. In: Proceedings of the 2008 International Workshops on Web Information Systems Engineering. Presented at: WISE '08; September 1-4, 2008, 2008;81-91; Auckland, New Zealand. URL: https://link.springer.com/chapter/10.1007/978-3-540-85200-1_10 [CrossRef]
- Cook EJ, Randhawa G, Guppy A, Sharp C, Barton G, Bateman A, et al. Exploring factors that impact the decision to use assistive telecare: perspectives of family care-givers of older people in the United Kingdom. Ageing Soc. 2018;38(9):1912-1932. [FREE Full text] [CrossRef]
- Martínez García MA, Fernández Rosales MS, López Domínguez E, Hernández Velázquez Y, Domínguez Isidro S. Telemonitoring system for patients with chronic kidney disease undergoing peritoneal dialysis: usability assessment based on a case study. PLoS One. Nov 06, 2018;13(11):e0206600. [FREE Full text] [CrossRef] [Medline]
- Saeed N, Manzoor M, Khosravi P. An exploration of usability issues in telecare monitoring systems and possible solutions: a systematic literature review. Disabil Rehabil Assist Technol. Apr 2020;15(3):271-281. [CrossRef] [Medline]
- Sánchez-Morillo D, Crespo M, León A, Crespo Foix LF. A novel multimodal tool for telemonitoring patients with COPD. Inform Health Soc Care. Jan 2015;40(1):1-22. [CrossRef] [Medline]
- Klack L, Ziefle M, Wilkowska W, Kluge J. Telemedical versus conventional heart patient monitoring: a survey study with German physicians. Int J Technol Assess Health Care. Oct 2013;29(4):378-383. [CrossRef] [Medline]
- Prescher S, Bourke AK, Koehler F, Martins A, Ferreira HS, Sousa TB, et al. Ubiquitous ambient assisted living solution to promote safer independent living in older adults suffering from co-morbidity. In: Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Presented at: EMBC' 12; August 28 to September 1, 2012, 2012;5118-5121; San Diego, CA, USA. URL: https://ieeexplore.ieee.org/document/6347145 [CrossRef]
- Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med Res Methodol. Mar 23, 2013;13:48. [FREE Full text] [CrossRef] [Medline]
Abbreviations
ISO: International Organization for Standardization |
PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews |
SDM: shared decision-making |
Edited by T Leung; submitted 26.07.22; peer-reviewed by W van Harten, Y Zisman-Ilani; comments to author 20.12.22; revised version received 27.02.23; accepted 31.03.23; published 15.08.23.
Copyright©Rehab Alhasani, Nicole George, Dennis Radman, Claudine Auger, Sara Ahmed. Originally published in JMIR Rehabilitation and Assistive Technology (https://rehab.jmir.org), 15.08.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Rehabilitation and Assistive Technology, is properly cited. The complete bibliographic information, a link to the original publication on https://rehab.jmir.org/, as well as this copyright and license information must be included.