Published on in Vol 5, No 2 (2018): Jul-Dec

Users’ Perspectives, Opportunities, and Barriers of the Strengthen Your Ankle App for Evidence-Based Ankle Sprain Prevention: Mixed-Methods Process Evaluation for a Randomized Controlled Trial

Users’ Perspectives, Opportunities, and Barriers of the Strengthen Your Ankle App for Evidence-Based Ankle Sprain Prevention: Mixed-Methods Process Evaluation for a Randomized Controlled Trial

Users’ Perspectives, Opportunities, and Barriers of the Strengthen Your Ankle App for Evidence-Based Ankle Sprain Prevention: Mixed-Methods Process Evaluation for a Randomized Controlled Trial

Original Paper

1Amsterdam Collaboration on Health and Safety in Sports, Department of Public & Occupational Health, Amsterdam Public Health Research Institute, VU University Medical Centre, Amsterdam, Amsterdam, Netherlands

2Consumer Safety Institute VeiligheidNL, Amsterdam, Netherlands

3School of Public Health, Physiotherapy and Population Sciences, University College Dublin, Dublin, Ireland

4School of Human Movement and Nutrition Sciences, Faculty of Health and Behavioural Sciences, University of Queensland, Brisbane, Brisbane, Australia

5Division of Exercise Science and Sports Medicine, Department of Human Biology, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa

*these authors contributed equally

Corresponding Author:

Evert Verhagen, PhD

Amsterdam Collaboration on Health and Safety in Sports

Department of Public & Occupational Health, Amsterdam Public Health Research Institute

VU University Medical Centre, Amsterdam

Van der Boechorststraat 7

Amsterdam, 1081BT

Netherlands

Phone: 31 204449691

Email: e.verhagen@vumc.nl


Background: The “Strengthen Your Ankle” neuromuscular training program has been thoroughly studied over the past 8 years. This process evaluation is a part of a randomized controlled trial that examined both the short- and long-term effectiveness of this particular program. Although it was shown previously that the program, available both in a printed booklet and as a mobile app, is able to effectively reduce the number of recurrent ankle sprains, participants’ compliance with the program is an ongoing challenge.

Objective: This process evaluation explored participants’ opinions regarding both the methods of delivery, using RE-AIM (Reach Effectiveness Adoption Implementation Maintenance) Framework to identify barriers and challenges to program compliance. Although Reach, Effectiveness, and Adaptation were the focus of a previous study, this paper focuses on the implementation and maintenance phases.

Methods: Semistructured interviews and online questionnaires were analyzed using qualitative content analysis. Fisher exact, chi-square, and t tests assessed between-group differences in quantitative survey responses. Interviews were assessed by thematic analysis to identify key themes.

Results: While there were no significant differences in the perceived simplicity, usefulness, and liking of the exercise during the 8 weeks of the neuromuscular training program, semistructured interviews showed that 14 of 16 participants agreed that an app would be of additional benefits over a booklet. After the 12-month follow-up, when asked how they evaluated the overall use of the app or the booklet, the users of the app gave a mean score of 7.7 (SD 0.99) versus a mean score 7.1 (SD 1.23) for the users of the booklet. This difference in mean score was significant (P=.006).

Conclusions: Although both the app and booklet showed a high user satisfaction, the users of the app were significantly more satisfied. Semistructured questionnaires allowed users to address issues they would like to improve in future updates. Including a possibility for feedback and postponement of exercises, an explanation of the use of specific exercises and possibly music were identified as features that might further improve the contentment of the program, probably leading to increased compliance.

Trial Registration: Netherlands Trial Register NTR4027; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4027 (Archived by Webcite at http://www.webcitation.org/70MTo9dMV)

JMIR Rehabil Assist Technol 2018;5(2):e13

doi:10.2196/rehab.8638

Keywords



Injuries, due to participation in sports and physical activities, are prevalent. Internationally, ankle sprains are one of the most common musculoskeletal injuries [1]. In particular, indoor and court sports have shown high incidences of ankle sprains with up to 7 injuries per 1000 hours of participation [2]. Generally considered a “minor” injury, ankle sprains pose a significant risk for long-term secondary complaints like instability and chronic pain [3]. For the prevention of acute lateral ankle sprains, numerous effective strategies have been developed and evaluated for their cost-effectiveness [2].

One of the many available interventions that has been shown to be effective in reducing the risk of recurrent ankle sprains, as well as protecting against secondary complaints, is neuromuscular training (NMT) [3-5]. Multiple variations of such training programs have been evaluated [6-8], including the “Strengthen Your Ankle” program. The “Strengthen Your Ankle” program consists of 6 exercises that are performed 3 times a week, over 8 weeks. Multiple trials have indicated that this program can be effective in reducing the injury incidence density [9,10] as well as being cost-effective [10,11]. Despite the proven value of the program in preventing recurrent injury risk, compliance with this and other NMT programs is an ongoing challenge [3]. Sufficient compliance with NMT programs is essential for successful prevention of ankle sprains [12]. Consequently, a free mobile app was developed as a novel and attractive means of providing athletes with the “Strengthen Your Ankle” program [13]. Details of the app have been described elsewhere [3]. A recent trial (NTR 4027) showed that the app neither increased compliance nor decreased recurrence of ankle sprains compared with a standard program administered via a paper booklet [3,4,13].

As with other preventive interventions, the translation of the evidence on ankle sprain prevention through NMT to the real-world context of sports remains a challenge, by which effective ankle sprain prevention in the community is lagging [14]. The success of introducing any intervention strategy in a practical context can be evaluated using the RE-AIM framework [15]. RE-AIM is a conceptual framework that was originally used to develop and evaluate health care programs. The goal of the RE-AIM framework is to “encourage program planners, evaluators, readers of journal articles, funders, and policy makers to pay more attention to essential program elements, including external validity, that can improve the sustainable adoption and implementation of effective, generalizable, evidence-based interventions” [16].

Although developed for use in health care settings, the RE-AIM framework has been previously used to evaluate the success of introducing strategies for sports injury prevention within a practical sports context [17,18]. Consequently, using the components of the RE-AIM framework, this study described the user experience of the “Strengthen Your Ankle” app and booklet to understand why compliance was challenged during program implementation.


Design and Participants

The full details of the “Strengthen Your Ankle” study have been described elsewhere [3,4,13]. In brief, 220 sports participants who experienced an ankle sprain during the past 2 months were included in this RCT. Participants were randomly assigned to either the app or booklet intervention group and were instructed to follow the embedded 8 week “Strengthen Your Ankle” NMT prevention program using either the app or the printed booklet.

Outcome Measures

The RE-AIM framework describes five dimensions to evaluate the practical feasibility of an intervention: “Reach,” “Effectiveness,” “Adoption,” “Implementation,” and “Maintenance” [16]. The dimensions “Reach” and “Adoption” are out of scope when describing the feasibility of an intervention within a controlled trial. As such, for the this study, we focused on the dimensions “Effectiveness,” “Implementation,” and “Maintenance.”

Effectiveness

The “Effectiveness” dimension describes the clinical impact of the studied intervention. The short- and long-term effectiveness of the app compared with the booklet for preventing ankle sprain recurrences were assessed in a RCT. The full methods and results of this trial have been published elsewhere [3,4,13]. In order to put the outcomes of the “Implementation” and “Maintenance” dimensions in context, we will briefly summarize the “Effectiveness” outcomes.

Implementation

Implementation concerns the participants’ use of the intervention strategies. In this study, we quantified use as compliance with the 8-week NMT program in each of the study groups, measured as a percentage of the total program completed. In addition, the participants’ attitudes and perceptions toward the delivery of the NMT programs were assessed.

During the 8 weeks of the NMT program, participants received a weekly online questionnaire. The questionnaire registered what percentage of the program was executed during the week, the amount of difficulty that was experienced while conducting each of the exercises, and the reason for a possible lack of compliance. For each of the 6 different exercises, participants indicated what percentage of the exercises they performed each week. Additionally, using a 5-point Likert scale, participants were asked how they perceived the exercises. When participants failed to complete the questionnaire, reminders were sent by email. The details on the questionnaire have been published previously [3].

After the 8-week training period, a more extensive evaluation questionnaire was completed, including closed and free-text questions on the subjectively-experienced value of the NMT program delivery mode, a subjective evaluation of the program, and the perceived disadvantages and advantages of the allocated intervention delivery mode. To measure satisfaction, all remaining participants (75 in the app group and 88 in the booklet group) were asked to give a 0-10 score for the app or booklet. An unpaired t test was performed to examine the difference in scores between the two groups.

Maintenance

“Maintenance” describes the long-term effectiveness of the intervention strategies. For this study, this dimension was defined as the percentage of participants still conducting the NMT program combined with the advantages the participants perceived related to the app or paper booklet use for intervention delivery.

After 12 months, semistructured interviews were conducted with individual participants to assess the perceived advantages of using the app over the paper booklet. All study participants were asked if they were willing to participate in a semistructured interview concerning the NMT program; 27% (32/119) of the remaining participants, evenly divided over the two study groups, responded positively. The interviews were structured using a preselected topic list on the individual experiences with the NMT program either through the booklet or app. All interviews were conducted and transcribed by one researcher (MA). Interviews were conducted via telephone until saturation was reached, that is, when interviews did not lead to new themes or information, within both study groups, resulting in 16 semistructured interviews with 8 randomly selected participants in the booklet group and 8 randomly selected participants in the app group. Multimedia Appendix 1 shows the question guide for the semistructured interviews, aimed at process evaluation, after finishing the 12-month intervention.

Data Analyses

Due to dropout during follow-up (n=57 after 8 weeks and a further n=44 after 12 months), sample sizes differed between questionnaires. The reasons for dropout were unknown. The participants’ answers on the 5-point Likert scales regarding attitudes and perceptions toward the program, as registered during the 8-week program, were averaged for each participant over the available follow-up moments. Independent sample t tests with assumed equal variances were conducted to assess for differences in the average Likert responses between the two study groups. The significance level was evaluated at P=.05. SPSS (version 22.0) and was used for all statistical analyses.

All semistructured interviews were audiorecorded and transcribed verbatim. In transcriptions, any personal information or information that was deducible to an individual was anonymized. Verbatim-transcribed interviews were thematically analyzed and fragmented on the basis of topical similarity using Atlas.ti [19]. Open, inductive coding was used line by line on the transcripts of the interviews and these codes were converged into subthemes [20]. Peer debriefing was used as an external check to the research process. This method of analysis was used after each interview and ended when no new codes arose and saturation was reached [19]. The final step in the analysis process was to submerge the subthemes to a limited number of main themes [19].


Effectiveness

Previous studies that looked at the effectiveness of the “Strengthen Your Ankle” program provided further details on the (cost)-effectiveness of the program in the short and long term [3,4]. In short, during the 8 weeks of the NMT, there were 93 self-reported recurrent ankle sprains, which resulted in injury incidence densities of 25.3 per 1000 hours of sport (95% CI 18.0 to 32.7) in the app group and 25.6 per 1000 hours of sport (95% CI 18.3 to 32.9) in the booklet group. There was no significant difference in the incidence densities of self-reported recurrences (HR [hazard ratio] 3.07; 95% CI 0.62 to 15.20) [1].

During the 12-month follow-up, there were 139 recurrent ankle injuries, resulting in injury incidence densities of 15.59 per 1000 hours of sport (95% CI 11.94 to 19.24) in the app group and 15.84 (95% 12.10 to 19.58) in the booklet group. Over the long term, this difference in injury density was not significant (HR 1.06; 0.76 to 1.49) [4].

Implementation

The first study in this larger research project looked at compliance during the 8 weeks of the NMT intervention. It was shown that the average compliance to the exercise scheme was 73.3% (95% CI 67.7% to 78.1%) in the app group and 76.7% (95% CI 71.9 to 82.3%) in the booklet group. No significant difference in compliance was found between the groups [3].

The weekly questionnaires (Table 1) showed that participants in both the app and booklet groups gave comparable scores with regard to simplicity, usefulness, and subjective evaluation of the exercises. Table 1 shows the averaged responses of the participants over the 8 weeks.

After the 8-week intervention period, 35 participants using the app and 22 participants using the booklet discontinued the study for unknown reasons. The remaining 75 users of the app found this method of NMT program delivery more user friendly, easier, fun to use, and less annoying and thought that the videos were more helpful than the booklet (Table 2). The latter question should be interpreted with caution because online videos were available for the booklet users (n=88), but many of the participants stated that they were not aware of this possibility.

Table 1. Participants’ attitudes and perceptions toward the allocated delivery of the NMT program during the 8-week intervention period.
Participants’ opinions and method of deliveryMean (SD)aMean differenceb (95% CI)P value
The exercises are simple.
0.03 (−0.19 to 0.25).79

App3.79 (0.86)


Booklet3.76 (0.78)

Due to the variation in exercises I stay motivated.
−0.16 (−0.36 to 0.05).13

App2.25 (0.82)


Booklet2.41 (0.71)

I find it easy to execute the exercises without help.
0.05 (−0.16 to 0.26).65

App3.72 (0.85)


Booklet3.67 (0.75)

The exercises give me a sense of security.
−0.01 (−0.25 to 0.23).96

App3.30 (0.94)


Booklet3.30 (0.87)

The exercises are painful.
−0.04 (−0.22 to 0.14).64

App3.94 (0.68)


Booklet3.98 (0.67)

The exercises don’t fit with my regular schedule.
0.09 (−0.14 to 0.32).47

App3.42 (0.87)


Booklet3.33 (0.88)

I have too little time to do the exercises.
−0.09 (−0.35 to 0.17).49

App3.29 (0.99)


Booklet3.38 (0.97)

I think the exercises take a long time.
−0.15 (−0.32 to −0.01).07

App2.00 (0.58)


Booklet2.16 (0.67)

The exercises make me tired.
−0.02 (−0.21 to 0.17).84

App3.87 (0.75)


Booklet3.89 (0.66)

I forget to execute the exercises.
−0.06 (−0.24 to 0.11).49

App2.34 (0.68)


Booklet2.41 (0.64)

The exercises are not useful to prevent a recurrent injury.

App3.42 (0.88)0.12 (−0.11 to 0.35).32
Booklet3.31 (0.84)
The exercises won’t help me.
0.07 (−0.13 to 0.26).50

App2.66 (0.77)

Booklet2.59 (0.71)

aScores present means (SD) of 5-point Likert scales (1=strongly agree; 5=strongly disagree).

bDifferences in scores between groups were analyzed by independent t tests with equal variances assumed.

Table 2. The subjectively-experienced value of the NMT program and perceived disadvantages and advantages of the allocated intervention delivery mode assessed directly after the 8-week intervention.
Participants’ opinions and method of deliveryMean (SD)aMean differenceb (95% CI)P value
The intervention is user friendly.
−0.43 (−0.75 to −0.11).009

App1.85 (0.98)
Booklet2.28 (1.10)
The intervention is easy to use.
−0.40 (−0.69 to −0.11).008

App1.84 (0.92)
Booklet2.24 (0.97)
The intervention looks attractive.
−0.06 (−0.35 to 0.23).68

App2.12 (0.90)
Booklet2.18 (0.97
Navigation of the intervention is clear.
−0.29 (−0.59 to 0.01).06

App2.13 (0.95)
Booklet2.42 (1.01)
The intervention gives enough information.
−0.29 (−0.59 to 0.01).06

App2.19 (0.95)
Booklet2.48 (0.97)
I would advise others to use the intervention.
−0.29 (−0.62 to 0.03).07

App2.08 (1.03)
Booklet2.38 (1.04)
It is annoying to use the intervention.
0.47 (0.12 to 0.81).008

App4.09 (1.09)
Booklet3.63 (1.13)
I have used the intervention with pleasure.
−0.18 (−0.48 to 0.12).23

App2.25 (0.95)
Booklet2.44 (0.97)
The videos helped me (online for the Booklet).
−0.99 (−1.31 to −0.68)< .001

App1.96 (1.07)
Booklet2.95 (0.96)
The written instructions helped me.
−0.07 (−0.35 to 0.21).64

App2.08 (0.98)
Booklet2.15 (0.84)
The schedule helped me.
0.08 (−0.23 to 0.38).62

App2.12 (1.10)
Booklet2.05 (0.87)
The intervention is boring.
−0.05 (−0.36 to 0.26).73

App3.48 (1.03)
Booklet3.53 (0.97)
The intervention makes it easier to do the exercises.
−0.36 (−0.65 to −0.07).02

App2.09 (0.94)
Booklet2.45 (0.95)
The intervention makes it fun to do the exercises.
−0.37 (−0.66 to −0.08).01

App2.68 (0.94)
Booklet3.06 (0.93)
The intervention is informative.
−0.14 (−0.39 to 0.11)0.26

App2.20 (0.74)
Booklet2.34 (0.84)
The intervention is trustworthy.
−0.17 (−0.42 to 0.09)0.13

App2.23 (0.84)
Booklet2.40 (0.870)
The explanation of the exercises is clear.
−0.22 (−0.52 to 0.10)0.17

App2.26 (1.07)
Booklet2.47 (0.91)

aScores present means (SD) of 5-point Likert scales (1=strongly agree; 5=strongly disagree).

bDifferences in scores between groups were analyzed through independent t tests with equal variances assumed.

Therefore, the answers of 53 of the booklet users were “neutral” when asked if the online videos were of help; this was in comparison with 5% (4/75) in the app group. Some participants failed to answer all the questions, the number of missing responses can be found in Table 2. Additional questions specifically related to possible improvements in the app, and not the booklet, (Multimedia Appendix 2) indicated that participants desired feedback after the exercises (44/75, 59%) and wanted the ability to postpone a training session (41/75, 55%). Overall, a t test showed that the users of the app were significantly more satisfied with the app (score 1 out of 10 with 10 referring to the highest score, mean±SD) compared with booklet users; 7.7 (SD 0.99) versus 7.1 (SD 1.23) P=.006.

Maintenance

At the end of the 12-month follow-up period, an additional 44 participants discontinued the study. These participants were asked if they were still doing (part of the) NMT program. Only 23% (28/122) of all participants still in the study responded affirmatively. We did not ask what amount of the program they were still doing.

Two main themes arose from the semistructured interviews that related to the design of the app and possible additional benefits of the app. Fourteen out of 16 participants stated that an app would provide an additional benefit compared with a booklet. The main reasons given were that most of the participants always had their mobile phones with them and that the app provided visual support and had a reminder function. The two participants who did not feel that the app offered any benefit found the exercises too easy, which made the app redundant.

Errors in navigation and explanation, the lack of feedback and music, and lack of explanation of the purpose of the exercises were the main disadvantages experienced by the app users. The greatest perceived disadvantages of the booklet were the big size when folded out, small font, lack of robustness, and errors in explanation. Table 3 shows the individual responses during the semistructured interviews to illustrate the flavor of the original data and demonstrate the prevalence of the themes, as suggested by King [21].

Table 3. Individual responses from semistructured interviews.
Method of delivery and respondentAdded benefit of the app?Reason givenPros (+) and suggestions for improvement (−) for the app
App

R1YesYou always have your phone with you
You forget the booklet
+ Easy to use
+ Agenda function
+ Videos with instructions

R2NoThe exercises are so easy, you don’t need an app+ Videos with instructions
+ Tick off done exercises
R3YesYou always have your phone with you
Seeing the app on my phone reminds you to do the exercises
+ Tick off done exercises
− Show why you need to do an exercise

R4YesThe app gives visual support+ Easy to use
+ Videos with instructions

R5YesYou always have your phone with you+ Easy to use


R6YesThe app is smaller and thus easier to use+ Easier navigation

R7YesThe app gives visual support
Seeing the app on my phone motivates you to do the exercises
+ Videos with instructions
+ Counting down the number of exercises

R8YesYou always have your phone with you+ Videos with instructions
+ Tick off done exercises
Booklet

R9YesYou always have your phone with you− Show why you need to do an exercise.
+ Reminder to do the exercises.

R10NoThe exercises are so easy, you don’t need an app− Stopwatch function
R11YesThe app gives visual support− Show why you need to do an exercise

R12YesYou always have your phone with you+ Reminder to do the exercise
− Possibility to postpone exercises

R13YesThe app gives visual support+ Videos with instructions


R14YesYou always have your phone with you
Seeing the app on my phone would remind you to do the exercises
− Direct translation of the app to a booklet

R15YesYou always have your phone with you− More variation in the exercises

R16YesAn agenda function would be easy− Direct translation of the app to a booklet

Principal Findings

Previous studies [3,4] have shown that using an app or a booklet with a special NMT program to prevent recurrent ankle sprains has resulted in comparable injury densities during both short- (8 weeks) and long-term (12 months) follow-ups and comparable compliance rates with the program. During the execution of the program during the first 8 weeks, the app and booklet were given comparable scores for simplicity, usefulness, and liking of the exercises. After the 12-month follow-up, the users of the app were significantly more satisfied with the app compared with the users of the booklet. The users of the app evaluated the app as more patient friendly, easier to use, and less annoying and thought that the videos were helpful. With the help of semistructured interviews, 14 out of 16 participants agreed that an app would be of additional benefit over a booklet, mainly due to use of instructional videos, phone portability, and the agenda function. Further suggestions for improving the app that were mentioned by various participants were the ability to postpone exercises and the provision of exercise feedback.

Interventions for preventing sport injuries require high participant compliance [3] Therefore, ways to increase compliance are a focus of many intervention studies [3]. The “Strengthen Your Ankle” program was developed in 2009. Since then, the program has been studied intensively [3,4,9-11,22]. It was shown that (1) the program was effective in reducing recurrent ankle sprains for those with high compliance [10], (2) the use of either the app or a booklet produced nonsignificant differences in injury densities in both the short and long term [3,4], and (3) both methods had comparable cost-effectiveness of implementation [23].

Over the years, compliance with the “Strengthen Your Ankle” program in RCTs has steadily increased from 23% [9] to 45% [10] and 75% [3], likely as a result of annual updates, increased acknowledgment of the usefulness of the program by the target population, and improvements in the program content. However, the reach of the target population still requires substantial attention. In 2011, the annual number of downloads of the “Strengthen Your Ankle” app reached 25,781, which corresponds to a low percentage (25,781/911,576, 2.6%) of potential users [18]. Some studies have looked at the use of apps in injury prevention over the last decade. What can be concluded from those studies is that numerous apps seek to prevent (re)injury. However, the scientific evidence supporting these app-based programs is nonexistent or scarce [22,24]. A recent review found that out of 18 apps concerned with preventing sports and physical activity-related injuries, only four included evidence regarding efficacy [22]. In addition to the app that is the focus of this study, one of those four apps dealt with ankle injury prevention using NMT. No information is available on the use or compliance of the other app [22].

This study aimed to explore user experiences with the NMT program, as well as with the app and booklet as delivery methods, by means of semistructured interviews. The information gathered can be used to further improve the methods of delivery and, thus, increase future reach and compliance. The interviews and questionnaires showed that the app and booklet can be successfully used to prevent recurrent ankle sprains and that both show high user satisfaction. Future updates may include options for feedback or postponement of exercises, an explanation of the use of specific exercises, and possibly music; these additions could further improve user perceptions of the program and hence increase compliance.

A limitation of this study, and that of previous studies on the “Strengthen Your Ankle” program, is the mismatch between compliance and adherence. Although both constructs have been used interchangeably, they are not synonymous. Adherence refers to a situation where a clinician or researcher develops a program in cooperation with the participant. The participant attempts to follow the program as best as possible, taking personal preferences and constraints into consideration. Adherence can be seen as what happens in real-life conditions when individuals with an ankle sprain try to follow the program; compliance is studied in clinical settings. The extent to which the participant obeys the program instructions is measured by compliance rates [12,24]. Research, ideally performed in a more or less controlled setting, implicitly focuses on compliance, rather than on adherence. However, the “Strengthen Your Ankle” program is meant to increase adherence for all individuals at risk for an ankle sprain, not only for those who participate in the studies involved. This study has tried to explore the barriers and opportunities that participants experienced while using the training program via an app or booklet within a controlled study setting. However, because the interviews were held after follow-up, that is, months after the participants had finished the 8 weeks of the training program, we expected to gain insight as to program performance in real-life situations.

A further limitation of this study is the possibility of selection bias for the semistructured interviews. It is possible that only those participants that carried a strong negative or positive view of the program agreed to participate because the invitation for the interviews was made only after termination of the 12-month follow-up. Additionally, the (single) interviewer did not structure the interviews and continued to question the participants when needed. This may have affected the validity of the data analyses. However, it is recognized that this characteristic is inherent to the flexible nature of thematic analysis and does not threaten the depth of analysis [5].

Conclusions

With the use of semistructured interviews and online questionnaires, we were able to evaluate users’ opinions on both the app and booklet. The users of the app were significantly more satisfied with the app although there was no significant difference in the perceived simplicity, usefulness, and liking of the exercise during the 8 weeks of the NMT program. In the interviews, users acknowledged the need for improvements. Future updates should take the users’ suggestions into account because adherence with the NMT program remains an ongoing challenge.

Acknowledgments

This study was funded by the Netherlands Organisation for Health Research and Development (ZonMw) grant number 525001003, Balance boards were provided by Disporta and Booklets were provided by VeiligheidNL.

We would like to thank the following partners for their collaboration and recruitment of study participants: Royal Dutch Society for Physical Therapy (KNGF), Dutch Society for Physical Therapy in Sports (NVFS), Dutch College of General Practitioners (NHG), Dutch Sports Medicine Society (VSG), Dutch Olympic Committee (NOC*NSF), Zilveren Kruis Achmea (ZKA), and Disporta.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Question guide for the process evaluation using semi-structured interviews after finishing the 12-month intervention period.

PDF File (Adobe PDF File), 37KB

Multimedia Appendix 2

Responses to process evaluation of the neuromuscular training program after the intervention period.

PDF File (Adobe PDF File), 42KB

  1. Gribble P, Bleakley C, Caulfield B, Docherty C, Fourchet F, Fong DTP, et al. 2016 consensus statement of the International Ankle Consortium: prevalence, impact and long-term consequences of lateral ankle sprains. Br J Sports Med 2016 Dec;50(24):1493-1495. [CrossRef] [Medline]
  2. Doherty C, Bleakley CM, Delahunt E, Holden S. Treatment and prevention of acute and recurrent ankle sprain: an overview of systematic reviews with meta-analysis. Br J Sports Med 2017 Jan;51(2):113-125. [CrossRef] [Medline]
  3. van Reijen M, Vriend I, Zuidema V, van Mechelen W, Verhagen E. Increasing compliance with neuromuscular training to prevent ankle sprain in sport: does the 'Strengthen your ankle' mobile App make a difference? A randomised controlled trial. Br J Sports Med 2016 Oct;50(19):1200-1205. [CrossRef] [Medline]
  4. van Reijen M, Vriend I, Zuidema V, van Mechelen W, Verhagen E. The “Strengthen your ankle” program to prevent recurrent injuries: A randomized controlled trial aimed at long-term effectiveness. J Sci Med Sport 2017 Jun;20(6):549-554. [CrossRef] [Medline]
  5. Verhagen E, Bay K. Optimising ankle sprain prevention: a critical review and practical appraisal of the literature. Br J Sports Med 2010 Dec;44(15):1082-1088. [CrossRef] [Medline]
  6. Pasanen K, Parkkari J, Pasanen M, Hiilloskorpi H, Makinen T, Jarvinen M, et al. Neuromuscular training and the risk of leg injuries in female floorball players: cluster randomised controlled study. British Journal of Sports Medicine 2008 Oct 01;42(10):502-505. [CrossRef]
  7. Parkkari J, Taanila H, Suni J, Mattila V, Ohrankämmen O, Vuorinen P, et al. Neuromuscular training with injury prevention counselling to decrease the risk of acute musculoskeletal injury in young men during military service: a population-based, randomised study. BMC Med 2011 Apr 11;9:35 [FREE Full text] [CrossRef] [Medline]
  8. Steffen K, Emery C, Romiti M, Kang J, Bizzini M, Dvorak J, et al. High adherence to a neuromuscular injury prevention programme (FIFA 11+) improves functional balance and reduces injury risk in Canadian youth female football players: a cluster randomised trial. Br J Sports Med 2013 Aug;47(12):794-802. [CrossRef] [Medline]
  9. Hupperets M, Verhagen E, van Mechelen W. Effect of unsupervised home based proprioceptive training on recurrences of ankle sprain: randomised controlled trial. BMJ 2009 Jul 09;339:b2684 [FREE Full text] [Medline]
  10. Janssen K, van Mechelen W, Verhagen E. Bracing superior to neuromuscular training for the prevention of self-reported recurrent ankle sprains: a three-arm randomised controlled trial. Br J Sports Med 2014 Aug;48(16):1235-1239 [FREE Full text] [CrossRef] [Medline]
  11. Hupperets M, Verhagen E, Heymans M, Bosmans J, van Tulder M, van Mechelen W. Potential Savings of a Program to Prevent Ankle Sprain Recurrence. Am J Sports Med 2010 Aug 10;38(11):2194-2200. [CrossRef] [Medline]
  12. Verhagen E, Hupperets M, Finch C, van Mechelen W. The impact of adherence on sports injury prevention effect estimates in RCTs: Looking beyond the CONSORT statement. J Sci Med Sport 2011:14-292. [Medline]
  13. Van RM, Vriend I, Zuidema V, van Mechelen W, Verhagen E. The implementation effectiveness of the 'Strengthen your ankle' smartphone application for the prevention of ankle sprains: design of a randomized controlled trial. BMC Musculoskelet Disord 2014 Jan 07;15:2 [FREE Full text] [CrossRef] [Medline]
  14. Verhagen E, Bolling C, Finch C. Caution this drug may cause serious harm! Why we must report adverse effects of physical activity promotion. Br J Sports Meda. 49 Jan 2015;49(1):1. [Medline]
  15. Ekegren CL, Donaldson A, Gabbe BJ, Finch CF. Implementing injury surveillance systems alongside injury prevention programs: evaluation of an online surveillance system in a community setting. Inj. Epidemiol 2014 Jul 24;1(1). [CrossRef]
  16. Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [Medline]
  17. Lindblom H, Waldén M, Carlfjord S, Hägglund M. Implementation of a neuromuscular training programme in female adolescent football: 3-year follow-up study after a randomised controlled trial. Br J Sports Med 2014 Oct;48(19):1425-1430. [CrossRef] [Medline]
  18. Vriend I, Coehoorn I, Verhagen E. Implementation of an app-based neuromuscular training programme to prevent ankle sprains: a process evaluation using the RE-AIM Framework. Br J Sports Med 2015 Apr;49(7):484-488. [CrossRef] [Medline]
  19. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006 Jan;3(2):77-101. [CrossRef]
  20. Baarda D, de GM, Teunissen J. Basisboek Kwalitatief Onderzoek. In: Handleiding voor het opzetten en uitvoeren van kwalitatief onderzoek. Groningen: Noordhoff Uitgevers B.V; 2001.
  21. King N. Using templates in the thematic analysis of text. In: Essential guide to qualitative methods in organizational research. London: Sage Publications; 2004:257-270.
  22. van Mechelen D, van Mechelen W, Verhagen E. Sports injury prevention in your pocket?! Prevention apps assessed against the available scientific evidence: a review. Br J Sports Med 2014 Jun;48(11):878-882. [CrossRef] [Medline]
  23. van Reijen M, Vriend I, van Mechelen W, Verhagen E. Preventing recurrent ankle sprains: Is the use of an App more cost-effective than a printed Booklet? Results of a RCT. Scand J Med Sci Sports 2017 May 19. [CrossRef] [Medline]
  24. Verhagen E, Bolling C. Protecting the health of the @hlete: how online technology may aid our common goal to prevent injury and illness in sport. Br J Sports Med 2015 Jan 22;49(18):1174-1178. [CrossRef]


ESSM: exercise science and sports medicine
HR: hazard ratio
NMT: neuromuscular training
RCT: randomized controlled trial
ZKA: Zilveren Kruis Achmea


Edited by G Eysenbach; submitted 03.08.17; peer-reviewed by C McKay, R Fallaize, A Samaranayaka; comments to author 23.11.17; revised version received 11.01.18; accepted 09.05.18; published 06.07.18

Copyright

©Miriam van Reijen, Marianne Asscheman, Ingrid Vriend, Willem van Mechelen, Evert Verhagen. Originally published in JMIR Rehabilitation and Assistive Technology (http://rehab.jmir.org), 06.07.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Rehabilitation and Assistive Technology, is properly cited. The complete bibliographic information, a link to the original publication on http://rehab.jmir.org/, as well as this copyright and license information must be included.