Published on 21.12.18 in Vol 5, No 2 (2018): Jul-Dec
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10439, first published Mar 17, 2018.
A Web-Based Knowledge Translation Resource for Families and Service Providers (The “F-Words” in Childhood Disability Knowledge Hub): Developmental and Pilot Evaluation Study
Background: The “F-words in Childhood Disability” (Function, Family, Fitness, Fun, Friends, and Future) are an adaptation and an attempt to operationalize the World Health Organization’s (2001) International Classification of Functioning, Disability and Health (ICF) framework. Since the paper was published (November 2011), the “F-words” have attracted global attention (>12,000 downloads, January 2018). Internationally, people have adopted the “F-words” ideas, and many families and service providers have expressed a need for more information, tools, and resources on the “F-words”.
Objective: This paper reports on the development and pilot evaluation of a Web-based knowledge translation (KT) resource, the “F-words” Knowledge Hub that was created to inform people about the “F-words” and to provide action-oriented tools to support the use of the “F-words” in practice.
Methods: An integrated research team of families and researchers at CanChild Centre for Childhood Disability Research collaborated to develop, implement, and evaluate the Knowledge Hub. A pilot study design was chosen to assess the usability and utility of the Web-based hub before implementing a larger evaluation study. Data were collected using a brief anonymous Web-based survey that included both closed-ended and open-ended questions, with the closed-ended responses being based on a five-point Likert-type scale. We used descriptive statistics and a summary of key themes to report findings.
Results: From August to November 2017, the Knowledge Hub received >6,800 unique visitors. In 1 month (November 2017), 87 people completed the survey, of whom 63 completed the full survey and 24 completed 1 or 2 sections. The respondents included 42 clinicians and 30 family members or individuals with a disability. The majority of people visited the Knowledge Hub 1-5 times (n=63) and spent up to 45 minutes exploring (n=61) before providing feedback. Overall, 66 people provided information on the perceived usefulness of the Knowledge Hub, of which 92% (61/66) found the Knowledge Hub user-friendly and stated that they enjoyed exploring the hub, and a majority (n=52) reported that the Knowledge Hub would influence what they did when working with others. From the open-ended responses (n=48), the “F-words” videos (n=21) and the “F-words” tools (n=15) were rated as the best features on the Knowledge Hub.
Conclusions: The “F-words” Knowledge Hub is an evidence-informed Web-based KT resource that was useful for respondents, most of whom were seen as “early adopters” of the “F-words” concepts. Based on the findings, minor changes are to be made to improve the Knowledge Hub before completing a larger evaluation study on the impact at the family, clinician, and organizational levels with a wider group of users. Our hope is that the “F-words” Knowledge Hub will become a go-to resource for knowledge sharing and exchange for families and service providers.
JMIR Rehabil Assist Technol 2018;5(2):e10439
- childhood disability;
- Knowledge Hub;
- knowledge-to-action framework;
- knowledge translation;
- Web-based resource
It has been several years since the paper “The ‘F-words’ in Childhood Disability: I swear this is how we should think!” was published in Child: Care, Health and Development . The “F-words” (Function, Family, Fitness, Fun, Friends, and Future) are an adaptation and operationalization of the World Health Organization’s International Classification of Functioning, Disability and Health (ICF) [ ]. The initial aim of the “F-words” paper was to spread awareness of the ICF and to encourage people to apply these modern ways of thinking and developmental approaches to childhood disability [ ]. Since it was first published (early November 2011) to December 2017, the paper has been cited over 140 times and downloaded over 12,000 times.
In 2014, based on considerable interest in the paper, we formed an integrated research team at CanChild Centre for Childhood Disability Research focused on disseminating the “F-words” into practice. At that time, several parents (ie, early adopters) had learned about the “F-words,” liked the ideas, and were interested in how to share the “F-words” message with more families. Recognizing the potential impact of an integrated approach to this work (ie, families and researchers working together), we partnered with family stakeholders to develop and evaluate knowledge translation (KT) strategies tailored to meet the families’ needs and preferences.
The first project involved the development, dissemination, and evaluation of a three-minute awareness video . A video was chosen as an initial dissemination strategy as it was engaging, relatively easy to produce, and could be freely shared with a broad audience. At that time, the “F-words” paper was not yet open access and thus was reaching a limited audience in the scientific and clinical communities.
We evaluated the video by tracking its reach and asking viewers to complete an anonymous Web-based survey. In the first 2 months, there were 715 views and 137 survey responses. Overall, we learned that 97.8% (134/137) of people “extremely liked” or “liked” the “F-words” ideas, 87.5% (120/137) indicated they would share the video, and 92.7% (127/137) wanted to learn more. The CanChild website was identified by 65.7% of respondents (90/137) as the most popular strategy for sharing further information on the “F-words” concepts. A complete report of our findings and the lessons learned from this project are published .
The awareness video was only the first step toward moving the “F-words” into practice. By January 2015, we had given >30 international presentations and the “F-words” ideas had continued to spread over social media. We were gratified by the uptake of these ideas around the world and were excited to see the imaginative ways in which people were adapting and adopting the “F-words” to local contexts. We were also learning a great deal about the application of the “F-words” by connecting and working with families and other stakeholders such as service providers and health care administrators around the world. Therefore, as a research team, we were acting as “knowledge brokers”  by working with interested people to share and exchange knowledge on the “F-words” concepts.
From our conversations with the families and service providers, it was evident that there was significant interest in having more information on the “F-words” as well as action-oriented resources and tools to assist with the application of the “F-words” into practice. Furthermore, as the “F-words” ideas continued to spread, we recognized the need (and opportunity) to compile and share all that was being done on the “F-words” ideas by building a centralized Web-based community for knowledge sharing and exchange. Therefore, in 2015, our research team decided to develop, implement, and evaluate the usability and utility of a Web-based KT resource: a website called “The ‘F-words’ in Childhood Disability Knowledge Hub.”
The purpose of the “F-words” Knowledge Hub was to inform families and service providers about “F-words”/ICF concepts and to provide action-oriented tools to support the uptake and use of the “F-words” in practice. The Knowledge Hub is currently hosted on CanChild’s website  and is meant to be an ever-growing resource for knowledge sharing and exchange. The CanChild website is world-renowned in the field of childhood disability and receives over 12,000 unique visitors each month from over 205 countries [ ].
Modern Approaches to Knowledge Translation
In the last several years, there has been increasing interest in using the internet as a platform for KT and the use of Web-based KT resources as a strategy for disseminating health research evidence in the field of childhood disability [- ]. Levac et al [ ] defined Web-based KT resources as “e-learning products that translate evidence-based knowledge to disseminate information that increases awareness, informs clinical practice, and stimulates practice change.” The Web-based KT resources include websites, educational modules, downloadable PDFs, blogs, and wikis [ , ]. Some of the advantages of Web-based resources are (1) the ability to be self-paced or self-directed; (2) accessibility and broad reach; (3) incorporation of engaging multimedia content; and (4) promotion of knowledge sharing and exchange [ , ].
While the current evidence base for Web-based KT strategies is limited, some studies have shown promising findings [, ]; however, more research is needed to identify the most effective Web-based KT strategies and to understand their impact on behavior change and patient outcomes [ , ]. Research is also needed to explore the impact of Web-based KT resources as a single intervention compared with multifaceted interventions, such as a combination of Web-based KT resources and educational outreach [ , ].
This paper reports on the development process, usability, and utility of the Knowledge Hub. The Knowledge-to-Action (KTA) framework was used as the guiding theoretical underpinning for this research . KT theories, models, and frameworks are recommended to guide the development, implementation, and evaluation of KT strategies [ - ]. The KTA framework provided a conceptual map of the KT process steps involved in transferring knowledge to practice [ ]. For this study, we focused on the three steps of the action cycle: “select, tailor, and implement the intervention,” “monitor knowledge use,” and “evaluate outcomes.” This study was part of a larger research program that had already addressed the earlier stages of the action cycle [ ].
Integrated Knowledge Translation Strategy
We implemented a formal integrated knowledge translation (iKT) strategy to develop, implement, and evaluate the Knowledge Hub. iKT involves the collaboration of researchers and knowledge users (eg, families and service providers) throughout all stages of the research or KT process  and has been found to increase the effectiveness and sustainability of KT interventions [ , ]. This project was led by an integrated team of children’s health researchers (PR, SB, JWG), family stakeholders (DG, JB, DK), and a doctoral student (AC), who coordinated the project.
All team members were involved in each stage of the project: (1) participating in initial planning stages; (2) providing feedback on the content and design of the Knowledge Hub; (3) creating and sharing tools/resources; (4) assisting with evaluation; and (5) disseminating the hub across their social networks. During the initial planning stages, team meetings were held by teleconference. We initially planned to develop an “F-words” Tool Kit as a paper-based resource designed to share knowledge and provide tools/resources to support the use of the “F-words” in practice; however, after extensive conversations with stakeholders and a review of the literature, we turned toward Web-based KT strategies (ie, the Knowledge Hub). AC led the development of the hub, but feedback was sought and received from all team members throughout the development process. Most team correspondence was done through email.
An area in which all three family stakeholders were heavily involved was the creation of the “F-words” tools; the “F-words” agreements, photo collage, goal sheet, and profile. Many of the ideas for the tools came from family stakeholders’ personal experiences of working with service providers and their perceptions of how the “F-words” could be used in practice. As an integrated research team, we discussed the purpose and goals for each tool, and then with the support of CanChild students we developed draft tool templates that could be distributed to all team members for feedback. When all team members had approved the tools, they were then posted on the Knowledge Hub.
Knowledge Hub Development Process
To help with the planning and development of the Knowledge Hub, we used Levac et al’s  best practice recommendations for designing Web-based KT resources. These were based on their experiences developing and evaluating Web-based KT resources, as well as a review of the KT and instructional design literature [ ]. They identified four main recommendations: (1) develop evidence-based user-centered content; (2) tailor content to the Web-based format; (3) evaluate impact; and (4) share the results and disseminate knowledge. Each recommendation had several specific steps; the full description of the application of Levac et al’s [ ] recommendations for this study is provided in .
Description of Knowledge Hub
The purpose of the Knowledge Hub is to have a single site where people can go to learn about and share ideas for utilizing the “F-words” concepts in practice. The Web-based hub  includes tools and resources created by our research team, as well as materials that have been generously shared by stakeholders from around the world. Everything on the Knowledge Hub is freely available to share and adapt to localized practice settings. The Knowledge Hub has 6 main sections: (1) the F-words Homepage; (2) ICF Resources; (3) F-words Footprint; (4) Family & Clinician Voices; (5) F-words Tools; and (6) F-words Research Team. A full description of the Knowledge Hub is provided in .
Knowledge Hub Evaluation
A pilot study design was used to assess the usability and utility of the Knowledge Hub, and to make any necessary changes, before implementing a larger evaluation study. Usability was measured with “usefulness” questions (ie, clear purpose, user-friendly, content meaningful or relevant) and utility was measured using “use” questions (ie, impact and use intent, change in knowledge, attitude, and behavior). Usability and utility testing is a critical component to the success of KT interventions [, ]. Visitors to the Knowledge Hub were asked to review the hub and voluntarily provide feedback by completing a brief anonymous Web-based open survey through McMaster University’s LimeSurvey system. Participants were told that by completing the survey they were giving their consent to participate in the study. A survey link was posted on the Knowledge Hub, and a recruitment email and poster were distributed through CanChild’s social networks. The recruitment poster is provided in .
The survey included both closed-ended and open-ended questions. The closed-ended responses had a five-point Likert-type scale that evaluated the visitors’ prior familiarity with the “F-words”, the perceived usefulness, and reported or intended use of the Knowledge Hub. Adaptive questioning was used (ie, some questions were conditionally displayed based on the responses to previous questions) to reduce the complexity of the survey. There were 37 questions in the survey. Google analytics evaluated the reach by tracking the number of visitors to the hub over a four-month period. Descriptive statistics were used to analyze the quantitative information, and descriptive content analysis was used to identify and synthesize the key themes from the open-ended questions. Ethics approval was obtained from Hamilton Integrated Research Ethics Board (Project# 2017-0977).
Google Analytic Data (Tracking the Reach)
Over the four-month evaluation period (August-November 2017), there were over 6,800 unique visitors to the Knowledge Hub, with the number of visitors increasing each month (). This could correspond with KT strategies implemented by the research team (eg, conference presentations, educational outreach visits, monthly CanChild newsletters featuring the Knowledge Hub) and spread of the Knowledge Hub by people who liked and were sharing it within their communication channels and social networks.
The survey went live on November 3, 2017, and data were collected for 1 month. A total of 87 respondents provided information, with 63 completing the full survey and 24 partially completing the survey (ie, 1 or 2 sections), providing a completion rate of 72%. Most people visited the Knowledge Hub 1-5 times (n=63) and spent up to 45 minutes exploring the hub (n=61) prior to providing feedback. The following results were based on the survey data.
Just under half the respondents that completed the survey lived in Canada (42/87, 48%). The only other country with >10 respondents was the United States (17/87, 20%). The remainder of respondents came from 13 countries. Respondents were asked to state the perspective from which they were viewing the Knowledge Hub (eg, family member, clinician etc). Of the 87 people who completed the survey, 42 were clinicians and 30 were family members (n=20) or individuals with a disability (n=10). There was a wide distribution of perspectives with many respondents (n=36) falling into >1 stakeholder category ().
Respondents’ Familiarity with the “F-words”
The majority of people (62/87, 71%) had heard of the “F-words” prior to visiting the Knowledge Hub and either “extremely liked the ideas” (38/62, 61%) or “liked the ideas” (21/62, 34%). Of the 62 people who were familiar with the “F-words”, 43 (69%) felt confident identifying and explaining the “F-words” ideas, 37 (60%) had shared them with others, and 35 (56%) indicated that they had used or applied them in practice prior to exploring the hub. To understand how people were using or applying the “F-words,” we asked for open-ended feedback. The majority of people who provided written responses were clinicians, researchers, people with disabilities, or family members. Depending on the stakeholder group, the use of the “F-words” concepts varied. Examples of how the “F-words” concepts have been used by each stakeholder group are shown in.
Perceived Usefulness of the Knowledge Hub
To evaluate the usefulness of the Knowledge Hub, respondents were asked to rate their overall satisfaction. Of the 87 people who started the survey, 66 people completed this section. Therefore, the following data are based on these 66 responses. Overall, 86% (57/66) of respondents felt the purpose was clear, 92% (61/66) found the Knowledge Hub user-friendly, and 92% (61/66) and 94% (62/66) perceived the content to be meaningful and relevant for families and service providers, respectively. The average scores ranged from 4.23 to 4.39 out of 5 for each category ().
|Characteristic||Value, n (%)|
|Country of residencea|
|United States||17 (20)|
|United Kingdom||2 (2)|
|South Africa||2 (2)|
|No answer||5 (6)|
|Type of stakeholderb,c|
|Family memberd||20 (23)|
|Friend of someone with a disability||17 (20)|
|Person with a disabilityd||10 (11)|
|No answer||2 (2)|
|No answer||5 (6)|
|Previously aware of the “F-words”|
|No answer||6 (7)|
aIncludes countries with >1 respondent.
bIncludes stakeholder groups with >5 respondents.
cSome respondents fit into >1 group (eg, clinician and educator).
dPrimary target audience.
|Level of uptake||Sample quotes|
|Applied the F-words to their own lives (n=5)|
|Used the F-words when speaking with therapists and teachers to assist with goal-setting and planning for their child (n=2)|
|Implemented the F-words to help with goal-setting with families, to frame conversations with families, and to help guide program planning and decision making (n=15)|
|Research or education|
|Incorporated the F-words into training for students (n=3)|
|Incorporated the F-words into publications and grant writing applications (n=1)|
|Health care organization|
|The F-words are influencing organizations in items such as facility planning, departmental missions, and the development of programs (n=2)|
|Item||Strongly agree, n (%)||Agree, n (%)||Neutral, n (%)||Disagree, n (%)||Strongly disagree, n (%)||No answer, n (%)|
|The purpose is clear||32 (48)||25 (38)||5 (8)||4 (6)||0 (0)||0 (0)|
|The hub is user-friendly||19 (29)||42 (64)||2 (3)||1 (2)||0 (0)||2 (3)|
|I enjoyed exploring the Knowledge Hub||26 (39)||35 (53)||3 (5)||0||0 (0)||2 (3)|
|The content is meaningful and relevant for families||32 (48)||29 (44)||4 (6)||1 (2)||0 (0)||0 (0)|
|The content is meaningful and relevant for service providers||31 (47)||31 (47)||3 (5)||1 (2)||0 (0)||0 (0)|
Respondents were also asked to indicate which sections of the Knowledge Hub they liked and what could be improved. A total of 65 people answered this question, all of whom indicated they liked at least one section of the Knowledge Hub, 57% (37/65) indicated that they liked all sections and 45% (29/65) indicated they had no further suggestions for improvements.shows the breakdown of the items respondents liked and the possible areas for improvement.
The survey also collected open-ended feedback to gain a better understanding of what were perceived to be the best features of the Knowledge Hub (48 respondents) and which areas needed improvement (25 respondents). The best features and areas for improvement were categorized into two aspects: content and format or design of the Knowledge Hub. The key themes within these areas were then identified based on the number of responses.shows the reported best features and summarizes the reported main areas for improvement.
|Itema||Liked, n (%)||Areas for improvement, n (%)|
|Homepage||26 (40)||6 (9)|
|International Classification of Functioning, Disability and Health Resources||12 (18)||8 (12)|
|F-words Footprint||14 (22)||2 (3)|
|Family and Clinician Voices||17 (26)||4 (6)|
|F-words Tools||23 (35)||7 (11)|
|Research Team||10 (15)||3 (5)|
|All of the above||37 (57)||1 (2)|
|None of the above||0 (0)||29 (45)|
|Other||1 (2)||9 (14)|
aPeople could select >1 item.
|Overall, the videos (n=21) and “F-words” tools (n=15) were identified as the best features of the Knowledge Hub.||"My favorite part of the Hub is the F-words Tools section! As an educator, access to tools and examples from children helps me to understand how the F-words come into practice in the classroom and at home." [Educator, Canada]|
|Many people also valued the stories and examples shared by families and clinicians on what the “F-words” meant to them and how they are using the “F-words” in practice (n=9).||"The writing by families and therapists were also quite valuable in seeing how these principles are applied in many different situations. They are also very engaging to read." [Researcher, USA]|
|Format or design|
|Key design features were that the hub was easy to navigate (n=8), user-friendly (n=7), and interesting or engaging (n=7).||"It\'s simple to use and navigate, visually interesting and love the video content." [Family member, friend, researcher, Canada]|
|The hub being publically available with sharable, downloadable content (n=5).||"Sharing the information is great but also providing the tools and resources for families and providers alike is crucial to getting the word out and to helping these families." [Clinician, USA]|
|More examples of the application of the F-words and the impact. This includes more case vignettes, as well as formal research studies implementing and evaluating the F-words tools (n=7).||"I think it would be important to expand the dissemination of the six F-words by conducting studies on its application and results obtained." [Clinician-researcher, Brazil]|
|Also, extending the F-words to other populations, including teachers, young children, and increasing the diversity of representation (n=4).||"Improve the representation of diverse (SES, racial, ethnic, disabilities) families and practitioners to discuss barriers and different strategies possible in a wide lens." [Person with a disability, family member, friend, student, researcher, educator, USA]|
|Format or design|
|Overall organization (eg, clearly identifying the different sections, resources, purpose of the hub, etc) (n=8).||"The content is excellent, some of the formatting could be improved to make it more user-friendly (lots of scrolling currently and hard to orient to all the great materials with that format)." [Student, researcher, support worker, Canada]|
|The need for better navigation from the homepage (n=5).||"Better navigation. From the home page I would like a “how to use this site” section that will guide me to what I need to be looking at use - either as a parent, as a therapist, as a researcher." [Family member, Canada]|
|The Knowledge Hub...||Strongly agree, n (%)||Agree, n (%)||Neutral, n (%)||Disagree, n (%)||Strongly disagree, n (%)||No answer, n (%)|
|...increased my understanding of the F-words concepts.||27 (43)||31 (49)||2 (3)||1 (2)||0 (0)||2 (3)|
|...influenced what I think about the F-words concepts.||20 (32)||29 (46)||11 (17)||1 (2)||0 (0)||2 (3)|
|...will be useful to me.||23 (37)||32 (51)||7 (11)||0 (0)||0 (0)||1 (2)|
|...will influence the things I do when I am working with others.||22 (35)||30 (48)||5 (8)||2 (3)||1 (2)||3 (5)|
The final section of the survey explored the use or intended use of the Knowledge Hub and the “F-words” concepts. Among the people who started the survey, 72% (63/87) completed this final section. The following data are based on responses from these 63 people ().
Overall, 97% (61/63) people indicated that they either “extremely liked” (42/63, 67%) or “liked” (19/63, 30%) the “F-words” concepts, 92% (58/63) people reported that the hub increased their understanding of the “F-words”, and 78% (49/63) people reported that the hub influenced their thinking about the “F-words”. We were also interested in participants’ confidence in identifying and explaining the F-words after exploring the Knowledge Hub. Overall, 90% (57/63) people indicated that they were either “extremely confident” (19/63, 30%) or “confident” (38/63, 60%). When asked whether the Knowledge Hub would be useful to them, 83% (52/63) people reported that it would influence the way they did things when working with others.
Lastly, respondents were asked to rate the Knowledge Hub as a KT tool for sharing information with families and service providers. Overall, 90% (57/63) people rated it 4 or 5 (on the 5-point Likert scale) as a KT tool for families, 98% (60/63) people rated it as 4 or 5 as a KT tool for service providers, and 97% (58/63) people planned to share the Knowledge Hub.
Reflections on the Development Process
From the beginning, it was important to us that the Knowledge Hub be cocreated with stakeholders. While our integrated team of families and researchers led the development process, many stakeholders outside of our research team were involved. For example, we worked with clinicians and health care administrators, who we knew were applying the “F-words” to share examples of how they were using the “F-words” in their organizations. These examples then served as examples of application for other service providers.
We believe early stakeholder involvement was crucial not only to the development of a meaningful and relevant resource but also to the dissemination of knowledge regarding the Knowledge Hub. Individuals interested in the Web-based hub were more likely to share it with their own communities, thus increasing its reach and potential impact as it was spread through broad communication channels and social networks . The importance of involving stakeholders such as families and service providers in the development of the KT resources has been recognized by other children’s health researchers [ - ].
Another key feature of the Knowledge Hub was its promotion of knowledge sharing and exchange . In comparison with other Web-based KT resources such as Web-based learning modules which are difficult to change after completion, as the Knowledge Hub is organic, it can be easily adapted and can grow over time. This not only encourages people to return to the Knowledge Hub but also inspires them to get involved and contribute to the conversation (ie, become “knowledge brokers” of the “F-words”) [ , ]. Having the Knowledge Hub freely available is crucial to supporting this global dissemination and uptake.
One common barrier reported in the literature was the time and resources needed to develop and implement KT interventions [, , ]. While our research team was responsible for developing and collating the content for the Knowledge Hub, we leveraged many of CanChild’s resources (eg, the CanChild website and CanChild KT staffs’ or students’ time) to design and maintain the Knowledge Hub. Creating and collating content for the Knowledge Hub also took a lot more time than initially expected. The development process involved iterative rounds of feedback from various stakeholders. We did not follow a structured system or timeline for collecting feedback, which led to a longer process. In the future, we would recommend the use of a structured process tailored to collecting feedback from a diverse group of stakeholders [ ].
A key facilitator for this project was the use of theory and best practice guidelines to inform the KT intervention [, ]. The KTA framework [ ], the Diffusion of Innovation theory [ ], and Levac et al’s [ ] best practice guidelines for developing Web-based educational resources were all used to inform the development process. Specifically, the KTA framework [ ] provided the “big picture” and was used as the overarching guide for the KT process. Levac et al’s [ ] best practice guidelines for Web-based KT resources helped us with specific details and steps needed to design the Knowledge Hub. These guidelines were useful as they were specifically tailored to our chosen KT strategy. Lastly, the Diffusion of Innovation theory informed the design and implementation of the Knowledge Hub through consideration of the characteristics of the innovation that support adoption (ie, relative advantage, complexity, compatibility, trialability, and observability), as well as the key factors that influence innovation dissemination (ie, time, social networks, and communication channels) [ ].
The main aim of the Knowledge Hub is to inform families and service providers about the “F-words”/ICF concepts and to provide action-oriented tools to support the uptake and use of the “F-words” in practice. As such, the goal of this pilot evaluation was to evaluate the usability and utility of the Knowledge Hub. The findings from this study revealed that these self-assigned goals were attained. Overall, the respondents reported that the Knowledge Hub was informative and useful and the “F-words” tools were one of the best features of the Knowledge Hub.
In general, the hub received high ratings with regard to both its perceived usefulness and potential use. While mixed-model analyses between groups were not completed, the high ratings given by all participants implied that the Knowledge Hub was perceived to be a meaningful resource for both service providers and families. This finding was consistent with earlier research from CanChild that found that when educational materials were clearly written and user-friendly, they were useful and impactful for multiple target audiences (ie, families and service providers) [, ]. Furthermore, while more structured research is still needed to evaluate the impact of the Knowledge Hub on family and service provider behavior, people’s reported intentions to use the hub were an encouraging preliminary finding. As outlined in behavior change literature, people’s attitudes have a significant influence on whether a change will happen [ , ].
We recognize that prior to exploring the Knowledge Hub, over 70% of people who completed the survey had previously heard of the “F-words.” Of these respondents, the majority felt confident identifying and explaining the “F-words” ideas, and about half of them indicated that they had used or applied the “F-words” in practice. Despite many respondents already being familiar with the “F-words” concepts, the majority stated that the Knowledge Hub increased their understanding of the “F-words” ideas. This is an important finding as it implies that the Knowledge Hub can increase perceived knowledge even if individuals have prior familiarity with the concepts. This probably occurred because the resources provide tangible materials that move beyond simple concept familiarity. Unfortunately, due to the low response rate from people for whom the “F-words” concept was new, it is not possible to say whether the Knowledge Hub is useful across all adopter categories (ie, from the early adopters— those who are already using the “F-words”—to the late adopters— those to whom the “F-words” are new) .
Conducting a pilot evaluation of the usability and utility of the Knowledge Hub is an important step toward ensuring its overall impact and sustainability [, ]. This pilot evaluation helped us to understand what people liked about the Knowledge Hub (eg, the videos, “F-words” tools, families’ and clinicians’ voices, etc) and what changes were needed to improve it (eg, re-organizing the homepage to support navigation throughout the hub). The evaluation also helped us understand who was accessing the Knowledge Hub (ie, mostly the early adopters of the “F-words” concepts) and what was needed to broaden the applicability of the Knowledge Hub to a wider audience (eg, extending the “F-words” to other populations and conducting research on the impact of using the “F-words” tools). These findings will both inform and complement future evaluations of the Knowledge Hub. Recognizing that experimental evaluations only identify whether an intervention is effective, process evaluations such as this are recommended to understand the reasons why interventions are (or are not) effective [ , ].
Study Limitations and Future Directions
Based on the respondents’ positive feedback, we anticipate that the Knowledge Hub will be a useful resource for both families and service providers. A limitation to this work is that feedback was gained from only a small sample of the people who visited the hub during this period. It is important to remember that the majority of people who provided feedback were those who were already familiar with the “F-words” concepts and also liked the “F-words” ideas. Thus, their potential biases must be recognized.
In order to reach a broader audience, more time is needed to actively disseminate the Knowledge Hub. While the preliminary findings after a one-month evaluation were reported here, in order to overcome selection bias (ie, those who already like the F-words ideas), the evaluation will remain posted on the Knowledge Hub and further feedback will be monitored. The hope is that over time more people (including those who are not already familiar with the “F-words”) will complete the survey.
The next step is to evaluate the impact of the Knowledge Hub and “F-words” concepts at the family, clinician, and organizational levels. As active implementation strategies are useful in supporting the dissemination and uptake of educational materials, we plan to combine the Knowledge Hub intervention with tailored invitational outreach visits to local children’s treatment centers (CTCs). Once again, this is a stakeholder-driven strategy as CTCs have contacted us and expressed a need for in-person educational training on the “F-words” concepts. Based on our positive experiences of working with families and service providers to develop the Knowledge Hub, this project will continue to be informed by an iKT strategy.
Working with families and service providers, we designed a theory-informed and evidence-informed Web-based KT resource that was perceived to be relevant and meaningful to families raising children with disabilities and to service providers working in the field. To date, the Knowledge Hub has mainly reached early adopters (ie, people who like the “F-words” ideas and are seeking more information) ; therefore, to reach a wider audience (ie, the early majority), further active implementation strategies are needed.
KT is not only the doing but also the studying of the KT process and outcomes. From the evaluation of the usability and utility of the Knowledge Hub, we now have a good understanding of what was done well and what can be improved. Based on the findings from this pilot study, we intend to make minor changes to the Knowledge Hub before conducting a larger evaluation study of the impact at the family, clinician, and organizational levels. Knowledge gained from this study is transferrable to other KT initiatives involving families and service providers. We hope that the findings and lessons learned from this integrated KT project will assist others in advancing iKT science and practice in other areas of childhood disability research.
We would like to thank all families, service providers, and researchers who were involved in this project. Developing and evaluating the Knowledge Hub was a collaborative effort, and we appreciate all the individuals’ time and contributions. We would like to specially thank the KT staff and students at CanChild who helped design the Knowledge Hub on the CanChild website. Finally, we thank everyone who shared with us how they adapted and used the “F-words” in practice and who provided permission to share their stories and resources on the Knowledge Hub. JWG holds the Scotiabank Chair in Child Health Research.
Conflicts of Interest
Multimedia Appendix 1
Application of Levac et al’s (2015) recommended best practices of Web-based knowledge translation resources.PDF File (Adobe PDF File), 45KB
Multimedia Appendix 2
Description of “F-words” Knowledge Hub.PDF File (Adobe PDF File), 28KB
Multimedia Appendix 3
“F-words” Knowledge Hub evaluation recruitment poster.JPG File, 109KB
- Rosenbaum P, Gorter JW. The 'F-words' in childhood disability: I swear this is how we should think!. Child Care Health Dev 2012 Jul;38(4):457-463. [CrossRef] [Medline]
- World Health Organization. International Classification of Functioning, Disability and Health (ICF). Geneva, Switzerland: World Health Organization; 2001.
- Cross A, Rosenbaum P, Grahovac D, Kay D, Gorter JW. Knowledge mobilization to spread awareness of the 'F-words' in childhood disability: lessons from a family-researcher partnership. Child Care Health Dev 2015 Nov;41(6):947-953. [CrossRef] [Medline]
- Russell DJ, Rivard LM, Walter SD, Rosenbaum PL, Roxborough L, Cameron D, et al. Using knowledge brokers to facilitate the uptake of pediatric measurement tools into clinical practice: a before-after intervention study. Implement Sci 2010 Nov 23;5:92 [FREE Full text] [CrossRef] [Medline]
- CanChild Website Homepage. 2018 URL: https://www.canchild.ca/ [accessed 2018-02-06] [WebCite Cache]
- Russell DJ, McCauley D, Novak I, Kolehmainen N, Shikako-Thomas K, D'Costa R, et al. Developing a Knowledge Translation (KT) Strategy for a Centre of Childhood Disability Research: Description of the Process. SRC 2015 Dec 02;7(1). [CrossRef]
- Levac D, Glegg SMN, Camden C, Rivard LM, Missiuna C. Best practice recommendations for the development, implementation, and evaluation of online knowledge translation resources in rehabilitation. Phys Ther 2015 Apr;95(4):648-662 [FREE Full text] [CrossRef] [Medline]
- Camden C, Rivard L, Pollock N, Missiuna C. Knowledge to practice in developmental coordination disorder: impact of an evidence-based online module on physical therapists' self-reported knowledge, skills, and practice. Phys Occup Ther Pediatr 2015 May;35(2):195-210. [CrossRef] [Medline]
- Jeong Y, Law M, DeMatteo C, Stratford P, Kim H. Knowledge Translation from Research to Clinical Practice: Measuring Participation of Children with Disabilities. Occup Ther Health Care 2016 Oct;30(4):323-343. [CrossRef] [Medline]
- Cunningham BJ, Rosenbaum P, Hidecker MJC. Promoting consistent use of the communication function classification system (CFCS). Disabil Rehabil 2016;38(2):195-204. [CrossRef] [Medline]
- Stockley D, Beyer W, Hutchinson N, DeLugt J, Chin P, Versnel J, et al. Using Interactive Technology to Disseminate Research Findings to a Diverse Population. CJLT / RCAT 2009 Sep 03;35(1). [CrossRef]
- Mairs K, McNeil H, McLeod J, Prorok JC, Stolee P. Online strategies to facilitate health-related knowledge transfer: a systematic search and review. Health Info Libr J 2013 Dec;30(4):261-277. [CrossRef] [Medline]
- De Angelis G, Davies B, King J, McEwan J, Cavallo S, Loew L, et al. Information and Communication Technologies for the Dissemination of Clinical Practice Guidelines to Health Professionals: A Systematic Review. JMIR Med Educ 2016 Nov 30;2(2):e16 [FREE Full text] [CrossRef] [Medline]
- Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education. Med Educ 2005 Jun;39(6):561-567. [CrossRef] [Medline]
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof 2006;26(1):13-24. [CrossRef] [Medline]
- Colquhoun H, Letts L, Law M, MacDermid J, Missiuna C. A scoping review of the use of theory in studies of knowledge translation. Can J Occup Ther 2010 Dec;77(5):270-279. [CrossRef] [Medline]
- Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol 2005 Feb;58(2):107-112. [CrossRef] [Medline]
- Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG). Designing theoretically-informed implementation interventions. Implement Sci 2006 Feb 23;1:4 [FREE Full text] [CrossRef] [Medline]
- Government of Canada. Canadian Institutes of Health Research. Guide to knowledge translation planning at CIHR: Integrated and end-of-grant approaches URL: http://www.cihr-irsc.gc.ca/e/45321.html [accessed 2018-11-11] [WebCite Cache]
- Camden C, Shikako-Thomas K, Nguyen T, Graham E, Thomas A, Sprung J, et al. Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disabil Rehabil 2015;37(15):1390-1400. [CrossRef] [Medline]
- Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci 2016 Mar 17;11:38 [FREE Full text] [CrossRef] [Medline]
- F-words in Childhood Disability Knowledge Hub.: CanChild Centre for Childhood Disability Research URL: https://www.canchild.ca/en/research-in-practice/f-words-in-childhood-disability [accessed 2018-02-06] [WebCite Cache]
- Menon A, Korner-Bitensky N, Chignell M, Straus S. Usability testing of two e-learning resources: methods to maximize potential for clinician use. J Rehabil Med 2012 Apr;44(4):338-345 [FREE Full text] [CrossRef] [Medline]
- Rogers E. Diffusion of Innovations. 5th edition. New York: Free Press; 2003:9780743222099.
- Hartling L, Scott S, Pandya R, Johnson D, Bishop T, Klassen TP. Storytelling as a communication tool for health consumers: development of an intervention for parents of children with croup. Stories to communicate health information. BMC Pediatr 2010 Sep 02;10:64 [FREE Full text] [CrossRef] [Medline]
- Reid K, Hartling L, Ali S, Le A, Norris A, Scott S. Development and Usability Evaluation of an Art and Narrative-Based Knowledge Translation Tool for Parents With a Child With Pediatric Chronic Pain: Multi-Method Study. J Med Internet Res 2017 Dec 14;19(12):e412 [FREE Full text] [CrossRef] [Medline]
- Campbell-Yeo M, Dol J, Disher T, Benoit B, Chambers CT, Sheffield K, et al. The Power of a Parent's Touch: Evaluation of Reach and Impact of a Targeted Evidence-Based YouTube Video. J Perinat Neonatal Nurs 2017;31(4):341-349. [CrossRef] [Medline]
- Wittmeier KDM, Klassen TP, Sibley KM. Implementation science in pediatric health care: advances and opportunities. JAMA Pediatr 2015 Apr;169(4):307-309. [CrossRef] [Medline]
- Rosenbaum P. Changing the discourse: we all must be knowledge brokers. Dev Med Child Neurol 2016 Dec;58(12):1204 [FREE Full text] [CrossRef] [Medline]
- Glegg S, Livingstone R, Montgomery I. Facilitating interprofessional evidence-based practice in paediatric rehabilitation: development, implementation and evaluation of an online toolkit for health professionals. Disabil Rehabil 2016;38(4):391-399. [CrossRef] [Medline]
- Estabrooks CA, Thompson DS, Lovely JJE, Hofmeyer A. A guide to knowledge translation theory. J Contin Educ Health Prof 2006;26(1):25-36. [CrossRef] [Medline]
- Law M, Teplicky R, King S, King G, Kertoy M, Moning T, et al. Family-centred service: moving ideas into practice. Child Care Health Dev 2005 Nov;31(6):633-642. [CrossRef] [Medline]
- Teplicky R, Law M, Rosenbaum P, Stewart D, DeMatteo C, Rumney P. Effective rehabilitation for children and adolescents with brain injury: evaluating and disseminating the evidence. Arch Phys Med Rehabil 2005 May;86(5):924-931. [CrossRef] [Medline]
- Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes 1991 Dec;50(2):179-211. [CrossRef]
- Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals' intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement Sci 2008 Jul 16;3:36 [FREE Full text] [CrossRef] [Medline]
- Scott SD, Rotter T, Hartling L, Chambers T, Bannar-Martin KH. A protocol for a systematic review of the use of process evaluations in knowledge translation research. Syst Rev 2014 Dec 23;3:149 [FREE Full text] [CrossRef] [Medline]
- Straus S, Tetroe J, Bhattacharyya O, Zwarenstein M, Graham I. Monitoring knowledge use and evaluating outcomes. In: Straus S, Tetroe J, Graham I, editors. Knowledge Translation in Health Care: Moving from Evidence to Practice. Chichester, West Sussex, United Kingdom: Wiley, John & Sons, Incorporated; 2013:227-236.
|CTC: children’s treatment center|
|ICF: International Classification of Functioning, Disability and Health|
|iKT: integrated knowledge translation|
|KT: knowledge translation|
Edited by G Eysenbach; submitted 17.03.18; peer-reviewed by R Grace, L Shields; comments to author 03.05.18; revised version received 17.05.18; accepted 03.07.18; published 21.12.18
©Andrea Cross, Peter Rosenbaum, Danijela Grahovac, Julie Brocklehurst, Diane Kay, Sue Baptiste, Jan Willem Gorter. Originally published in JMIR Rehabilitation and Assistive Technology (http://rehab.jmir.org), 21.12.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Rehabilitation and Assistive Technology, is properly cited. The complete bibliographic information, a link to the original publication on http://rehab.jmir.org/, as well as this copyright and license information must be included.