Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 15.01.19 in Vol 6, No 1 (2019): Jan-Jun

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/11127, first published May 31, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Global Consensus From Clinicians Regarding Low Back Pain Outcome Indicators for Older Adults: Pairwise Wiki Survey Using Crowdsourcing

    1Department of Rehabilitation Sciences, Hong Kong Polytechnic University, Hong Kong, China (Hong Kong)

    2Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark

    3Rush University Medical Center, Chicago, IL, United States

    4Population Health Research Institute, Hamilton, ON, Canada

    5Department of Physiotherapy, University of Sydney, Sydney, Australia

    6Institute of Bone and Joint Research, University of Sydney, Sydney, Australia

    *these authors contributed equally

    Corresponding Author:

    Arnold YL Wong, PT, MPhil, PhD

    Department of Rehabilitation Sciences

    Hong Kong Polytechnic University

    Room ST512, 5/F, Ng Wing Hong Building

    11 Yuk Choi Road

    Hong Kong,

    China (Hong Kong)

    Phone: 852 2766 6741

    Fax:852 2330 8656

    Email:


    ABSTRACT

    Background: Low back pain (LBP) is one of the most debilitating conditions among older adults. Unfortunately, existing LBP outcome questionnaires are not adapted for specific circumstances related to old age, which may make these measures less than ideal for evaluating LBP in older adults.

    Objective: To explore the necessity of developing age-specific outcome measures, crowdsourcing was conducted to solicit opinions from clinicians globally.

    Methods: Clinicians around the world voted and/or prioritized various LBP outcome indicators for older adults on a pairwise wiki survey website. Seven seed outcome indicators were posted for voting while respondents were encouraged to suggest new indicators for others to vote/prioritize. The website was promoted on the social media of various health care professional organizations. An established algorithm calculated the mean scores of all ideas. A score >50 points means that the idea has >50% probability of beating another randomly presented indicator.

    Results: Within 42 days, 128 respondents from 6 continents cast 2466 votes and proposed 14 ideas. Indicators pertinent to improvements of physical functioning and age-related social functioning scored >50 while self-perceived reduction of LBP scored 32.

    Conclusions: This is the first crowdsourcing study to address LBP outcome indicators for older adults. The study noted that age-specific outcome indicators should be integrated into future LBP outcome measures for older adults. Future research should solicit opinions from older patients with LBP to develop age-specific back pain outcome measures that suit clinicians and patients alike.

    JMIR Rehabil Assist Technol 2019;6(1):e11127

    doi:10.2196/11127

    KEYWORDS



    Introduction

    Low back pain (LBP) is a debilitating condition [1,2] that causes functional decline in older adults [3]. The predicted percentage of adults aged 60 years and over will triple by 2050 [4], which may inevitably increase incidences of noncommunicable conditions (including musculoskeletal disorders) [5]. It has been estimated that 30% of seniors aged 65 years and over in the United States live with LBP [6].

    Since the sequelae of LBP has larger impacts on physical function and quality of life of older adults than younger individuals [7,8], it is essential to effectively treat the affected older adults. Unfortunately, the efficacy of different LBP interventions in older adults remains uncertain because many clinical trials on LBP interventions exclude older patients [9], and existing LBP outcome measures do not consider age-related physical and psychosocial changes in older adults and may not comprehensively evaluate the impact of LBP on those older adults [3,10]. Although more studies have evaluated the efficacy of various LBP interventions on older adults [11,12], there is no consensus regarding the necessity of developing age-specific outcome measures for older adults with LBP. Some clinicians believe that LBP outcome indicators for older adults should not differ from those for young adults, whereas others argue that older adults need another set of LBP outcome indicators given their comorbidities and altered psychosocial conditions [13,14]. Given the controversy, it is important to broadly solicit clinicians’ opinions on the importance of various key LBP outcome indicators to determine the necessity of developing new or adapting existing LBP outcome measures for older adults.

    Crowdsourcing is a research approach collating information and solutions from a group of people or experts using the internet in a controlled manner. Specifically, an organization presents a complex problem to a specific group of internet users who will provide solutions to the challenge or problem on a voluntary or employee-paid basis. The organizer then analyzes the findings for further applications [15]. Crowdsourced results are highly applicable to the target audience and end users because they are involved in deriving the solutions [15]. Multiple health disciplines have adopted crowdsourcing to monitor disease outbreaks, analyze gene expression data, interpret medical images, or record drug responses [16-18]. Collectively, crowdsourcing can facilitate knowledge translation and inform biomedical research [19].

    Our study aimed to use a crowdsourcing approach to identify global clinicians’ opinions regarding the relative importance of various LBP outcome questionnaire indicators for older adults.


    Methods

    Creation of a Pairwise Wiki Survey

    Our study adopted a pairwise wiki survey approach via a crowdsourcing method, which allows prioritization of ideas [20]. Briefly, a pairwise wiki survey involves a single question with multiple potential answers. Respondents contribute to the survey by (1) making pairwise comparisons between two randomly presented answers (ie, voting between two ideas) and/or (2) adding new ideas for future respondents to vote. This approach quantifies responses based on the relative priority of different answers from all respondents and integrates respondents’ new ideas for prioritization (vote up or down) using an established algorithm [20]. Unlike traditional surveys, respondents do not confine their responses to the choices offered by the researchers [21]. Therefore, influences of researchers’ preexisting knowledge or biases are minimized during data collection [20].

    A pairwise wiki survey was created on a free open-source website, All Our Ideas (www.allourideas.org), to let respondents vote on ideas about “Which outcome measures/improvements can indicate significant low back pain improvement in elderly?” [22]. A brief description of the research objective along with the research question (Figure 1) and 7 seed answer items were posted on the website for voting on June 12, 2016. The 7 seed answers were determined by a panel of clinicians with 7 to 22 years of relevant clinical experience and are as follows:

    • Able to walk independently with or without walking aids
    • Able to do grocery shopping without significant increase in pain
    • No longer requires support from caregivers
    • Able to take care of grandkids
    • Able to meet friends independently
    • Doesn’t need to see physicians/clinicians because of low back pain
    • At least a 2-point decrease in pain on visual analogue scale

    The panel comprised a physiotherapist specializing in spinal pain management, a physiotherapist specializing in geriatric rehabilitation, an orthopedic surgeon, and a geriatrician. These seed answers were aligned with the core set of outcome domains (physical functioning, pain intensity, and health-related quality of life) derived from a Delphi study for measuring and reporting nonspecific LBP in clinical trials [23]. To evaluate the relevance of age-related outcome indicators in assessing LBP improvements of older adults, an age-specific outcome indicator (ie, being able to take care of grandkids) was added as one of the 7 seed answers. Only 7 clinically relevant seed answers were included because they were used as catalysts to stimulate constructive contributions from respondents and minimize biases from the panel. Respondents were encouraged to contribute their new ideas about potential LBP outcome indicators for older adults on the website (Figure 1). The primary investigator determined the appropriateness of the ideas submitted by respondents. Respondent-contributed answers were deactivated for voting if they were duplicates of existing answers/ideas, irrelevant to the question of interest (ie, LBP outcome indicators for older adults), or comments/questions about the appropriateness of the study design, website, answers, or research objectives. This study was approved by an ethics board committee and conducted according to the Declaration of Helsinki.

    Figure 1. Research objective and research question.
    View this figure

    To advertise the survey to targeted clinicians (ie, physiotherapists, occupational therapists, chiropractors, osteopaths, physicians, nurses, physicians, gerontologists, and general practitioners), 3 strategies were adopted. First, standardized messages with the survey hyperlink were posted on the Facebook accounts of multiple health care professional organizations (Multimedia Appendix 1) identified using various key words: chiropractic, chiropractors, general practice, general practitioners, geriatric, geriatricians, gerontological, gerontology, manual therapists, manual therapy, medical, medicine, nurses, nursing, orthopaedics, orthopedics, physical therapists, physical therapy, physiotherapists, physiotherapy, osteopathic, osteopathy, occupational therapists, or occupational therapy. Briefly, the Facebook message explained that a group of researchers was conducting a survey to solicit clinicians’ opinions regarding various LBP outcome indicators for older adults with LBP and that respondents could contribute to the online survey by selecting their preferred outcome indicators or suggesting new outcome indicators (Textbox 1). Second, similar key words were used to identify various target groups (Multimedia Appendix 1) and a standardized Tweet message alongside the hashtag of these groups was used to advertise the survey (Textbox 1). A second round of the advertisement was sent to the same social media sites on July 3, 2016. Third, the primary investigator sent personal messages through Facebook messenger to invite 15 lead clinicians and clinician-scientists (orthopedic specialists, physiotherapists, chiropractors, and nurses) in Australia, Canada, Hong Kong, Denmark, Norway, and the United States to cast their votes and share the survey hyperlink on their personal Facebook pages or the Facebook pages of their respective local professional organizations. Only a small number of personal messages were sent because this pilot study mainly aimed to use social media to promote the survey.


    Textbox 1. Standardized Facebook and Twitter messages that were posted or linked to various physiotherapy, chiropractic, osteopathic, occupational therapy, medical, and gerontology professional groups.
    View this box
    Figure 2. Resulting scores of all the answer items displayed on the website.
    View this figure

    Data Analysis

    The website uses a published algorithm to estimate the chance of a given answer item in beating another randomly presented item for a randomly chosen respondent [20]. Briefly, a binomial model was chosen to estimate the probability of a win for each answer item. Assuming a uniform prior probability for a binomial variable, the resulting posterior probability to a win follows a beta distribution [24]. By multiplying the expected value of that beta distribution by 100, the resulting estimated score (ranging from 0 to 100) would represent the winning percentage of a given item. If a given item scores 0, it is expected to lose for all pairwise comparisons. Conversely, if an item scores 100, it is anticipated to always win. The resulting scores of all the answer items are displayed on the website (Figure 2).

    Additionally, raw data (ie, the number of responses of each respondent, actual responses of each respondent, time spent on each comparison, number of new ideas from each participant, and response time of each respondent) were downloaded from the website for descriptive analysis using SPSS Statistics version 20.0 (IBM Corp). The binomial confidence interval of the mean score of each answer was also calculated [25].


    Results

    Number of Respondents and Responses

    Over 42 days, 128 respondents contributed 2466 responses. During the same period, 179 visitors visited the website without casting any vote (a response rate of 41.7%). Respondents came from 60 cities in 22 countries on 6 continents (Table 1). The United States, China (Hong Kong), Australia, Canada, and Great Britain were the top 5 countries with the highest number of responses (range 239-541) and respondents (range 10-31).

    The median number of responses per respondent was 17 (range 1-142) (Figure 3). The median time spent on each comparison by the respondent was 4.7 seconds (range 0.4-314.9 seconds). Fourteen new ideas were proposed by the respondents (Table 2). Six respondents suggested 1 new idea each, 1 proposed 2, and 1 proposed 6. Nine out of 14 new ideas were proposed within the first 3 days of the survey, but the last active idea (If trunk flexion is indicated as a significant factor increasing low back pain in the first assessment, then straight leg raise would be one of the indicators) was proposed on the day 35. Three contributed ideas were not activated for voting because they were deemed to be inappropriate or duplicate. Given the respondent-contributed ideas, the number of active ideas in the survey increased from 7 to 18. Sixteen activated ideas were self-reported outcome indicators and 2 were related to physical examinations. The median number of times each activated idea was presented to respondents for comparison was 585 (range 55-686).

    Prioritization of Answers

    Nine out of 18 activated answer items scored more than 50, implying that these answers had a more than 50% chance of beating other answers in pairwise comparisons. The top 3 high-scoring ideas (able to perform 80% of the daily activities prior to the current episode of LBP, able to walk independently with or without walking aids, and able to do grocery shopping without significant increase in pain) had mean scores of 72, 69, and 66, respectively. Two of the top 5 high-scoring ideas were suggested by respondents (Figure 4). As hypothesized, outcome indicators related to pain and physical impairments yield only low scores. Specifically, the items “at least a 2-point decrease in visual analogue scale” and “actually improvement in straight leg raise (more than 20 degrees) is quite good” scored only 32 and 18, respectively.

    Table 1. Number of responses and respondents by country.
    View this table
    Figure 3. Distribution of responses per participant.
    View this figure
    Table 2. Answer items created by the researchers and respondents/users.
    View this table
    Figure 4. Rank scores of various potential low back pain outcome indicators for geriatric patients as estimated by the established algorithm on the website. LBP: low back pain; SLR: straight leg raise.
    View this figure

    Twitter Versus Facebook

    Of 128 respondents referred to our wiki survey website, 94 (73.4%) were from Facebook and 34 (26.6%) were from Twitter. Most of the respondents (78/128, 60.7%) used a cell phone or computer tablet to participate in the survey; the rest (50/128, 39.3%) completed their surveys on their computers.


    Discussion

    Principal Findings

    This is the first online crowdsourcing research to collect global clinician opinions regarding the relative importance of different LBP outcome indicators for older adults. As hypothesized, the majority of the respondents (clinicians) deemed that functional improvements were more important than improvements of pain or physical examinations. While some self-reported LBP outcome indicators identified in our study (eg, able to perform 80% of the daily activities prior to the current episode of LBP) might be true for other age groups, our respondents generally agreed that the age-specific functional outcome indicator (eg, able to take care of grandkids) was an important self-reported outcome indicator for older adults with LBP. These findings highlight that age-specific LBP outcome indicators, which have been ignored in existing self-reported LBP outcome measures, should be considered in the future development of new outcome measures for older adults with LBP.

    Interestingly, 5 out of 7 seed answers derived from a panel of health care experts were deemed to be important by respondents. In fact, seed answers contribute to 56% of the answers scoring more than 50 points (Table 2). These results indicate that many clinicians around the world agreed on using certain seed answers to be the key LBP outcome indicators for older patients. Since several respondent-contributed outcome indicators were also rated as important, our study substantiates the feasibility and value of using a pairwise wiki survey to identify LBP outcome indicators for older people with LBP.

    Participant responses were highly related to the advertising strategy. Since our study was mainly promoted on Facebook and Twitter accounts of various health care professional associations in the United States, Hong Kong, Australia, Canada, and Great Britain, greater response rates were attained from these regions. Interestingly, although our advertisements were posted on Facebook and Twitter accounts, 73.4% of the respondents were referred from Facebook, which indicates that Facebook was a more effective social media for recruiting clinician respondents in similar research than Twitter.

    It is noteworthy that the confidence interval of one LBP outcome indicator (If trunk flexion is indicated as a significant factor increasing LBP in the first assessment, then straight leg raising test could be one of the indicators) was relatively large. This was attributed to the fact that this idea was received 7 days before the completion of data collection. Since this idea was only presented 55 times to respondents for comparison, its confidence interval was wide. Although this might affect the relative ranking of this outcome indicator, it would not affect the conclusion on the top priority outcome indicators because the most important outcome indicators should have been suggested at the early stage of the survey.

    Limitations

    As with any clinical-based or survey type of research, inherent study limitations exist. Since the study did not involve older patients with LBP, our findings are limited to clinicians’ perspectives. Future research is warranted to solicit opinions from the target patient population during the process of developing a new LBP outcome measure for older adults.

    Like many internet-based surveys, the study was limited by sample representativeness [26] because it did not collect participants’ detailed demographic information (eg, age, gender, years of education and clinical experience, health care disciplines). However, our respondents were highly likely to be clinicians because the survey was (1) not searchable on common search engines (eg, Google) unless the exact survey Web address was used, (2) only openly advertised on the Facebook and Twitter accounts of relevant professional bodies, and (3) promoted by personal emails sent to clinicians and clinician-scientists. This notion was further corroborated by the fact that the respondent-contributed ideas and voting results demonstrated high face validity to the research topic from the clinicians’ perspective.

    The response rate of the study was 41.7%. In comparison, the response rate for Delphi studies that evaluated core outcome sets for LBP were between 45% and 52% [23]. This slight discrepancy might be attributed to the recruitment methods (open advertisements on social media vs personal invitations). Previous studies have found that response rates of internet surveys for clinicians are usually lower than traditional paper surveys [27-29]. While multiple reasons may explain the low response rate among clinicians (eg, lack of time, perceived low priority of surveys, and concerns about confidentiality) [29], response rates can be improved by sending multiple reminders or personalized letters [29,30]. Future studies should adopt multiple strategies (eg, incentives [31], personalized invitations [32], multiple reminders [30] and advertisements [33], or endorsements from professional associations [34]) to improve response rates and total number of respondents.

    Since traditional surveys require researchers to determine all the details (eg, questions, orders of questions, and multiple plausible answers) prior to data collection, this top-down approach may introduce investigator biases and limit the knowledge that can be learned from respondents [20]. Conversely, our pairwise wiki survey used an ongoing collaborative approach to encourage respondents to create knowledge that was not anticipated by the researchers. Similar to a focus group that allows participants to react to others’ responses [35], user-contributed ideas collected from the wiki survey were continuously evaluated by future respondents. The success of this bottom-up interactive approach is reflected from our findings that respondents from all continents (except Antarctica) contributed 2 folds of new LBP outcome indicators within a short period of time and some of the proposed indicators were ranked as highly relevant LBP outcome indicators for older adults.

    Our survey collected information based on the respondents’ eagerness to participate. While some respondents cast a single vote, others contributed heavily to the voting and/or new idea suggestions (Figure 3) [36]. Unlike traditional surveys that prohibit high contributors from answering extra questions and discard incomplete questionnaires from data analysis, a pairwise wiki survey collects as much or as little information as the respondent is willing to offer. Since wiki surveys value contributions from all respondents equally regardless of their time or effort spent on answering, wiki surveys may solicit more useful information from respondents than traditional surveys [20].

    Conclusions

    Our study reveals a novel method for soliciting opinions from clinicians around the globe during the process of developing a new clinical outcome measure. Traditionally, the development of a new self-administered clinical outcome questionnaire involves a process of literature review, conduction of multiple focus groups or meetings among content experts (eg, clinicians, patients, scholars) to determine relevant items and/or domains in a questionnaire, and further modifications of items after pilot testing on target populations [37]. A pairwise wiki survey can be implemented as a low-cost adjunct survey tool to solicit ideas from a large population of clinicians or patients globally following the initial draft of items pooled from a panel of content experts. The survey results not only can broaden the perspectives to inform further panel discussions but allow rapid preliminary feedback from target users. However, further studies are warranted to evaluate the effectiveness of such an approach in improving the psychometric properties of the resulting questionnaires (eg, whether the inclusion of crowdsourcing-identified items would improve the internal consistency or responsiveness of questionnaires).

    While our approach has revealed the relative importance of different LBP outcome indicators perceived by clinicians, the respondents’ rationales for choosing or prioritizing their answers remains unclear. Future qualitative research (eg, interviews or focus groups) should investigate clinician reasons for prioritizing various LBP outcome indicators and solicit information from older adults regarding their perceived important LBP outcome indicators. Collectively, our findings can be incorporated with patient and expert opinions obtained from qualitative and/or Delphi research to develop a new outcome measure for geriatric patients with LBP. This study has laid the foundation for developing better outcome measures for older patients with LBP. Such knowledge has the potential to ultimately contribute to better clinical management or treatment algorithms for older adults with LBP.

    Overall, this is the first global crowdsourcing study to address LBP outcome questionnaire indicators for older adults. The study found that clinicians deemed functional improvements more important LBP outcome indicators for older adults with LBP than pain reduction or improvements of physical examinations. Clinicians generally perceive age-specific social functioning as an important outcome assessment domain for older adults with LBP. While further studies are warranted to compare our findings with the opinions obtained from older adults with LBP and/or leading spine experts, our study has laid the foundation for developing better outcome measures for older adults with LBP. In addition, this proof-of-concept study has also provided a framework to illustrate that global crowdsourcing approaches in spine research are viable and achievable, hopefully providing impetus for other investigators to adopt such an approach for future spine research.

    Acknowledgments

    This work was supported by the Hong Kong Polytechnic University start-up fund (grant number 1-ZE4G).

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    List of professional organizations on Facebook and Twitter selected to advertise the project.

    PDF File (Adobe PDF File), 34KB

    References

    1. Fourney DR, Andersson G, Arnold PM, Dettori J, Cahana A, Fehlings MG, et al. Chronic low back pain: a heterogeneous condition with challenges for an evidence-based approach. Spine (Phila Pa 1976) 2011 Oct 01;36(21 Suppl):S1-S9. [CrossRef] [Medline]
    2. GBD 2015 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 315 diseases and injuries and healthy life expectancy (HALE), 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 2016 Dec 08;388(10053):1603-1658 [FREE Full text] [CrossRef] [Medline]
    3. Wong A, Samartzis D. Low back pain in older adults: the need for specific outcome and psychometric tools. J Pain Res 2016;9:989-991 [FREE Full text] [CrossRef] [Medline]
    4. Department of Economic and Social Affairs, Population Division, United Nations. World Population Ageing 2009   URL: http://www.un.org/esa/population/publications/WPA2009/WPA2009_WorkingPaper.pdf [accessed 2018-11-04] [WebCite Cache]
    5. Hunter DJ, Reddy KS. Noncommunicable diseases. N Engl J Med 2013 Oct 03;369(14):1336-1343. [CrossRef] [Medline]
    6. Patel KV, Guralnik JM, Dansie EJ, Turk DC. Prevalence and impact of pain among older adults in the United States: findings from the 2011 National Health and Aging Trends Study. Pain 2013 Dec;154(12):2649-2657 [FREE Full text] [CrossRef] [Medline]
    7. Hicks G, Gaines J, Shardell M, Simonsick E. Associations of back and leg pain with health status and functional capacity of older adults: findings from the retirement community back pain study. Arthritis Rheum 2008 Sep 15;59(9):1306-1313 [FREE Full text] [CrossRef] [Medline]
    8. Wong AYL, Forss KS, Jakobsson J, Schoeb V, Kumlien C, Borglin G. Older adult's experience of chronic low back pain and its implications on their daily life: study protocol of a systematic review of qualitative research. Syst Rev 2018 May 24;7(1):81 [FREE Full text] [CrossRef] [Medline]
    9. Paeck T, Ferreira M, Sun C, Lin C, Tiedemann A, Maher C. Are older adults missing from low back pain clinical trials? A systematic review and meta-analysis. Arthritis Care Res (Hoboken) 2014 Aug;66(8):1220-1226 [FREE Full text] [CrossRef] [Medline]
    10. Wong AY, Karppinen J, Samartzis D. Low back pain in older adults: risk factors, management options and future directions. Scoliosis Spinal Disord 2017;12:14 [FREE Full text] [CrossRef] [Medline]
    11. Leonhardt C, Kuss K, Becker A, Basler H, de Jong J, Flatau B, et al. Graded exposure for chronic low back pain in older adults: a pilot study. J Geriatr Phys Ther 2017;40(1):51-59. [CrossRef] [Medline]
    12. Morone NE, Greco CM, Moore CG, Rollman BL, Lane B, Morrow LA, et al. A mind-body program for older adults with chronic low back pain: a randomized clinical trial. JAMA Intern Med 2016 Mar;176(3):329-337. [CrossRef] [Medline]
    13. Heine C, Browning CJ. Communication and psychosocial consequences of sensory loss in older adults: overview and rehabilitation directions. Disabil Rehabil 2002 Oct 15;24(15):763-773. [CrossRef] [Medline]
    14. Stubbs B, Schofield P, Binnekade T, Patchay S, Sepehry A, Eggermont L. Pain is associated with recurrent falls in community-dwelling older adults: evidence from a systematic review and meta-analysis. Pain Med 2014 Jul;15(7):1115-1128. [CrossRef] [Medline]
    15. Brabham D, Ribisl K, Kirchner T, Bernhardt J. Crowdsourcing applications for public health. Am J Prev Med 2014 Feb;46(2):179-187. [CrossRef] [Medline]
    16. Schootman M, Nelson EJ, Werner K, Shacham E, Elliott M, Ratnapradipa K, et al. Emerging technologies to measure neighborhood conditions in public health: implications for interventions and next steps. Int J Health Geogr 2016 Dec 23;15(1):20 [FREE Full text] [CrossRef] [Medline]
    17. Shah N, Guo Y, Wendelsdorf KV, Lu Y, Sparks R, Tsang JS. A crowdsourcing approach for reusing and meta-analyzing gene expression data. Nat Biotechnol 2016 Dec;34(8):803-806. [CrossRef] [Medline]
    18. Swan M. Crowdsourced health research studies: an important emerging complement to clinical trials in the public health research ecosystem. J Med Internet Res 2012;14(2):e46 [FREE Full text] [CrossRef] [Medline]
    19. Budge EJ, Tsoti SM, Howgate DJ, Sivakumar S, Jalali M. Collective intelligence for translational medicine: crowdsourcing insights and innovation from an interdisciplinary biomedical research community. Ann Med 2015;47(7):570-575. [CrossRef] [Medline]
    20. Salganik MJ, Levy KEC. Wiki surveys: open and quantifiable social data collection. PLoS One 2015;10(5):e0123483 [FREE Full text] [CrossRef] [Medline]
    21. Krosnick JA. Survey research. Annu Rev Psychol 1999 Feb;50:537-567. [CrossRef] [Medline]
    22. Wong A. Which outcome measures/improvements can indicate significant low back pain improvement in elderly.   URL: http://www.allourideas.org/olderpeoplewithlowbackpain [accessed 2018-10-18] [WebCite Cache]
    23. Chiarotto A, Deyo RA, Terwee CB, Boers M, Buchbinder R, Corbin TP, et al. Core outcome domains for clinical trials in non-specific low back pain. Eur Spine J 2015 Jun;24(6):1127-1142. [CrossRef] [Medline]
    24. Hoff P. A First Course in Bayesian Statistical Methods. 2nd Edition. New York: Springer Science & Business Media; 2009.
    25. Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934 Dec;26(4):404. [CrossRef]
    26. Braithwaite D, Emery J, De Lusignan S, Sutton S. Using the Internet to conduct surveys of health professionals: a valid alternative? Fam Pract 2003 Oct;20(5):545-551 [FREE Full text] [Medline]
    27. Hollowell CM, Patel RV, Bales GT, Gerber GS. Internet and postal survey of endourologic practice patterns among American urologists. J Urol 2000 Jun;163(6):1779-1782. [Medline]
    28. Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, et al. Internet versus mailed questionnaires: a randomized comparison (2). J Med Internet Res 2004 Sep 24;6(3):e30 [FREE Full text] [CrossRef] [Medline]
    29. VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof 2007 Dec;30(4):303-321. [CrossRef] [Medline]
    30. Fischbacher C, Chappel D, Edwards R, Summerton N. Health surveys via the Internet: quick and dirty or rapid and robust? J R Soc Med 2000 Jul;93(7):356-359 [FREE Full text] [Medline]
    31. Bosnjak M, Tuten TL. Prepaid and promised incentives in web surveys. Soc Sci Comput Rev 2016 Aug 18;21(2):208-217. [CrossRef]
    32. Heerwegh D, Loosveldt G. Personalizing e-mail contacts: its influence on web survey response rate and social desirability response bias. Int J Public Opin Res 2007 Mar 04;19(2):258-268. [CrossRef]
    33. Klofstad CA, Boulianne S, Basson D. Matching the message to the medium. Soc Sci Comput Rev 2007 Dec 18;26(4):498-509. [CrossRef]
    34. Asch DA, Christakis NA. Different response rates in a trial of two envelop styles in mail survey research. Epidemiology 1994 May;5(3):364-365. [Medline]
    35. Merton R. The focused interview and focus groups: continuities and discontinuities. Pub Opin Q 1987 Jan;51(4):550-556. [CrossRef]
    36. Anderson C. The Long Tail: Why the Future of Business Is Selling Less of More. New York: Hyerion; 2006.
    37. Eccleston C, Jordan A, McCracken L, Sleed M, Connell H, Clinch J. The Bath Adolescent Pain Questionnaire (BAPQ): development and preliminary psychometric evaluation of an instrument to assess the impact of chronic pain on adolescents. Pain 2005 Nov;118(1-2):263-270. [CrossRef] [Medline]


    Abbreviations

    LBP: low back pain


    Edited by G Eysenbach; submitted 31.05.18; peer-reviewed by M Alshehri; accepted 09.10.18; published 15.01.19

    ©Arnold YL Wong, Henrik H Lauridsen, Dino Samartzis, Luciana Macedo, Paulo H Ferreira, Manuela L Ferreira. Originally published in JMIR Rehabilitation and Assistive Technology (http://rehab.jmir.org), 15.01.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Rehabilitation and Assistive Technology, is properly cited. The complete bibliographic information, a link to the original publication on http://rehab.jmir.org/, as well as this copyright and license information must be included.