What works to recruit general practices to trials? A rapid review

Eligibility criteria

Types of interventions. Eligible studies were those that reported or tested any strategy to recruit general practices to participate in a randomised controlled trial (RCT). Examples of strategies could include invitation letters, endorsement, and financial incentives for participating practices. Studies reporting strategies to recruit practices by recruiting physicians were also included.

Types of studies. Eligible studies included any RCT, be they clinical trials or implementation trials (trials of implementation strategies)⁶. Both full scale RCTs and pilot RCTs²⁹ were eligible for inclusion. Observational, cross-sectional, and qualitative studies were excluded. Studies which only reported strategies to recruit patients through general practices were excluded, as were strategies to recruit primary care physicians only.

Only studies that mentioned recruitment in the title or abstract were eligible for inclusion. Both studies that specifically focused on recruitment within the trial and studies that simply reported their strategy as part of the description of the main trial (but recruitment was not the focus of the study) were eligible. Studies were not excluded on the basis of the design used to evaluate recruitment strategies; this could be qualitative or quantitative.

Hypothetical trials were excluded, i.e., studies that ask potential participants whether they would take part in a trial if it was run, but the trial does not exist. Studies focusing on recruitment of healthcare organisations other than general practices, were excluded.

Types of articles. Only peer-reviewed papers in academic journals were considered eligible. Protocol papers and conference abstracts were excluded. Selection of relevant papers was restricted to English-language publications only.

Types of outcomes. Primary outcomes were the proportion of eligible practices recruited and the time taken to recruit practices. Secondary outcomes included author reflections on the effectiveness of the recruitment strategy and whether it was successful or not, and the cost of the strategy. Studies had to report one of the primary outcomes to be included in the review. The full inclusion and exclusion criteria are summarised below in Table 1.

Information sources

The following electronic databases were searched for relevant studies: PubMed (1946 to present), Embase (1947 to present), and Cochrane Central Library. All searches were conducted on the 22^nd of June 2021. The search strategies used a combination of text words and relevant indexing, related to trials, recruitment, and general practices. The full search strategy is provided as extended data³⁰. Reference lists of included studies were also screened.

Study selection

Articles were retrieved from databases and imported to Mendeley reference management software for de-duplication. Articles were imported into Rayyan for screening. Approaches to rapid reviews can include using one person for screening with verification of a subset of records by another³¹. Titles and abstracts of identified articles were screened by one reviewer (DB) with independent screening of 25% of the articles by a second reviewer (FR). Each full text was screened independently by two reviewers (DB and FR). Disagreements were resolved through consensus or referral to a third reviewer (SMH).

Data extraction and quality assessment

Using a standard form, DB extracted data on the study title, authors, the aim of the study, the year of publication, the country in which the study was carried out, the nature of the population, the nature of the study setting and the study design used to test the recruitment strategy (if applicable), details of the strategy and its effectiveness, the profile of recruited practices if provided (e.g., size, location, staff, and other descriptors used by the study authors), and any author reflections or descriptions of the strategy impact, and/or recruitment of ‘hard to reach’ practices. We did not apply a standard definition of ‘hard to reach’. Practices were considered ‘hard to reach’ if defined as such by the author. A second reviewer extracted the data in duplicate (FR). Authors were contacted for additional data if required. Any disagreements were discussed and resolved via referral to a third reviewer (SMH) to achieve consensus.

We did not undertake a quality assessment since the aim was to characterise practice recruitment strategies and describe the outcomes of those strategies.

Data synthesis

A narrative synthesis approach was used to combine and summarise the findings from included studies. Given the wide range of strategies, we summarised the findings from the five most common strategies. Before completing the review, we expected that the results would not be standardised in a way that permitted meaningful pooling to undertake a meta-analysis. Furthermore, given the short timeline within which to conduct the review (June to September 2021), we did not anticipate it would be feasible to combine quantitative results in a meta-analysis. Strategies were organised into categories through discussion between DB and FR. The categories were informed by preliminary searches of the literature, and existing studies of recruitment^2,22. We also categorised the overarching recruitment approach. Recruitment approaches which used multiple strategies in one phase were referred to as ‘multiple strategy approach’ whereas those which staggered one or more strategies were referred to as ‘single staggered approach’. Where possible, the strategy types were compared in terms of the profile of practices recruited to explore potential links/patterns between the strategy and participant profile.

Study overview

From 9,162 articles identified from three databases, 112 full texts were screened for eligibility based on inclusion and exclusion criteria, of which 18 were included (Figure 1). One additional study was identified from reference lists. Therefore, a total of 19 studies were included in the review, predominantly from the US (n=6; 31.6%)^32–37 and Australia (n=6; 31.6%)^{11,14,23,38–40}, with other regions in Europe (Ireland: n=1; 5.3%)¹⁶ and Germany: n=1; 5.3%)⁴¹, the UK (n=4; 21.1%)^42–44 and England and Wales (1; 5.3%)⁴⁵ also represented. Most studies were published after 2010 (n=12, 63.2%).

Figure 1. PRISMA flow diagram.

Most studies (n=18) did not formally evaluate the effect of recruitment strategies, these studies descriptively reported the outcome of the recruitment strategy or strategies. One study by Lee et al.²³ evaluated recruitment strategies through a two-arm RCT embedded in a clinical trial.

Recruitment strategies

Overall, 13 studies (66.7%)^{13,14,16,32,35,36,38,40,41–45} used one or more strategy staggered across different phases to reach a certain group of practices and did not distinguish the outcome of individual strategies within the approach (‘single arm, staggered approach’). The remaining studies used multiple recruitment approaches in one phase (‘multiple strategy approach’) comprising one or more strategies, sometimes comparing the outcomes of each approach separately³⁴. Where possible, the strategy types were compared in terms of practice profile recruited to infer potential associations between the strategy and participant profile.

The most common strategies used were:

The most common combination was personalised letter and phone calls (n=5, 26.3%) or in person visits and phone calls (n=3, 15.8%) or a combination of all three (n=2, 10.5%). The least common strategies reported (only used by one study) were: mass distribution of recruitment materials at professional meetings or conferences, magazine editorials, flyers distributed via patients, project webpage, clinical audit activity, informational videos, and information cards. The full list of strategies is provided in Table 2.

In-person meetings

Among the 12 studies which included in-person visits or meetings at practices as a recruitment strategy, meetings were typically conducted by member(s) of the research team^36,39 or the team and Principal Investigator (PI)^32,40. Some specified that a clinical member (e.g., research nurse or GP researcher) of the team conducted the visits^16,38,41 while others did not provide this detail^{11,13,33,34,37}. Four studies involved conducting in-person meetings with practices with which some of which the team already had links^32,36,39,41.

In terms of timing, most visits were made to practices which expressed an interest after initial letters/emails of invitation or cold calls^{13,16,36–39,41}, presentations at conferences³⁹ or after a webinar delivered to practices within a research network after initial outreach by the medical director³². Ellis et al., made visits to practice groups after obtaining permission from medical directors of a regional network of primary care practices. Some studies provided more detail on the format and content of these visits^11,16,37,38 (Table 3).

Phone calls

Most of the nine studies which used phone calls provided no further detail on what the call involved. In some studies, the phone call was the initial¹⁶ or primary⁴² contact mode used, in others, phone calls were part of a suite of strategies^34,35,37, or used in a final round of recruitment, to provide a ‘boost’ after letter/email invitations^23,44,45.

Two studies explicitly mentioned making “cold calls” to practice managers^16,33 identifying practices within a certain radius of the study region, from telephone books, medical society' membership list and provider directories³⁴. Randomly ordered lists of practices were used to confirm practice eligibility and ascertain their initial interest¹⁶ Leathem et al. provided more detail on the content of the calls, that they included a brief explanation and mention of the financial resource, and that they avoided making these calls at especially busy times such as Monday mornings and Friday afternoons and asked reception staff for appropriate times to speak to a GP or practice manager, not wanting to use time slots reserved for patients.

Personalised mails or letters

Seven studies used emails^{11,33–35,39,43,45}, ten used letters^{11,16,23,34,35,37,38,42,44,45} and three used a combination of both^11,35,45. Four studies described issuing reminders using letters or emails^33,35,43,45. No definition of what was considered a personalised or personal email or letter was provided by the studies which used these strategies but based on the study context this was taken to be emails/letters which were addressed to a specific named person in the practice as opposed to the practice more generally. For example, one study reported using personalised emails along with generic email ‘blasts’ to mailing lists³³.

Five of the thirteen studies emailed or mailed practices within an existing network^11,33,43, or used existing contacts^32,34,41, sometimes relying on individuals within those networks to circulate the emails^34,43 Three described using opinion leaders³⁴, or local clinicians or organisations³⁸ endorsing the study by sending the letters^34,38, or using the letterhead of the local sponsoring organization, along with signature from a study physician³⁷. Where a network or existing contacts were not used, different groups were targeted: GP academics¹¹, GP attendees of a professional forum¹¹; practices in an existing network database supplemented with publicly available addresses from business directories^11,37.

Four of thirteen studies described the content of the email or letter, including key features of the trial^23,45, expected requirements^16,23,45 and projected workload¹⁶, and financial and CPD compensations for participation²³, role of the sponsor³⁸, rationale for the study^38,45, and information about the study funder^38,45.

Where a network or existing contacts were not used, different ‘pools’ or groups were targeted: GP academics¹¹, GP attendees of an aged care forum¹¹; practices in an existing network database supplemented with publicly available addresses from business directories^11,37.

Financial incentives

Nine studies mentioned using a financial incentive as a recruitment strategy^{13,16,23,32,33,35,39,41,46}, seven specifying the scale of the^13,16,32,33 practice fee ranging from $4,500³³ to €1,000/£700¹⁶) or fee per patient (€100 per completely documented patient⁴¹, $50AUD for eligible participant.^23,39).

Existing networks and links

Six studies targeted practices with which they had existing links or had involved in a previous study^{33,34,36,39,41,46}, six targeted practices within an existing practice or research network^{11,14,32,33,35,43}. One did both, targeting practices in an existing primary care network, but also starting with sites that had engaged in previous projects to minimize the number of “cold calls”³³.

Quantitative

Outcomes reported were yield as a proportion of the total practices targeted (n= 18, 94.7%)^{11,16,23,32–39,41–46}, yield as a proportion of practices who expressed an interest (n=4, 21.1%)^34,36,38,44, time taken for recruitment (n=6, 31.6%)^{11,16,35,37,42,45} and cost (n=3, 15.8%)^23,34,37.

Yield. The reported yield (% of practices recruited of those targeted) ranged from 0.40%³⁴ to 83%^44,46, with an average response of 46.8% (SD = 24.8). Ellis et al.³⁴ compared 10 different strategies; reporting the lowest yield (0.40%) cumulatively from six opt-in marketing tactics [i.e., mass advertising, conference distribution, mass fax, direct mail, opinion leader email, in-person presentation] (n=53/13,290) compared to 51% (n=90/176) from a survey-based strategy. This involved adding survey items to gauge interest in the study into an annual network survey of primary care providers, and then following up with practices who indicated an interest. Of the six opt out strategies, they reported mass advertising (published article and medical society inserts), an opinion leader email, and medical conference distribution were the least effective strategies.

Studies that reported recruitment rates over 80%^36,44,46, aimed to recruit smaller numbers of practices. They issued invitation letters with a follow-up phone call to non-responders⁴⁴, or conduced presentations by the PI and study coordinator⁴⁶ to, or in-person meetings with³⁶, practices with an existing affiliation with the University or research team.

Time taken to recruit

Six studies reported the time taken to recruit practices^{11,16,35,37,42,45} which ranged from 12 months (recruitment target: 48 practices)¹⁶ to 1.5 months (recruitment target: 27 practices)¹¹. One study reported the number of contacts⁴⁵; reporting the total number of times practices were contacted from the initial invitation (post or email or phone call) to randomisation as 6.8 (SD = 3.5), and the mean number of contacts required to gain an expression of interest (EOI) from the initial postal contact invitation to randomisation as 3.01 (SD = 1.6). More detail on recruitment time is provided in Table 4.

Cost

Three studies reported the cost of the recruitment strategies^23,34,37 which ranged from $6,471.69 to $41,340 for the postcard teaser³⁴ to $3,834 for a direct mail to physicians³⁶. Comparisons were not valuable given the variation in approach and different health systems.

Comparison

Four studies^11,23,34,37, narratively compared the outcomes of different recruitment approaches but did not use a formal evaluation design. Two compared the response rate (RR) (i.e., % responded to the approach)^23,37 one compared the recruitment yield (RY) (i.e., % recruited by using the approach), and one reported both RR and RY³⁴.

Lee et al.²³ compared the response (any response to the mailout, positive or negative) to a standard mailout approach (personalised invitation letters followed by a max of three phone calls) to a standard approach preceded by a postcard teaser campaign involving two postcards with different short phrases, one without affiliations, one with logos and affiliations sent to practices. They reported a RR of 5.9% (n = 11/186) in the Teaser Campaign group and 7.5% (n = 14/186) in the Standard Mail group, reporting the former did not increase the odds of a response to the subsequent letter (OR = 1.18, CI = 0.75–1.85, p = 0.49).

Tan et al.¹¹ used eight approaches reporting the recruitment yield of each approach: mailout to GPs in network database supplemented by publicly available business directories (RY = 0.7% , 9/1322), an online survey (RY =0.4% , 6/1400,), presentations at practices (RY = 28%, 5/18), word of mouth within professional network (RY = 30%, 3/10 professionals making informal contact were recruited, but the total contacted is unknown), medication management workshop (RY = 14.2% , 2/14), University Department of GP presentation (RY = 100%, 1/1 of those making informal contact, total attending = 30-50), contact with the practice via a third party to relay trial information and promote the trial while offering a clinical audit activity (a quality improvement process for continuing GP professional development) (RY = 100%, 1/1 making informal contact were recruited, but total contacted is unknown), and newsletters and email invites (RY = 0%, total contacted/reached is unknown).

Ellis et al.³⁴ reported a RR of 0.4% (53/13290), to the six opt-in marketing strategies they employed: mass advertising (medical society inserts RR: 0.04%; 2/5350; published article RR: 0%; 0/3500)), conference (RR: 0.94% (1/106)), mass fax distribution (RR:1.13%; 44/3882), minority provider (RR:1.25%; 4/319), opinion leader (RR:1.02%; 1/98), in-person provider presentation (RR: 2.86%; 1/35),

About a third (RR:34%; 18/53) of responders to the marketing strategies (1-6) were recruited to participate in the study. The approaches that recruited the most practices per effort and the most cost-effective approaches were in-person meetings (RR:41.7%; 5/12), followed by building on previous relationships (RR:33.3%; 9/27) and borrowing from established networks (RR:10.8%; 19/176).

Finally, McBride et al.³⁷ used three approaches, direct mail to physicians with return postcards (RR = 3.6%, 90/2485), recruitment via physicians at managed care organisations using the organisation letterhead (RR =25.5%, 11/43), and direct mail to practice leaders (RR =61.3%, 27/44).

Profile of recruited practices

Few studies provided a detailed profile description, and where they did, the characteristics reported varied. Seven studies^{11,13,16,32,34,38,42} reported the profile of practices which had been recruited. Based on the limited detail available, smaller proportions (<50%) of recruited practices appeared to be single-handed (ranging from 16% to 33% of recruited practices)^16,34,38,42, practices with a predominantly female GP workforce (34%)³⁴, and based in urban areas (9%³⁸ and 22%³²).

The percentage of single-handed practices recruited, compared across five studies (using different strategies) varied: 16%⁴², 18%¹¹, 19%⁴⁶, 28%³⁸, and 33%¹⁶. Tan et al. reflected that medium-sized practices (69% of practices recruited) were easier to recruit quickly than large practices, attributing this to the availability of a key person who could act as a facilitator (e.g. practice manager or nurse).

Two studies compared the profile of recruited practices to practices which were not recruited^13,42 and neither reported significant differences in practice profiles. Down et al., who provided the most detailed comparison⁴² reported weak associations with the staff gender proportions in the practices; recruited practices had a lower average proportion of male GPs, and were less likely to have all male GPs. However, overall, they reported there were no ‘convincing’ differences in staffing (number of GPs, % UK trained GPs, proportion of single-handed practices, proportion with only non-UK trained GPs) and list sizes, between practices who were recruited and those who were not.

Qualitative

Five studies also included qualitative findings related to recruitment efforts^{11,13,33–35}, including weekly debriefing notes from meetings between research staff and recruiters³³ or qualitative data from notes³⁵, recruiters’ online diary entries³³, administrative data³⁵, communication and recruitment material³⁵, a survey³⁴ and semi-structured interviews with providers/practice staff^13,33. Though not a primary aim of this review, of note, these studies reflected on barriers (out-of-date information about the practice³³; lack of engagement with recruiters within the primary care network³³, competing practice priorities³³ and lost paperwork³⁵) and facilitators (perceived importance of the research topic^13,34, and a general interest in research^13,34) of recruitment.

Two authors reflected in more detail on the perceived pros and cons or challenges associated with deploying a strategy. One felt that although they were building on existing relationship with practices by going through a network, supplementary recruitment strategies were still often required to engage sites, including additional personal calls and site visits³³. Tan et al.¹¹ felt that more successful strategies involved direct contact with GPs (individualised invitations, online GP surveys, and face-to-face presentations); in particular in person practice visits, despite costs, were key, providing time to build rapport with GPs and other practice staff, and from their perspective allowed them to engage practices who might not have otherwise been recruited. They felt that providing information by phone, “passing the message” by GPs, practice nurses or staff, or leaving written material for GPs was not a fruitful strategy. They reported GPs and practice staff being “overwhelmed” by emails and post.

Summary

This rapid review of strategies used to recruit primary care practices to trials yielded 19 studies. The most common strategies included in-person meetings/visits by members of the research team with phone calls to practices, financial incentives, personalised emails, or letters (as opposed to an email ‘blast’ or generic letters) and targeting practices which took part in previous studies or with which the team had existing links or were part of an existing practice or research network. Most studies used a single recruitment approach, be it using one strategy or layering different strategies. Six studies used multiple recruitment approaches at the same time, with three comparing the outcomes of the different approaches; suggesting direct mail to practice leaders compared favourably to direct mail to physicians³⁷ a postcard teaser did not have a significant impact on recruitment compared to standard mail out²³, and a combination of both opt-in and opt-out approaches yielded the most practices recruited per effort and proved to be the most cost effective³⁴. Studies reporting the highest recruitment yield (>80% of practices contacted were recruited)^36,44,46 used invitation letters with a follow-up phone call to non-responders⁴⁴, presentations or in-person meetings with practices which had an existing affiliation with the University or research team^36,46.

Implications

Two factors made it challenging to draw conclusions about the most effective recruitment strategies from the included studies. First, the level of detail provided on recruitment strategies varied across all studies. All studies bar two specifically focused on recruitment within the trial (i.e., specific aim to describe or evaluate recruitment). These two studies just reported their strategy as part of the description of the main trial^41,43. Often studies provided very limited detail on the types of strategy and the yield. The CONSORT checklist includes reporting recruitment and follow-up period (items 14a and b)⁴⁷ and additional guidance on the reporting of embedded methodological studies (to test interventions to improve recruitment to trials) and highlights the need for sufficient details (how, where and when) to allow replication of recruitment interventions⁴⁷. The success of practice recruitment approaches has important implications for trial conduct, including the length of time engaged in recruitment and cost, and the types of practices and subsequently patients included in the trial.

The second factor making it challenging to draw conclusions from this review was that only six studies^{11,23,33,34,37,39} directly compared strategies and reported the outcomes of each strategy. There was limited information available to determine the relative strengths of a strategy. Only one study tested the effectiveness of strategies¹⁶. While there has been a drive for trialists to embed evaluations of recruitment strategies within their trials⁴⁸, this work has largely focused on strategies to recruit patients rather than practices²². A systematic review of studies to date indicates that effective strategies for patients include follow up phone calls after non-response to mailed invitation, and informing people what they will be receiving in the trial. How to optimise recruitment strategies to target general practices is unknown. For example, some studies used endorsement as a strategy, using clinicians at different points to communicate with practices. It would be helpful to understand whether certain messengers or messages are more effective than others.

With a lack of standardised reporting of practice recruitment strategies and their effectiveness, knowledge of what works (and does not work) is not shared across studies. Capturing this type of information is increasingly important. The number of trials based in general practice is growing both internationally^49,50 and in Ireland⁵¹; with the establishment of the Primary Care Network Ireland, there is potentially a wealth of information among teams in Ireland about what works well in terms of recruitment, yet this is not systematically captured or shared.

In spite of the limitations of poorer reporting and the lack of studies of strategy effectiveness, there are three key lessons arising from this review. First, while there appears to be some potential benefit from capitalising on existing relationships and networks, it is important to consider how to supplement this approach to ensure a diverse range of practices are recruited. This was an issue flagged by Carr et al. in their discussion piece on practice readiness to engage in research. The authors emphasized the need to identify and ‘optimise’ recruitment approaches which can appeal to clinicians in general practices which do not have existing links to networks or universities and may therefore tend to be missed for research opportunities²⁷. By supporting development of relationships between practices and researchers, primary care networks can play an invaluable role in in supporting recruitment for larger scale clinical trials⁵².

Second, informing potential participants about the direct benefits the study offers is important to encourage recruitment⁵². The use of incentives was a common strategy, both financial and CPD, and these incentives were flagged in early communication with the practices. Ward et al., in their practical guide on recruitment, advise determining the schedule of payments in advance, and whether there will be a flat rate, payment for fee-for-service, or combination of both methods used⁵² – both approaches were used in the identified studies. However, the level of incentive varied between studies, making it difficult to know what financial compensation is acceptable and the level which would affect recruitment. The level of incentive will also depend on the nature of the trial, what is required of practices and for how long. Future studies could consider using experimental designs (e.g., discrete choice experiments) to elicit practice preferences. Studies evaluating a more standardised approach to incentives are needed to provide more robust evidence regarding the use of financial or CPD incentives.

Third, certain recruitment approaches reported very limited yield, often broad mailouts or marketing campaigns^11,34 However, it was not easy or possible to determine a denominator in these instances. While approaches such as targeting practices known the study team or from previous studies may have delivered a higher yield, it is important to note they were targeting much smaller pool of practices. It is important to consider whether such an approach can ensure a diverse range of practices; those less accustomed to participating in research which may be less ‘research ready’ may not be reached²⁷. Trialists should consider the potential value (and costs) of approaches which cast a ‘wider net’ (e.g., like the mass marketing approach used by Ellis et al.), versus more targeted approaches. To better understand their value, it would be helpful if trialists documented and compared the practices identified and recruited through these means to determine whether they lead to recruitment of a different profile of practice compared to those recruited through other more direct approaches.

Limitations

To our knowledge this is the first study to synthesise the evidence on strategies used to recruit general practices for trials. However, there are some limitations. As this was a rapid review, to identify and include studies we relied on recruitment being mentioned in the title or abstract. Ideally a review would identify and include all trials conducted in primary care and review the methods section to gather extra details on recruitment. While we followed best practice guidance for rapid reviews³¹, we did reduce the number of screeners at the title and abstract screening stage and searched a limited number of databases. Lastly, we did not undertake a quality assessment since the aim was to describe outcomes of strategies – not all studies specifically tested strategies.