Longitudinal survey of self-reported level of entrustment across the first year of clinical practice

Introduction

It has consistently been found that junior doctors lack the skills required to care for patients, and feel unprepared for clinical practice^1,2. The training and support received by junior doctors is variable, and dependent on the setting and clinical teams in which they are placed³. Moreover, junior doctors continue to the next stages of training, not based upon a determination of whether they have developed the necessary competencies, but rather as a result of the time they have spent in the role⁴. The issues associated with a time-based approach to training has led to an interest in a Competency Based Medical Education (CBME) for the training of junior doctors. CBME is concerned with outcomes of the education experience, independent of time spent in education⁵. A particular approach to CBME that has received interest within the context of the training of junior doctors is Entrustable Professional Activities (EPAs)⁶.

EPAs are an effort to bridge the gap between the theory of competencies and practical clinical work⁷. EPAs are units of professional practice that capture essential competencies in which trainees must become proficient⁸. EPAs describe discrete activities that can be entrusted to trainees, are essential to the profession, and encapsulate one or more core competencies⁹. EPAs use observable work descriptors (e.g., clerk a patient) as opposed the person-descriptors (e.g., good communicator)⁹. EPAs provide supervisors with a framework to support the delegation of activities to trainees⁸. EPAs have potential for improving the efficiency of training of junior doctors, and ensure that they perform only the activities they have been deemed safe to perform¹⁰.

In Ireland, a framework of seven EPAs (see Table 1) has been developed for the intern year (the first year of clinical practice after graduation in the Republic of Ireland). This EPA framework was developed through an iterative eight-stage consensus building with key stakeholders (see O’Dowd et al.⁴ for a description of this process). Levels of entrustment of trainees for each EPA are made on a scale of 1 to 5:

The desirable minimum level of entrustability in these EPAs for interns is to be able to execute with reactive supervision (level 3), or supervision at a distance and/or post hoc (level 4). The rationale for the need to reach this level of entrustability is that interns tend to work autonomously with limited support from more senior doctors- particularly during night and weekend shifts^11,12. Lack of supervision has been identified as a contributor to more than half of errors made by interns that ‘played on their mind’¹³. This issue of limited supervision is further compounded by the fact that junior doctors are also often unwilling to seek guidance and clinical support from seniors^12,14. Therefore, ideally, interns should be able to execute these EPAs with indirect supervision.

The aim of this study was to assess self-reported level of entrustment within, and frequency of performance of the seven Entrustable Professional Activities (EPAs) comprising the EPA framework for interns in Ireland at four time points in the intern year. This will allow an indication of when, and if, interns are able to execute these EPAs without the need for direct supervision.

Methods

The study is reported in accordance with the Consensus-Based Checklist for Reporting of Survey Studies (CROSS)¹⁵.

Results

Discussion

Medical training has traditionally comprised a time-based apprenticeship model both in the Republic of Ireland as well as internationally. As such, doctors advance to the next stages of training based on time, and not competence. EPAs have been promoted as an approach to support competency-based assessments^4–6,9,10 and improve patient safety. The purpose of our study was to assess self-reported level of entrustment within, and frequency of performance of, the seven Entrustable Professional Activities (EPAs) comprising the EPA framework for interns in Ireland in order to identify when, and if, interns are able to, execute them without the need for direct supervision.

Prior to starting internship, the respondents reported a need for direct supervision to carry out many of the EPAs, and the more advanced procedures in EPA 3. This finding is consistent with other studies that have found that high percentages of newly graduated medical students report feeling under-prepared to begin working in a hospital¹¹, and variability in their confidence to perform specific clinical skills¹. A national survey of junior doctors in Ireland found that only 51% believed that medical school prepared them well for their intern year¹⁷. However, even after only three months of clinical practice there was a large change in in the self-reported level of entrustability, with a much greater proportion of interns reporting that they could complete many of the EPAs and clinical skills with either reactive supervision, or supervision at a distance as compared to the baseline assessment. This is certainly a positive finding, and suggests that internship is effective in increasing the level of competence of junior doctors. However, there may be a cost to patients in terms of compromising safety, the efficiency of the health service (e.g. the need to repeat procedures¹⁸, ordering unnecessary tests¹⁹) as well as to the interns themselves (e.g. stress or burnout¹³) in terms of this largely on-the-job approach to skill development. These costs are worth considering, and may help to justify the resources required to establish a competency-based approach to education and training. There is also need to consider what level of entrustability is required for newly graduated medical students as this has implications for by the training and supervision required by interns. This is an issue that has been addressed by the UK General Medical Council for practical procedures and skills²⁰. It is suggested that this should also be considered within the broader context of the EPAs.

The interns reported carrying out the majority of the EPAs and clinical procedures at least once a week, with an increase in the frequency of performance during the year. It could be postulated that as interns complete the EPAs and clinical procedures more, they then become more confident in their ability- as indicated by the moderate correlation between self-reported entrustability and frequency of performance. However, there were five clinical procedures (nasogastric tube insertion; urinary catheter insertion; preparation, reconstitution, dilution and administration of IV drugs; blood sampling and blood cultures from central line and tunnelled line; and sterile field set-up) that, even by the third time point, the majority of the interns had not carried out in the past week. This information should be considered in terms of how many repetitions are required to become competent in a procedure. For example, it was found that it took junior doctors between 19 and 146 repetitions of peripheral venous cannulation procedures in the clinical environment to reach an acceptable level of performance²¹. Therefore, despite their confidence in their abilities, it may be that the intern are not receiving sufficient exposure to these clinical procedures in order to reach competency. Particularly for procedures for which the interns receive limited exposure, there is a need to give additional opportunities for them to practice and receive feedback on these skills. It is suggested that simulation-based education provides a mechanism for interns to become competent, in a safe learning environment.

Recommendations

The Irish intern year EPAs provide a framework for medical schools to design teaching and assessment to better prepare newly graduated medical students for internship and clinical practice. A higher level of preparedness of medical students at the beginning of internship would have a number of benefits. The newly graduated students could be formally assessed to ensure they are competent to start internship, it would reduce the need for ‘boot camps’and on-the-job learning of these EPAs. Given that most learning occurred during the first three months of training, it is suggested that extra support should be available for interns during this time period. A particular approach used in the ITN that was the subject of our research is a ‘buddy’ intern programme. In this programme, ‘buddy’ interns are recruited from a pool of intern volunteers who have just completed the intern year. The ‘buddy’ interns work alongside the new interns for the first month and provide support and mentoring, skills training, and work night shifts. There was strong support for this programme from the participating newly graduated doctors²².

There is a need to assess entrustability, beyond self-report. It has been suggested that there is general tendency for doctors to be over confident in their ability to complete particular clinical tasks²³. Therefore, we cannot draw conclusions about actual levels of entrustability as the interns’ competence has not been formally assessed. Such assessment will allow the identification of those interns that are not able to perform these without direct supervision, and provide them with extra training required. Approaches to the assessment of EPAs that have particular potential include the use of simulation and portfolios^6,24,25. However, there is a need for research on how such tools can be used- to include a consideration of the feasibility of these assessments being carried out by busy clinical supervisors¹⁰.

Conclusion

After three months of clinical practice, almost all of the interns believed they could execute all of the EPAs in the Irish intern EPA framework without the need for direct supervision. This is certainly a positive finding. However, consideration should be given as to how to better prepare medical students to execute these EPAs as an integral part of their undergraduate training, how the interns can be better supported during the first quarter of internship, and there is an urgent need to carry out formal assessments of entrustability, rather than relying on self-report. Adopting this type of competency based approach will benefit the interns (as they will know they have the competencies required to perform the job), their supervisors (who know they can trust the interns), and most importantly their patients- the main beneficiaries of well-trained interns.

Data availability