Predicting perineal trauma during childbirth using data from a general obstetric population

Study population

A national project called ‘The Maternal and Newborn Clinical Management System (MN-CMS)’ was rolled out in the Republic of Ireland in December 2016⁷. With this system, an electronic health record was created resulting in a move from paper clinical notes, allowing for all maternal and newborn information to be stored on one record. The first maternity hospital to implement the electronic health record in the Republic of Ireland was Cork University Maternity Hospital (CUMH). As a result, we used anonymised data from all singleton vaginal deliveries at CUMH from January 2019 to December 2020 to develop and internally validate a risk prediction model for third and fourth degree perineal tears.

We obtained ethical approval from the Clinical Research Ethics Committee of the Cork Teaching Hospitals (CREC) (ECM4(v)09/04/2020) in June 2020. The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklist was used as a guideline for reporting our study⁸ (available here).

Candidate predictors and outcome

In order to identify candidate predictors, we reviewed existing literature, used expert opinion (comprising obstetricians, epidemiologists and experts on the MN-CMS), and examined the distribution of the predictor in the data available to us, (for example, we excluded any variables with <5 exposed cases)⁹ to identify routinely measured candidate predictors for third and fourth degree tears.

Candidate predictors considered for model development included infant sex, maternal age, maternal body mass index (BMI), parity, mode of delivery, birthweight, post-term delivery, induction of labour and public/private antenatal care.

A description of candidate predictors is as follows: infant sex was categorised as male/female. Maternal age: this was recorded in units of years at the initial prenatal visit. Maternal BMI: maternal height (cm) and weight (kg) at initial prenatal visit were used to calculate maternal BMI. This was categorised as underweight <18.5, normal weight ≥18.5 to ≤24.9, overweight ≥25 to ≤29.9 and obese ≥30 (due to small numbers, underweight and normal weight were combined). Parity: this was recorded as number of previous completed pregnancies and was re-categorised nulliparous or multiparous. Mode of delivery (with manual support technique¹⁰) was categorised into four different groups as follows: spontaneous vaginal delivery with episiotomy, spontaneous vaginal delivery without episiotomy, forceps delivery and ventouse delivery. We grouped ventouse delivery (with and without episiotomy) and forceps delivery (with and without episiotomy) together due to the small number of these instrumental deliveries that occurred without episiotomy. An episiotomy was defined as a right mediolateral incision (i.e., a cut made by the doctor or midwife during childbirth that begins in the middle of the vaginal opening and extends down toward the buttocks at a 45-degree angle). Caesarean deliveries were excluded as only vaginal deliveries are at risk of perineal tears. Birthweight was measured to the nearest gram and analysed as an increase in risk per 100 gram increase in birthweight. Post-term delivery was defined as delivery at ≥40 weeks’ gestation (with estimated due date confirmed by first trimester ultrasound) compared to delivery at or before full-term (i.e., delivery at <40 weeks’ gestation). Induction of labour was recorded in the MN-CMS if any of the following methods were administered: artificial rupture of membranes, Dilapan-S®, balloon catheter, or prostaglandin gel. Public antenatal care was defined as free antenatal care through the Maternity and Infant Care Scheme in Ireland. This is available to anyone who lives in Ireland or intends to live there for at least one year. Private antenatal care was defined as choosing to pay a consultant's fee/hospital fee so that a particular obstetrician would provide the care throughout the pregnancy/birth and that recovery would take place in a private/semi-private hospital room.

Outcome: Degree of tear was diagnosed by an obstetrician or midwife at time of birth and recorded in the MN-CMS. Third/fourth degree tears were tears that extended into the anal sphincter complex or involved both the anal sphincter complex and anorectal mucosa.

Statistical analysis

All statistical analyses were performed using Stata MP 14.2 (RRID:SCR_012763) (free alternative, RStudio). We used univariable analysis to examine associations between candidate predictors and third/fourth degree tears. To develop the prediction model, any variables that were statistically significant in the univariable analysis (i.e., p-value < 0.1) were included in multivariable logistic regression with backward stepwise selection (with a p-value of 0.1 for exclusion). Therefore, all candidate predictors considered statistically significant in the univariable analysis were included at first and the least useful predictors (i.e., the variable that is the least statistically significant) were subsequently removed one-by-one.

Sample size calculations: We used the pmsampsize command to calculate the minimum sample size and number of events required for model development to minimise overfitting. Assuming an outcome event proportion (prevalence) of 0.015, a c-statistic of 0.75, a target shrinkage factor of 0.9, and 12 predictors/categories, then a minimum sample size of 7,995 (with 120 events) would be required to minimise overfitting¹¹.

Model performance: Spline functions with 3, 4 and 5 knots were used to assess non-linear functions for any continuous predictors included in model development. These were plotted against the original variable to compare the linear function with the spline function, while Akaike information criterion (AIC) and Bayesian information criterion (BIC) statistics were calculated to examine model fit.

We examined model performance by assessing overall fit, discrimination and calibration. Brier Score and Cragg & Uhler’s (Nagelkerke) R² assessed overall fit. The area under the receiver operating characteristic curve (ROC) C-statistic assessed discrimination (i.e., how well the model differentiates between those patients who experience third/fourth degree tears and those who do not⁹). Calibration-in-the-large (CITL), calibration slope (C-slope) and calibration plot (pmcalplot) of observed against expected probabilities across 10 risk groups of individuals assessed calibration (i.e., how closely the predictions of the models match the observed outcomes in the data^9,12).

Internal validation: To examine internal validation of our model, assess overfitting and calculate the optimism adjusted C-statistic, CITL and C-slope, we used bootstrapping techniques (with 1,000 repetitions). Finally, a graphical representation of our prediction model (i.e., nomogram) was developed to provide individualised risk assessment for third/fourth degree tears.

Risk prediction model

Third/fourth degree tears: Three variables were considered the best combined predictors of third/fourth degree tears using multivariable logistic regression with backward stepwise selection (C-statistic: 0.75, 95% CI: 0.71, 0.79). These included parity (specifically nulliparous), mode of delivery (specifically forceps delivery or ventouse delivery) and increasing birthweight (per 100 gram increase) (Table 2).

We developed a nomogram to provide an individualised risk assessment of third/fourth degree perineal tear using these predictors (Figure 1). For example, a forceps delivery (score 1.5), birthweight of ~4,600 grams (score 7), and nulliparous woman (score 3.5), the total score is 12, corresponding to an ~10% risk of third/fourth degree perineal tear.

Figure 1. Nomogram providing individualised risk assessment of third/fourth degree perineal tear.

For example, a forceps delivery (score 1.5), birthweight of ~4,600 grams (score 7), and nulliparous woman (score 3.5), the total score is 12, corresponding to an ~10% risk of third/fourth degree perineal tear.

Model performance and internal validation

There was little difference between the shape of the linear function for birthweight compared to the spline function using 3 and 4 knots, while 5 knots overfit the data (Figure A1, found as Extended data⁸). The AIC and BIC statistics were lowest for the linear function; therefore, birthweight was analysed as a linear function.

The result for the Brier Score and Cragg & Uhler’s (Nagelkerke) R² were 0.014 and 0.083, respectively. Average model predictions matched average observed outcomes for the 10 risk groups of individuals (i.e., the deciles of risk that were used as cut-off points to compare observed and expected probabilities in groups of individuals), as indicated by the calibration plot, suggesting good calibration. The majority of the deciles are clustered in the bottom left, suggesting most women have low risk of third/fourth degree tears. There is some miscalibration at the individual level in the higher risk individuals as suggested by the LOWESS smoother. However, there is very little data at the higher risk probabilities as indicated by the spike plot towards the bottom of the graph (Figure A2, found as Extended data⁸).

The original apparent C-statistic was 0.75 (95% CI: 0.71, 0.79). After bootstrapping, there was minimal optimism adjustment to the C-statistic, suggesting good internal performance (optimism adjusted C-statistic: 0.74). The miscalibration in CITL and C-slope were small indicating that overfitting was unlikely to be an issue (Table 2).

Strengths and limitations

This study contained some limitations that are important to note. First, we did not have access to data on previous history of third/fourth degree tears, length of second stage of labour, birthing position, or indication for instrumental delivery, which may have improved the accuracy of our model. However, before additional candidate predictors are added to a prediction model it is important to consider availability of an appropriate sample size to minimise overfitting. Additionally, regarding a lack of data on previous history of third/fourth degree tears, evidence examining risk of recurrence of third/fourth tears in subsequent pregnancies is inconsistent, and women who had an anal sphincter injury in their first pregnancy are more likely to have a caesarean section in their subsequent pregnancy to avoid a recurrent tear^16–19. Second, to minimise overfitting and maximise the number of events and total sample size for the current study, we grouped third and fourth degree tears together and used data from all singleton vaginal deliveries at CUMH during 2019 and 2020 to develop and internally validate our model. Ideally, we would have used 2020 data to conduct a temporal external validation in order to examine reproducibility of our model. However, despite this limitation, a geographical external validation would still be needed to assess generalisability of our findings. As it is recommended that external validation is carried out by an independent research team, we included the estimates needed to calculate the linear predictor of our model to allow for an independent external validation and objective evaluation of model performance²⁰. Third, previous evidence suggests that third/fourth degree tears may be subject to overdiagnosis potentially as a result of anxiety or fear of missing a diagnosis²¹. However, the rate of third/fourth degree tear reported in the current study (1.53%) was similar to that of the national estimate for 2019–2020 (1.6%–1.9%), reducing the possibility of misclassification of the outcome²². Fourth, there are many benefits of using secondary data for research purposes, in particular regarding the need for fewer resources. In addition to this, the data used in the current study are real world data and while it has deficits, it reflects outcome in practice. However, a reliance on existing data can be a limitation in terms of data availability, unmeasured variables, and uncertainty around data quality. For example, we did not have access to data on why episiotomies were performed, and while episiotomy was defined according to standard practice at CUMH (i.e., right mediolateral incision), this data could not be validated as we were reliant on secondary data only for the current study. Episiotomies angled at 40–60° are associated with a reduced risk of third and fourth degree tears compared to episiotomies with a more acute angle²³. Therefore, a validated measure of episiotomy is necessary to maximise model performance. Finally, this study was limited to singleton deliveries only, therefore results of the prediction model should not be generalised to multiple pregnancies.

There are also several strengths in this study. First, we conducted an internal validation of our model allowing us to assess overfitting and calculate an optimism adjusted C-statistic. Second, the amount of missing data for model development was small (<1%), minimising the likelihood of selection bias driven by missing data. Third, we conducted an appropriate sample size calculation to ensure a sufficiently large sample size in order to minimise overfitting. Finally, we developed a nomogram to graphically represent our prediction model. This provides an individualised risk assessment enabling the user to quickly and easily estimate the probability of sustaining a third/fourth degree tear.