Comparing non-invasive diabetes risk scores for detecting patients in clinical practice: a cross-sectional validation study [version 1; peer review: awaiting peer review]

Background: Type 2 diabetes (T2DM) is a significant cause of morbidity and mortality, thus early identification is of paramount importance. A high proportion of T2DM cases are undiagnosed highlighting the importance of effective detection methods such as non-invasive diabetes risk scores (DRSs). Thus far, no DRS has been validated in an Irish population. Therefore, the aim of this study was to compare the ability of nine DRSs to detect T2DM cases in an Irish population. Methods: This was a cross-sectional study of 1,990 men and women aged 46–73 years. Data on DRS components were collected from questionnaires and clinical examinations. T2DM was determined according to a fasting plasma glucose level ≥7.0 mmol/l or a glycated haemoglobin A1c level ≥6.5% (≥48 mmol/mol). Receiver operating characteristic curve analysis assessed the ability of DRSs and their components to discriminate T2DM cases. Results: Among the examined scores, area under the curve (AUC) values ranged from 0.71–0.78, with the Cambridge Diabetes Risk Score (AUC=0.78, 95% CI: 0.75–0.82), Leicester Diabetes Risk Score (AUC=0.78, 95% CI: 0.75–0.82), Rotterdam Predictive Model 2 (AUC=0.78, 95% CI: 0.74–0.82) and the U.S. Diabetes Risk Score (AUC=0.78, 95% CI: 0.74–0.81) demonstrating the largest AUC values as continuous variables and at optimal cut-offs. Regarding individual DRS components, anthropometric measures displayed the largest AUC values. Conclusions: The best performing DRSs were broadly similar in terms of their components; all incorporated variables for age, sex, BMI, hypertension and family diabetes history. The Cambridge Diabetes Open Peer Review Reviewer Status AWAITING PEER REVIEW Any reports and responses or comments on the article can be found at the end of the article. HRB Open Research Page 1 of 11 HRB Open Research 2021, 4:70 Last updated: 25 AUG 2021


Introduction
Type 2 diabetes (T2DM) is a significant cause of morbidity and mortality due to chronic hyperglycaemia and its complications. Importantly, the persistent hyperglycaemia that is associated with diabetes may cause serious health complications such as cardiovascular disease and impairment and malfunction of the renal, ophthalmic, vascular and nervous systems 1,2 . The risk of T2DM is determined by an interplay of genetic and metabolic factors, including ethnicity, family history, previous gestational diabetes, in addition to a number of lifestyle behavioural factors 2 . Globally, 422 million people had diabetes in 2014, the majority of which were T2DM, a rise from 108 million in 1980 2 . In 2012, 3.7 million deaths were attributed to diabetes and elevated glucose levels worldwide, with 43% of these deaths occurring in subjects under the age of 70 2 . In a 2003 report, the International Diabetes Federation (IDF) estimated that 89,800 Irish adults aged 20-79 had diabetes 3 . A more recent report in 2019 estimated a prevalence 148,200 T2DM cases, with 46,200 of these cases being undiagnosed 4 . This rise is likely due to a combination of an ageing population, increasing levels of obesity, poor diet and physical inactivity 5 .
It is important to note that a long asymptomatic period often precedes a clinical diagnosis of T2DM, allowing time for complications of hyperglycaemia to develop 5 . A previous study suggested that 41% of diabetes cases in Ireland's middle-aged population are undiagnosed 1 . According to the IDF, the estimated prevalence of undiagnosed diabetes in Europe and North America is 40.7% and 37.8% respectively 6 . Importantly, there is a significant financial burden on healthcare systems globally with regards to complications of T2DM; in 2006, diabetes care made up 10% of total healthcare expenditure in the Republic of Ireland alone 7 . More recently, the health costs and utilisation of health services attributed to diabetes in Ireland were explored. That study found that T2DM was responsible for a 39% increase in GP visits and 52% higher hospital admission rates, with an estimated additional cost of almost €89 million per year 8 . According to the IDF, Ireland currently ranks 6 th in the world for mean healthcare expenditure per person with diabetes 4 . These findings further highlight the importance of early diagnosis of the condition -before complications arise.
Non-invasive DRSs have been developed to identify patients with undiagnosed diabetes, thus indicating which patients should undergo further invasive blood testing. This could allow for earlier diagnosis and initiation of treatment in order to attenuate complications related to T2DM and improve patient outcomes 9 . Non-invasive DRSs are typically based on readily available simple and non-invasive health information; examples include the Finnish Diabetes Risk Score (FINDRISC), the Rotterdam Predictive Models and the Cambridge Diabetes Risk Score 10,11 among others. Each of these scores have been developed to identify undiagnosed diabetes cases and their use has been incorporated into diabetes prevention strategies 10 . Importantly, non-invasive DRSs present a more cost-effective and feasible method for case-finding on a large-scale compared to invasive blood testing alone 10 .
However, a DRS validated in one population may not be generalisable to others 10,12 , and currently no DRS has been developed in an Irish population. Given the high estimated prevalence of undiagnosed diabetes cases in Ireland, there is a need to assess the usefulness of these tools in an Irish sample. Therefore, the aim of this study was to apply nine DRSs to an Irish dataset to determine which score is most appropriate for use in an Irish population.

Study population
The Cork and Kerry Diabetes and Heart Disease Study (Phase II -Mitchelstown Cohort) was a single centre study conducted between 2010 and 2011 13 . A random sample was recruited from a large primary care centre in Mitchelstown, County Cork, Ireland. The Livinghealth Clinic serves a population of approximately 20,000 Caucasian-European subjects, with a mix of urban and rural residents. Stratified sampling was employed to recruit equal numbers of men and women from all registered attending patients in the 46-73-year age group. In total, 3,807 potential participants were selected from the practice list. Following the exclusion of duplicates, deaths and subjects incapable of consenting or attending appointment, 3,051 were invited to participate in the study and of these, 2,047 (49% male) completed the questionnaire and physical examination components of the baseline assessment (response rate: 67%). Individuals with pre-existing cardiovascular disease or T2DM were not excluded from the cohort. Details of the study design, sampling procedures and methods of data collection have been reported previously 13 .
Ethics committee approval conforming to the Declaration of Helsinki was obtained from the Clinical Research Ethics Committee, University College Cork. A letter signed by the contact GP in the clinic was sent out to all selected participants with a reply slip indicating acceptance or refusal. All subjects gave signed informed consent, including permission to use their data for research purposes.
Clinical and anthropometric data Clinical measurements were taken by researchers who were thoroughly trained according to the study research protocols. Participants attended the clinic the morning after an overnight fast (minimum 8 hours) and blood samples were taken on arrival. Data on age, sex, medical history and lifestyle behaviours were collected using a self-completed General Health Questionnaire 14 . Study participants answered questions regarding personal hypertension diagnosis/treatment, corticosteroid use and personal and family diabetes diagnosis/treatment. Smoking status was defined as never, former and current smokers. Diet was assessed using a modified version of the EPIC Food Frequency Questionnaire, validated for use in the Irish population 15 . Physical activity levels were assessed using the validated International Physical Activity Questionnaire (IPAQ) 16 and subjects were classified as having low, moderate or high physical activity levels. Blood pressure (BP) was measured using an Omron M7 Digital BP monitor (Omron Healthcare Co. Ltd., Japan) on the right arm after a 5-minute rest in a seated position. The mean of the second and third measurements was used in analyses 17 .
The weight and height of each participant was measured to the nearest 0.1 kg and 0.1 cm respectively. Portable electronic Tanita WB-100MA weighing scales (Tanita Corporation, IL, USA) were placed on a firm, flat surface and were calibrated weekly to ensure accuracy. Height was measured using a portable Seca Leicester height/length stadiometer (Seca, Birmingham, UK) and body mass index (BMI) was calculated as weight divided by the square of height. Waist circumference was measured midway between the lowest rib and iliac crest on bare skin. Participants were instructed to breathe in, and then out, and to hold their breath while measurement was made to the nearest 0.1 cm using a Seca 200 measuring tape. The mean of two independent readings was used in analyses.

Biological analysis
Fasting glucose and glycated haemoglobin A 1c (HbA 1c ) levels were measure by Cork University Hospital Biochemistry Laboratory. Fasting plasma glucose levels were collected the morning after an overnight fast using sodium fluoride EDTA collection tubes. Glucose concentrations were determined using a glucose hexokinase assay (Olympus Life and Material Science Europa Ltd., Lismeehan, Co. Clare, Ireland) and HbA 1c levels were measured in the haematology laboratory on an automated high-pressure liquid chromatography instrument Tosoh G7 [Tosoh HLC-723 (G7), Tosoh Europe N.V, Tessenderlo, Belgium]. For this study, T2DM was determined according to a fasting glucose level ≥7.0mmol/l or a HbA 1c level ≥6.5% (≥48mmol/mol). Participants with a self-reported diagnosis of diabetes, but who did not have positive test results, and subjects for whom we were missing glucose or HbA 1c data (n = 57) were excluded from the analysis. The final sample consisted of 1,990 subjects.

Diabetes risk scores
Nine non-invasive diabetes DRSs were assessed. These included the Brazil Diabetes Risk Score 18 , Cambridge Diabetes Risk Score 19 , Danish Diabetes Risk Score 20 , the FINDRISC Models (Full and Concise) 21 , the Leicester Diabetes Risk Score 22 , the Rotterdam Predictive Models (1 and 2) 23 and the U.S. Diabetes Risk Score 24 . These were chosen as they include readily available health information and have been developed to identify undiagnosed T2DM cases. Further details regarding the study populations and components used in each DRS are summarised in Table 1 and Supporting Table 1 (see data availability section) 25 .

Statistical analysis
Descriptive characteristics were examined. Categorical features are presented as percentages and continuous variable are displayed as a mean (plus or minus one standard deviation) or a median and interquartile range for skewed data. Statistical differences were evaluated using chi-square tests, independent t-tests or a Mann-Whitney U.
Receiver operating characteristic curve (ROC) analysis was used to assess the ability of individual DRSs, and their components to discriminate prevalent T2DM cases. The area under the curve (AUC) provides a scale of 0.5 to 1.0, with 0.5 representing random chance and 1.0 indicating perfect discrimination, by which to compare the ability of a score or marker to detect a positive result. Optimal cut-off thresholds for DRSs were determined using the method that finds the cut-point which has a sensitivity and specificity closest to the top left of the ROC space. The diagnostic properties of optimal DRS cut-offs were compared by determining AUC values, sensitivity, specificity, positive and negative likelihood ratios (+LR, -LR) and positive and negative predictive values (PPV, NPV).
Using DRSs which had the highest AUC values, we determined the percentage of patients who would undergo further invasive testing, and the proportion of prevalent T2DM cases that would be correctly detected, according to stratification by DRS cut-offs corresponding to 95%, 90%, 85%, and 80% fixed sensitivities.
Data analysis was conducted using IBM SPSS Version 25 (IBM Corp, Armonk, NY, USA) for Windows. For all analyses, a P value (two-tailed) of less than .05 was considered to indicate statistical significance.

Descriptive characteristics
Baseline characteristics for the full sample, and for the participants with and without T2DM, are presented in Table 2. The number of participants with T2DM was 160, representing a prevalence of 8.0%. Participants with T2DM were more likely to be male, were older, had a greater body weight, a higher BMI and waist circumference as well as a higher systolic BP and were more likely to be taking prescribed antihypertensive medication. They also were more likely to report lower physical activity levels and a family history of diabetes compared to participants without T2DM.

Diagnostic statistics
Diagnostic statistics for DRSs, and individual DRS components, are shown in Table 3 and Supporting Table 2 (see data availability section) 25

Discussion
In this study of 1,990 middle-to older-aged Irish men and women, we compared the ability of nine non-invasive DRSs to discriminate prevalent T2DM cases in a clinical setting.   These findings highlight the importance of obesity assessment within clinical practice 26 .
As previously discussed, each of the best performing scores incorporate variables for age, sex, BMI, hypertension and family diabetes history. In addition the Rotterdam Predictive Model 1 23 and the U.S. Diabetes Risk Score 24 also include a measure of physical activity, while the Leicester Diabetes Risk Score 22 additionally incorporates waist circumference and ethnicity, the latter was not relevant to this study population. The Cambridge Diabetes Risk Score 19 also includes the use of prescribed steroids and smoking status. The full FINDRISC model 21 incorporates history of high blood glucose as one of its variables, which was not collected during this study. It should be noted that the FINDRISC models were originally designed to predict risk of T2DM development. However, their use as tools for discriminating prevalent diabetes cases has also been validated 21,27 . Also, the Danish Diabetes Risk Score records cycling activity as a proxy for physical activity 20 ; when constructing this model using the Mitchelstown dataset, physical activity levels were determined from the participant's IPAQ categorisation.
The predictive capability of the examined DRSs in our study was lower than reported in the development studies for the Brazil Diabetes Risk Score (AUC = 0.71 vs AUC = 0.77) 18 and both FINDRISC models (AUC = 0.72 (Concise); AUC = 0.73 (Full) vs AUC = 0.80 for both) 21 . Several scores performed better in our sample than in their development studies. These included the Leicester Diabetes Risk Score (AUC = 0.78 vs AUC = 0.72) 22 19 . These findings further emphasise the importance of validating a score within the intended population of use to assess suitability 10,12 .

Strengths and limitations
Our study has several strengths. This research is the first to examine the ability of non-invasive DRSs to discriminate T2DM cases in a middle-to older-aged Irish population.
Our results are of potential clinical significance with regards to case-finding and the use of DRSs to detect prevalent diabetes cases. The importance of identifying individuals with undiagnosed diabetes cannot be understated, given the litany of complications which arise in untreated cases. These complications, which include cardiovascular disease, vision loss, renal failure and cognitive decline, present a significant burden to healthcare systems, can have devastating consequences for individuals and importantly, are preventable 1,2 . Notwithstanding these strengths, several limitations can be identified. These include measurement of HbA 1c and fasting plasma glucose at one time point and lack of Oral Glucose Tolerance test results as a comparison test. Another limitation of this study is the relatively modest number of diabetes cases available for analysis, which precluded stratification by sex and age. A larger sample would have conferred greater power and precision to our results. The cohort of this study is aged 46-73 years and therefore may not be generalizable to a younger population, however, T2DM is primarily a disease of middle-to-older aged people. An additional limitation of this study is that our data were derived from a single primary care-based sample, therefore the possibility that the sample may not be representative of the national population must be noted. However, previous research suggests that approximately 98% of Irish adults are registered with a GP and that, even in the absence of a universal patient registration system, it is possible to perform population-based epidemiological studies that are representative using our methods 28 . In addition, as Ireland is an ethnically homogeneous population 29 , the relationships observed in this study may be comparable to other middle-to older-aged Irish adults.

Conclusions
In summary, our findings indicate that of the examined non-invasive DRSs, the Cambridge Diabetes Risk Score, the Leicester Diabetes Risk Score, Rotterdam Predictive Model 2 and the U.S. Diabetes Risk Score demonstrated the greatest discriminatory ability and potential for use in a clinical setting. The Cambridge Diabetes Risk Score had the largest AUC value at an optimal cut-off, can be easily accessed online for use in a clinical setting 30 and may be the most appropriate for use in an Irish population to detect undiagnosed diabetes cases. In light of the increasing prevalence of T2DM worldwide 2 , and the significant estimated proportion of those undiagnosed 1,6 , an efficient method to identify undiagnosed patients is needed. Non-invasive DRSs may present a feasible