Glen Philip Martin

ObjectiveUnplanned reattendances to the pediatric emergency department (PED) occur commonly in clinical practice. Multiple factors influence the decision to return to care, and understanding risk factors may allow for better design of clinical services. We developed a clinical prediction model to predict return to the PED within 72 hours from the index visit.MethodsWe retrospectively reviewed all attendances to the PED of Royal Manchester Children's Hospital between 2009 and 2019. Attendances were excluded if they were admitted to hospital, aged older than 16 years or died in the PED. Variables were collected from Electronic Health Records reflecting triage codes. Data were split temporally into a training (80%) set for model development and a test (20%) set for internal validation. We developed the prediction model using LASSO penalized logistic regression.ResultsA total of 308,573 attendances were …

ObjectivesIn rheumatology, there is a clinical need to identify patients at high risk (>50%) of not responding to the first-line therapy methotrexate (MTX) due to lack of disease control or discontinuation due to adverse events (AEs). Despite this need, previous prediction models in this context are at high risk of bias and ignore AEs. Our objectives were to (i) develop a multinomial model for outcomes of low disease activity and discontinuing due to AEs 6 months after starting MTX, (ii) update prognosis 3-month following treatment initiation, and (iii) externally validate these models.Study Design and SettingA multinomial model for low disease activity (submodel 1) and discontinuing due to AEs (submodel 2) was developed using data from the UK Rheumatoid Arthritis Medication Study, updated using landmarking analysis, internally validated using bootstrapping, and externally validated in the Norwegian Disease …

DESIGN:Single-center retrospective external validation study.DATA SOURCES:Routinely collected healthcare data in the ICU electronic patient record. Curated data recorded for each ICU admission for the purposes of the UK Intensive Care National Audit and Research Centre (ICNARC).SETTING:The ICU at Manchester Royal Infirmary, Manchester, United Kingdom.PATIENTS:Three hundred forty-nine patients admitted to ICU with confirmed COVID-19 Pneumonitis, older than 18 years, from March 1, 2020, to February 28, 2022. Three hundred two met the inclusion criteria for at least one model. Fifty-five of the 349 patients were admitted before the widespread adoption of dexamethasone for the treatment of severe COVID-19 (pre-dexamethasone patients).OUTCOMES:Ability to be externally validated, discriminate, and calibrate.METHODS:Articles meeting the inclusion criteria were identified, and those that gave …

MethodsThe PATTErn study (A study of Physical Activity paTTerns and major health Events in older people with implantable cardiac devices) enrolled participants aged 60+ undergoing remote cardiac monitoring. Frailty was measured using the Fried criteria and gait speed (m/s), and physical functioning by NYHA class and SF-36 physical functioning score. Activity was reported as mean time active/day across 30-days prior to enrolment (30-day PA). Multivariable regression methods were utilised to estimate associations between PA and frailty/functioning (OR= odds ratio, β= beta coefficient, CI= confidence intervals).ResultsData were available for 140 participants (median age 73, 70.7% male). Median 30-day PA across the analysis cohort was 134.9 mins/day (IQR 60.8–195.9). PA was not significantly associated with Fried frailty status on multivariate analysis, however was associated with gait speed (β= 0.04, 95% CI 0.01–0.07, p= 0.01) and measures of physical functioning (NYHA class: OR 0.73, 95% CI 0.57–0.92, p= 0.01, SF-36 physical functioning: β= 4.60, 95% CI 1.38–7.83, p= 0.005).ConclusionsPA from cardiac devices was associated with physical functioning and gait speed. This highlights the importance of reviewing remote monitoring PA data to identify patients who could benefit from existing interventions. Further research should investigate how to embed this into clinical pathways.

External validation studies are an important but often neglected part of prediction model research. In this article, the second in a series on model evaluation, Riley and colleagues explain what an external validation study entails and describe the key steps involved, from establishing a high quality dataset to evaluating a model’s predictive performance and clinical usefulness.

Evaluating the performance of a clinical prediction model is crucial to establish its predictive accuracy in the populations and settings intended for use. In this article, the first in a three part series, Collins and colleagues describe the importance of a meaningful evaluation using internal, internal-external, and external validation, as well as exploring heterogeneity, fairness, and generalisability in model performance.

Introduction/Background The geographical heterogeneity seen in treatment patterns for patients with advanced ovarian cancer is profound, long standing, worrying, and impacts upon survival. A tool that could demonstrate the impact of a patient’s treatment on their predicted survival is needed to counsel patients about their treatment options and reduce treatment variance.We used detailed clinical datasets to develop a model for predicting treatment dependent survival in ovarian cancer patients.Methodology Data were collected using a data dictionary for all cases of ovarian cancer presenting to six cancer centres in England between 1/1/2018 and 31/12/2019.A Cox Proportional Hazard model was built using internal-external cross validation to estimate data heterogeneity between centres. Variables were assessed for non-linear relationships. Backwards selection was used to optimise fit. The hazard ratios for …

Excess body mass index (BMI) is associated with a higher risk of at least 13 cancers, but it is usually measured at a single time point. We tested whether the overweight‐years metric, which incorporates exposure time to BMI ≥25 kg/m2, is associated with cancer risk and compared this with a single BMI measure. We used adulthood BMI readings in the Atherosclerosis Risk in Communities (ARIC) study to derive the overweight‐years metric. We calculated associations between the metric and BMI and the risk of cancers using Cox proportional hazards models. Models that either included the metric or BMI were compared using Harrell's C‐statistic. We included 13,463 participants, with 3,876 first primary cancers over a mean of 19 years (SD 7) of cancer follow‐up. Hazard ratios for obesity‐related cancers per standard deviation overweight‐years were 1.15 (95% CI: 1.05–1.25) in men and 1.14 (95% CI: 1.08–1.20 …

Careful handling of missing data is crucial to ensure that clinical prediction models are developed, validated, and implemented in a robust manner. We determined the bias in estimating predictive performance of different combinations of approaches for handling missing data across validation and implementation. We found four strategies that are compatible across the model pipeline and have provided recommendations for handling missing data between model validation and implementation under different missingness mechanisms.

Background and aimsSepsis is a serious and life-threatening condition caused by a dysregulated immune response to an infection. Recent guidance issued in the UK gave recommendations around recognition and antibiotic treatment of sepsis, but did not consider factors relating to health inequalities. The aim of this study was to summarise the literature investigating associations between health inequalities and sepsis.MethodsSearches were conducted in Embase for peer-reviewed articles published since 2010 that included sepsis in combination with one of the following five areas: socioeconomic status, race/ethnicity, community factors, medical needs and pregnancy/maternity.ResultsFive searches identified 1,402 studies, with 50 unique studies included in the review after screening (13 sociodemographic, 14 race/ethnicity, 3 community, 3 care/medical needs and 20 pregnancy/maternity; 3 papers examined …

PurposeSepsis is a life-threatening organ dysfunction caused by dysregulated host response to infection. The purpose of the study was to measure the associations of specific exposures (deprivation, ethnicity, and clinical characteristics) with incident sepsis and case fatality.MethodsTwo research databases in England were used including anonymized patient-level records from primary care linked to hospital admission, death certificate, and small-area deprivation. Sepsis cases aged 65–100 years were matched to up to six controls. Predictors for sepsis (including 60 clinical conditions) were evaluated using logistic and random forest models; case fatality rates were analyzed using logistic models.Results108,317 community-acquired sepsis cases were analyzed. Severe frailty was strongly associated with the risk of developing sepsis (crude odds ratio [OR] 14.93; 95% confidence interval [CI] 14.37–15.52). The …

Clinical prediction models are increasingly used across healthcare to support clinical decision making. Existing methods and models are time-invariant and thus ignore the changes in populations and healthcare practice that occur over time. We aimed to compare the performance of time-invariant with time-variant models in UK National Adult Cardiac Surgery Audit data from Manchester University NHS Foundation Trust between 2009 and 2019. Data from 2009-2011 were used for initial model fitting, and data from 2012-2019 for validation and updating. We fitted four models to the data: a time-invariant logistic regression model (not updated), a logistic model which was updated every year and validated it in each subsequent year, a logistic regression model where the intercept is a function of calendar time (not updated), and a continually updating Bayesian logistic model which was updated with each new …

This study aimed to (1) add to the limited evidence base regarding genital injury associated with digital vaginal penetration and (2) identify predisposing or protective factors to the identification of a genital injury. Data collection was performed retrospectively on the paper case files of 120 female adult (>18 years) patients alleging digital vaginal penetration with no penile vaginal penetration that had an acute FME at Saint Mary's Sexual Assault Referral Centre (SARC) Manchester. Descriptive statistics were used to investigate differences in the demographics of those reporting digital penetration, with and without injuries. Overall, 18% had genital injuries noted at the time of the FME. Posterior fourchette was the most common location of genital injury and abrasion was the most common injury type. It is worth further noting that all 22 patients where an injury was noted were of white ethnicity, only 12 patients in the …

Background Most existing clinical prediction models do not allow predictions under interventions. Such predictions allow predicted risk under different proposed strategies to be compared and are therefore useful to support clinical decision making. We aimed to compare methodological approaches for predicting individual level cardiovascular risk under three interventions: smoking cessation, reducing blood pressure, and reducing cholesterol. Methods We used data from the PREDICT prospective cohort study in New Zealand to calculate cardiovascular risk in a primary care setting. We compared three strategies to estimate absolute risk under intervention: (a) conditioning on hypothetical interventions in non-causal models; (b) combining existing prediction models with causal effects estimated using observational causal inference methods; and (c) combining existing prediction models with causal effects reported in published literature. Results The median absolute cardiovascular risk among smokers was 3.9%; our approaches predicted that smoking cessation reduced this to a median between a non-causal estimate of 2.5% and a causal estimate of 2.8%, depending on estimation methods. For reducing blood pressure, the proposed approaches estimated a reduction of absolute risk from a median of 4.9% to a median between 3.2% and 4.5% (both derived from causal estimation). Reducing cholesterol was estimated to reduce median absolute risk from 3.1% to between 2.2% (non-causal estimate) and 2.8% (causal estimate). Conclusions Estimated absolute risk reductions based on non-causal methods were different to those based on …

Background/Aims As recommended by the NICE guidelines, methotrexate (MTX) is typically prescribed as the first line therapy for patients with rheumatoid arthritis (RA). However, around 40% of patients do not respond to MTX at 6 months and around 80% experience adverse events (AEs). Our previous systematic review identified that clinical prediction models of MTX outcomes suffered from methodological limitations, including a lack of validation, suboptimal handling of missing data, and no consideration of competing risks, such as patients discontinuing due to adverse events (AEs). These shortcomings resulted in high risk of bias and should be addressed to aid the implementation of outcome prediction in clinical practice. Therefore, this study aimed to (i) develop a multinomial prediction model for estimating an individual’s risks of not achieving low disease activity (LDA) and discontinuing due …

ObjectivesAssessment of frailty prior to aortic valve intervention is recommended in European and North American valvular heart disease guidelines. However, there is a lack of consensus on how it is best measured. The Clinical Frailty Scale (CFS) is a well-validated measure of frailty that is relatively quick to calculate. This meta-analysis sought to examine whether the CFS predicts mortality and morbidity following either transcatheter aortic valve implantation (TAVI) or surgical aortic valve replacement (SAVR).MethodsNine electronic databases were searched systematically for data on clinical outcomes post-TAVI/SAVR, where patients had undergone preoperative frailty assessment using the CFS. The primary endpoint was 12-month mortality. TAVI and SAVR data were assessed and reported separately. For each individual study, the incidence of adverse outcomes was extracted according to a CFS score of 5–9 …

ObjectiveMultimorbidity, the presence of two or more long-term conditions, is a growing public health concern. Many studies use analytical methods to discover multimorbidity patterns from data. We aimed to review approaches used in published literature to validate these patterns.Study Design and SettingWe systematically searched PubMed and Web of Science for studies published between July 2017 and July 2023 that used analytical methods to discover multimorbidity patterns.ResultsOut of 31,617 studies returned by the searches, 172 were included. Of these, 111 studies (64%) conducted validation, the number of studies with validation increased from 53.13% (17 out of 32 studies) to 71.25% (57 out of 80 studies) in 2017-2019 to 2022-2023, respectively. Five types of validation were identified: assessing the association of multimorbidity patterns with clinical outcomes (n = 79), stability across subsamples (n …

Multinomial prediction models (MPMs) have a range of potential applications across healthcare where the primary outcome of interest has multiple nominal or ordinal categories. However, the application of MPMs is scarce, which may be due to the added methodological complexities that they bring. This article provides a guide of how to develop, externally validate, and update MPMs. Using a previously developed and validated MPM for treatment outcomes in rheumatoid arthritis as an example, we outline guidance and recommendations for producing a clinical prediction model, using multinomial logistic regression. This article is intended to supplement existing general guidance on prediction model research. This guide is split into three parts: 1) Outcome definition and variable selection, 2) Model development, and 3) Model evaluation (including performance assessment, internal and external validation, and model recalibration). We outline how to evaluate and interpret the predictive performance of MPMs. R code is provided. We recommend the application of MPMs in clinical settings where the prediction of a nominal polytomous outcome is of interest. Future methodological research could focus on MPM-specific considerations for variable selection and sample size criteria for external validation.

When designing a study to develop a new prediction model with binary or time‐to‐event outcomes, researchers should ensure their sample size is adequate in terms of the number of participants (n) and outcome events (E) relative to the number of predictor parameters (p) considered for inclusion. We propose that the minimum values of n and E (and subsequently the minimum number of events per predictor parameter, EPP) should be calculated to meet the following three criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of 0.9, (ii) small absolute difference of 0.05 in the model's apparent and adjusted Nagelkerke's R2, and (iii) precise estimation of the overall risk in the population. Criteria (i) and (ii) aim to reduce overfitting conditional on a chosen p, and require prespecification of the model's anticipated Cox‐Snell R2, which we show can be obtained from previous …

Background: In clinical prediction modelling, missing data can occur at any stage of the model pipeline; development, validation or deployment. Multiple imputation is often recommended yet challenging to apply at deployment; for example, the outcome cannot be in the imputation model, as recommended under multiple imputation. Regression imputation uses a fitted model to impute the predicted value of missing predictors from observed data, and could offer a pragmatic alternative at deployment. Moreover, the use of missing indicators has been proposed to handle informative missingness, but it is currently unknown how well this method performs in the context of clinical prediction models. Methods: We simulated data under various missing data mechanisms to compare the predictive performance of clinical prediction models developed using both imputation methods. We consider deployment scenarios where …