Effect measures

medRxiv

Background Recent systematic reviews and meta-analyses on the effects of interventions to prevent obesity in children aged 5 to 18 years identified over 200 randomized trials. Interventions targeting diet, activity (including physical activity and sedentary behaviours) and both diet and activity appear to have small but beneficial effects, on average. However, these effects varied between studies and might be explained by variation in characteristics of the interventions, for example by the extent to which the children enjoyed the intervention or whether they aim to modify behaviour through education or physical changes to the environment. Here we develop a novel analytic framework to identify key intervention characteristics considered likely to explain differential effects. Objectives To describe the development of the analytic framework, including the involvement of school-aged children, parents, teachers and other stakeholders, and to present the content of the finalized analytic framework and the results of the coding of the interventions. Design and methods We first conducted a literature review to find out from the existing literature what different types of characteristics of interventions we should be thinking about, and why. This information helped us to develop a comprehensive map (called a logic model) of these characteristics. We then used this logic model to develop a list of possible intervention characteristics. We held a series of workshops with children, parents, teachers and public health professionals to refine the list into a coding scheme. We then used this to code the characteristics of each intervention in all the trials which aimed to …

BackgroundAcute respiratory infections are a common reason for consultation with primary and emergency healthcare services. Identifying individuals with a bacterial infection is crucial to ensure appropriate treatment. However, it is also important to avoid over-prescription of antibiotics, to prevent unnecessary side effects and antimicrobial resistance.

BackgroundHigh temperatures and heatwaves are occurring more frequently and lasting longer because of climate change. A synthesis of existing evidence of heat-related health impacts in the Western Pacific Region (WPR) is lacking. This review addresses this gap.MethodsThe Scopus and PubMed databases were searched for reviews about heat impacts on mortality, cardiovascular morbidity, respiratory morbidity, dehydration and heat stroke, adverse birth outcomes, and sleep disturbance. The last search was conducted in February 2023 and only publications written in English were included. Primary studies and reviews that did not include specific WPR data were excluded. Data were extracted from 29 reviews.FindingsThere is strong evidence of heat-related mortality in the WPR, with the evidence concentrating on high-income countries and China. Associations between heat and cardiovascular or …

Journal of the College of Community Physicians of Sri Lanka

ImportanceMendelian randomization (MR) studies use genetic variation associated with modifiable exposures to assess their possible causal relationship with outcomes and aim to reduce potential bias from confounding and reverse causation.ObjectiveTo develop the STROBE-MR Statement as a stand-alone extension to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guideline for the reporting of MR studies.Design, Setting, and ParticipantsThe development of the STROBE-MR Statement followed the Enhancing the Quality and Transparency of Health Research (EQUATOR) framework guidance and used the STROBE Statement as a starting point to draft a checklist tailored to MR studies. The project was initiated in 2018 by reviewing the literature on the reporting of instrumental variable and MR studies. A group of 17 experts, including MR methodologists, MR study design …

BackgroundRecent evidence suggests that PD-1/PD-L1 immunotherapy improves outcomes in patients with brain metastatic non-small cell lung cancer.MethodsRecords were searched electronically on MEDLINE, Embase and BIOSIS. Hazard ratios and their 95% confidence intervals for overall survival and progression free survival, and treatment-related adverse events data were extracted. Risk of bias was assessed in included studies using the Cochrane Collaboration's revised tool to assess risk of bias in randomized trials.ResultsPD-1/PD-L1 immunotherapy increased overall survival by 33% and progression free survival by 47% compared with chemotherapy. Two studies had a high risk of bias. Treatment-related adverse events were reported in 95%, 89% and 65% of patients receiving chemoimmunotherapy,chemotherapy and single agent immunotherapy, respectively.ConclusionPD-1/PD-L1 inhibitors alone …

IntroductionNetwork meta-analyses (NMAs) have gained popularity and grown in number due to their ability to provide estimates of the comparative effectiveness of multiple treatments for the same condition. The aim of this study is to conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs.Methods and analysisWe included papers that present items related to bias, reporting or methodological quality, papers assessing the quality of NMAs, or method papers. We searched MEDLINE, the Cochrane Library and unpublished literature (up to July 2020). We extracted items related to bias in NMAs. An item was excluded if it related to general systematic review quality or bias and was included in currently available tools such as ROBIS or AMSTAR 2. We reworded items, typically structured as questions, into concepts (i.e. general notions).ResultsOne hundred eighty-one articles …

Environment International

BackgroundObservational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies.ObjectiveTo develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome.Methods and resultsROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that …

European Journal of Epidemiology

Fixed-effect meta-analysis has been used to summarize genetic effects on a phenotype across multiple Genome-Wide Association Studies (GWAS) assuming a common underlying genetic effect. Genetic effects may vary with age (or other characteristics), and not allowing for this in a GWAS might lead to bias. Meta-regression models between study heterogeneity and allows effect modification of the genetic effects to be explored. The aim of this study was to explore the use of meta-analysis and meta-regression for estimating age-varying genetic effects on phenotypes. With simulations we compared the performance of meta-regression to fixed-effect and random-effects meta-analyses in estimating (i) main genetic effects and (ii) age-varying genetic effects (SNP by age interactions) from multiple GWAS studies under a range of scenarios. We applied meta-regression on publicly available summary data to estimate the …

arXiv preprint arXiv:2402.18298

Quantitative evidence synthesis methods aim to combine data from multiple medical trials to infer relative effects of different interventions. A challenge arises when trials report continuous outcomes on different measurement scales. To include all evidence in one coherent analysis, we require methods to `map' the outcomes onto a single scale. This is particularly challenging when trials report aggregate rather than individual data. We are motivated by a meta-analysis of interventions to prevent obesity in children. Trials report aggregate measurements of body mass index (BMI) either expressed as raw values or standardised for age and sex. We develop three methods for mapping between aggregate BMI data using known relationships between individual measurements on different scales. The first is an analytical method based on the mathematical definitions of z-scores and percentiles. The other two approaches involve sampling individual participant data on which to perform the conversions. One method is a straightforward sampling routine, while the other involves optimization with respect to the reported outcomes. In contrast to the analytical approach, these methods also have wider applicability for mapping between any pair of measurement scales with known or estimable individual-level relationships. We verify and contrast our methods using trials from our data set which report outcomes on multiple scales. We find that all methods recreate mean values with reasonable accuracy, but for standard deviations, optimization outperforms the other methods. However, the optimization method is more likely to underestimate standard deviations …

arXiv preprint arXiv:2401.01806

Network meta-analysis (NMA) combines evidence from multiple trials to compare the effectiveness of a set of interventions. In public health research, interventions are often complex, made up of multiple components or features. This makes it difficult to define a common set of interventions on which to perform the analysis. One approach to this problem is component network meta-analysis (CNMA) which uses a meta-regression framework to define each intervention as a subset of components whose individual effects combine additively. In this paper, we are motivated by a systematic review of complex interventions to prevent obesity in children. Due to considerable heterogeneity across the trials, these interventions cannot be expressed as a subset of components but instead are coded against a framework of characteristic features. To analyse these data, we develop a bespoke CNMA-inspired model that allows us to identify the most important features of interventions. We define a meta-regression model with covariates on three levels: intervention, study, and follow-up time, as well as flexible interaction terms. By specifying different regression structures for trials with and without a control arm, we relax the assumption from previous CNMA models that a control arm is the absence of intervention components. Furthermore, we derive a correlation structure that accounts for trials with multiple intervention arms and multiple follow-up times. Although our model was developed for the specifics of the obesity data set, it has wider applicability to any set of complex interventions that can be coded according to a set of shared features.

Living reviews are an increasingly popular research paradigm. The purpose of a ‘living’ approach is to allow rapid collation, appraisal and synthesis of evolving evidence on an important research topic, enabling timely influence on patient care and public health policy. However, living reviews are time- and resource-intensive. The accumulation of new evidence and the possibility of developments within the review’s research topic can introduce unique challenges into the living review workflow.To investigate the potential of software tools to support living systematic or rapid reviews, we present a narrative review informed by an examination of tools contained on the Systematic Review Toolbox website. We identified 11 tools with relevant functionalities and discuss the important features of these tools with respect to different steps of the living review workflow. Four tools (NestedKnowledge, SWIFT-ActiveScreener …

bmj

Various methods are available to help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis. This limitation of existing methods leaves users to decide their own approach for judging the risk of bias in a meta-analysis result. In this paper, Page and colleagues describe the ROB-ME (risk of bias due to missing evidence) tool, a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.

Cochrane Evidence Synthesis and Methods

Background Replication is essential to the scientific method. It is unclear what systematic reviewers think about the replication of systematic reviews (SRs). Therefore, we aimed to explore systematic reviewers' perspectives on (a) the definition and importance of SR replication; (b) incentives and barriers to conducting SR replication; and (c) a checklist to guide when to replicate an SR. Methods We searched PubMed for SRs published from January to April 2021, from which we randomly allocated 50% to this survey and 50% to another survey on data sharing in SRs. We sent an electronic survey to authors of these SRs (n = 4669) using Qualtrics. Quantitative responses were summarized using frequency analysis. Free‐text answers were coded using an inductive approach. Results The response rate was 9% (n = 409). Most participants considered “replication of SRs” as redoing an SR (68%) or reanalyzing …

BMJ Evidence-Based Medicine

A systematic review identifies, appraises and synthesises all the empirical evidence from studies that meet prespecified eligibility criteria to answer a specific research question. As part of the appraisal, researchers use explicit methods to assess risk of bias in the results’ from included studies that contribute to the review’s findings, to improve our confidence in the review’s conclusions. Randomised controlled trials included in Cochrane Reviews have used a specific risk of bias tool to assess these included studies since 2008. In 2019, a new version of this tool, Risk of Bias 2 (RoB 2), was launched to improve its usability and to reflect current understanding of how the causes of bias can influence study results. Cochrane implemented RoB 2 in a phased approach, with users of the tool informing guidance development. This paper highlights learning for all systematic reviewers (Cochrane and non-Cochrane) from the …

The Western Pacific Region (WPR) is on the front line of climate change challenges. Understanding how these challenges affect the WPR populations' mental health is essential to design effective prevention and care policies. Thus, the present study conducted an umbrella scoping review that examined the influence of climate change on mental health in the WPR, using review articles as a source of information. Ten review articles were selected according to eligibility criteria, and the findings were synthesized according to the socio-economic status of the countries identified: Australia, the Republic of Korea, the Philippines, Vietnam, the Pacific Islands (broadly), and China. The findings revealed that each country and sub-region has its own unique profile of climate change-related challenges and vulnerable populations, highlighting the need for specific approaches to mental health care. Specifically, the influence of …

BMJ Evidence-Based Medicine

BackgroundNetwork meta-analysis (NMA) is increasingly used in guideline development and other aspects of evidence-based decision-making. We aimed to develop a risk of bias (RoB) tool to assess NMAs (RoB NMA tool). An international steering committee recommended that the RoB NMA tool to be used in combination with the Risk of Bias in Systematic reviews (ROBIS) tool (i.e. because it was designed to assess biases only) or other similar quality appraisal tools (eg, A MeaSurement Tool to Assess systematic Reviews 2 [AMSTAR 2]) to assess quality of systematic reviews. The RoB NMA tool will assess NMA biases and limitations regarding how the analysis was planned, data were analysed and results were presented, including the way in which the evidence was assembled and interpreted.ObjectivesConduct (a) a Delphi process to determine expert opinion on an item’s inclusion and (b) a knowledge user …

BackgroundVaccination of infants with pneumococcal conjugate vaccines (PCV) is recommended by the World Health Organization. Evidence is mixed regarding the differences in immunogenicity and efficacy of the different pneumococcal vaccines.MethodsIn this systematic-review and network meta-analysis, we searched the Cochrane Library, Embase, Global Health, Medline, clinicaltrials.gov and trialsearch.who.int up to February 17, 2023 with no language restrictions. Studies were eligible if they presented data comparing the immunogenicity of either PCV7, PCV10 or PCV13 in head-to-head randomised trials of young children under 2 years of age, and provided immunogenicity data for at least one time point after the primary vaccination series or the booster dose. Publication bias was assessed via Cochrane's Risk Of Bias due to Missing Evidence tool and comparison-adjusted funnel plots with Egger's test …

Background The use of Mendelian randomization (MR) in epidemiology has increased considerably in recent years, with a subsequent increase in systematic reviews of MR studies. We conducted a systematic review of tools designed for assessing risk of bias and/or quality of evidence in MR studies and a review of systematic reviews of MR studies. Methods We systematically searched MEDLINE, Embase, the Web of Science, preprints servers and Google Scholar for articles containing tools for assessing, conducting and/or reporting MR studies. We also searched for systematic reviews and protocols of systematic reviews of MR studies. From eligible articles we collected data on tool characteristics and content, as well as details of narrative description of bias assessment. Results Our searches retrieved 2464 records to screen, from which 14 tools, 35 …

BackgroundIn 2019 Cochrane adopted a new tool (RoB 2) for assessing risk of bias in randomised controlled trials. The tool incorporated new advances in bias research and differed markedly from its predecessor in how it was applied. Cochrane introduced a phased implementation of the RoB 2 tool, with mentored support from the Cochrane Methods support unit (MSU) to assist authors who were new to using it. The MSU team noticed some commonly occurring misconceptions and misapplications by authors. These could potentially lead to inappropriate assessments of bias which could affect certainty in evidence.Objectives To compile a list of 10 top tips to help authors apply the RoB 2 toolMethodsFrom June 2019 to December 2020 the MSU team assessed 144 reviews and protocols and made a record of the most common misapplications and misconceptions in using RoB 2. We made them into a short list of ten useful hints and tips for any review authors, not just Cochrane authors, new to using RoB 2.ResultsThe top tips cover all domains of the RoB 2 tool and are as follows. 1) State the effect of interest 2) State the outcome to be assessed 3) Pilot the tool to achieve consistency 4) Apply the tool to a specific numerical result 5) Answer all of the signalling questions 6) Consider whether baseline imbalance reflects problems with the randomisation process rather than chance 7) Don’t assume a lack of blinding increases risk of bias 8) Participants changing interventions is not always a cause for concern 9) Avoid arbitrary thresholds for missing data and consider the analysis used 10) A” statistical analysis plan” can be a protocol or a trial register …

Background: Risk of bias (RoB) assessment of randomized clinical trials (RCTs) is vital to answering systematic review questions accurately. Manual RoB assessment for hundreds of RCTs is a cognitively demanding and lengthy process. Automation has the potential to assist reviewers in rapidly identifying text descriptions in RCTs that indicate potential risks of bias. However, no RoB text span annotated corpus could be used to fine-tune or evaluate large language models (LLMs), and there are no established guidelines for annotating the RoB spans in RCTs.Objective: The revised Cochrane RoB Assessment 2 (RoB 2) tool provides comprehensive guidelines for RoB assessment; however, due to the inherent subjectivity of this tool, it cannot be directly used as RoB annotation guidelines. Our objective was to develop precise RoB text span annotation instructions that could address this subjectivity and thus aid the corpus annotation.Methods: We leveraged RoB 2 guidelines to develop visual instructional placards that serve as text annotation guidelines for RoB spans and risk judgments. Expert annotators employed these visual placards to annotate a dataset named RoBuster, consisting of 41 full-text RCTs from the domains of physiotherapy and rehabilitation. We report inter-annotator agreement (IAA) between two expert annotators for text span annotations before and after applying visual instructions on a subset (9 out of 41) of RoBuster. We also provide IAA on bias risk judgments using Cohen's Kappa. Moreover, we utilized a portion of RoBuster (10 out of 41) to evaluate an LLM using a straightforward evaluation framework. This evaluation …