Lifeng Lin

ObjectiveTo apply a hierarchical model (HM) that addresses measurement error in regression of the treatment effect on the control group event rate (CR). We compare HM to weighted linear regression (WLR) which is subject to measurement error and mathematical coupling.Study design and settingWe reviewed published hierarchical models that address measurement error and implemented a Bayesian version in open-source code to facilitate adoption by meta-analysts. We compared WLR and HM across a very large convenience sample of meta-analyses published in the Cochrane Database of Systematic Reviews.ResultsWe applied both approaches (WLR and a HM that addresses measurement error) to 3,193 meta-analyses that included 33,071 studies (average 10.28 studies per meta-analysis). A statistically significant slope suggesting an association between the treatment effect and CR was demonstrated …

ObjectiveSeveral systematic reviews and meta-analyses have been conducted to summarize the evidence for the efficacy of various labor induction agents. However, the most effective agents or strategies have not been conclusively determined. We aimed to perform a meta-review and network meta-analysis of published systematic reviews to determine the efficacy and safety of currently employed pharmacologic, mechanical, and combined methods of labor induction.Data SourcesWith the assistance of an experienced medical librarian, we performed a systematic search of the literature using PubMed, EMBASE, and the Cochrane Central Register of Control Trials. We systematically searched electronic databases from inception to May 31, 2021.Study Eligibility CriteriaWe considered systematic reviews and meta-analyses of randomized controlled trials comparing different agents or methods for inpatient labor …

BackgroundEmpirical evidence suggests that lack of blinding may be associated with biased estimates of treatment benefit in randomized controlled trials, but the influence on medication-related harms is not well-recognized. We aimed to investigate the association between blinding and clinical trial estimates of medication-related harms.MethodsWe searched PubMed from January 1, 2015, till January 1, 2020, for systematic reviews with meta-analyses of medication-related harms. Eligible meta-analyses must have contained trials both with and without blinding. Potential covariates that may confound effect estimates were addressed by restricting trials within the comparison or by hierarchical analysis of harmonized groups of meta-analyses (therefore harmonizing drug type, control, dosage, and registration status) across eligible meta-analyses. The weighted hierarchical linear regression was then used to estimate …

<?show [AQ ID=GQ2 POS=22pt]?><?show [AQ ID=GQ4]?><?show [AQ ID=GQ5 POS=12pt]?>Assessing heterogeneity between studies is a critical step in determining whether studies can be combined and whether the synthesized results are reliable. The statistic has been a popular measure for quantifying heterogeneity, but its usage has been challenged from various perspectives in recent years. In particular, it should not be considered an absolute measure of heterogeneity, and it could be subject to large uncertainties. As such, when using to interpret the extent of heterogeneity, it is essential to account for its interval estimate. Various point and interval estimators exist for . This article summarizes these estimators. In addition, we performed a simulation study under different scenarios to investigate preferable point and interval estimates of . We found that the Sidik–Jonkman method gave precise point estimates for when …

Based on 52 studies with samples mostly from English-speaking countries, the current study used Bayesian network meta-analysis to investigate the intervention effectiveness of different reading comprehension strategy combinations on reading comprehension among students with reading difficulties in 3rd through 12th grade. We focused on commonly researched strategies: main idea, inference, text structure, retell, prediction, self-monitoring, and graphic organizers. Results showed (1) instruction of more strategies did not necessarily have stronger effects on reading comprehension; (2) there was no single reading comprehension strategy that produced the strongest effect; (3) main idea, text structure, and retell, taught together as the primary strategies, seemed the most effective; and (4) the effects of strategies only held when background knowledge instruction was included. These findings suggest strategy …

ObjectiveThis systematic review and meta-analysis aimed to conduct a thorough and contemporary assessment of maternal and neonatal outcomes associated with water birth in comparison with land-based birth.Data SourcesWe conducted a comprehensive search of PubMed, EMBASE, CINAHL, and gray literature sources, from inception to February 28, 2023.Study Eligibility CriteriaWe included randomized and nonrandomized studies that assessed maternal and neonatal outcomes in patients who delivered either conventionally or while submerged in water.MethodsPooled unadjusted odds ratios with 95% confidence intervals were calculated using a random-effects model (restricted maximum likelihood method). We assessed the 95% prediction intervals to estimate the likely range of future study results. To evaluate the robustness of the results, we calculated fragility indices. Maternal infection was designated …

Objective To evaluate the reliability of three artificial intelligence (AI) chatbots (ChatGPT, Google Bard, and Chatsonic) in generating accurate references from existing obstetric literature. Study Design Between mid-March and late April 2023, ChatGPT, Google Bard, and Chatsonic were prompted to provide references for specific obstetrical randomized controlled trials (RCTs) published in 2020. RCTs were considered for inclusion if they were mentioned in a previous article that primarily evaluated RCTs published by the top medical and obstetrics and gynecology journals with the highest impact factors in 2020 as well as RCTs published in a new journal focused on publishing obstetric RCTs. The selection of the three AI models was based on their popularity, performance in natural language processing, and public availability. Data collection involved prompting the AI chatbots to provide references according to …

Network meta‐analysis (NMA) is a statistical procedure to simultaneously compare multiple interventions. Despite the added complexity of performing an NMA compared with the traditional pairwise meta‐analysis, under proper assumptions the NMA can lead to more efficient estimates on the comparisons of interventions by combining and contrasting the direct and indirect evidence into a form of evidence that can be used to underpin treatment guidelines. Two broad classes of NMA methods are commonly used in practice: the contrast‐based (CB‐NMA) and the arm‐based (AB‐NMA) models. While CB‐NMA only focuses on the relative effects by assuming fixed intercepts, the AB‐NMA offers greater flexibility on the estimands, including both the absolute and relative effects by assuming random intercepts. A major criticism of the AB‐NMA, on which we aim to elaborate in this paper, is that it does not retain …

When conducting systematic reviews and meta-analyses of continuous outcomes, the mean differences (MDs) and standardized mean differences (SMDs) are 2 commonly used choices for effect measures. The SMDs are motivated by scenarios where studies collected in a systematic review do not report the continuous measures on the same scale. The standardization process transfers the MDs to be unit-free measures that can be synthesized across studies. As such, some evidence synthesis researchers tend to prefer the SMD over the MD. However, other researchers have concerns about the interpretability of the SMD. The standardization process could also yield additional heterogeneity between studies. In this paper, we use simulation studies to illustrate that, in a scenario where the continuous measures are on the same scale, the SMD could have considerably poorer performance compared with the MD in …

Meta‐analysis is a widely used tool for synthesizing results from multiple studies. The collected studies are deemed heterogeneous when they do not share a common underlying effect size; thus, the factors attributable to the heterogeneity need to be carefully considered. A critical problem in meta‐analyses and systematic reviews is that outlying studies are frequently included, which can lead to invalid conclusions and affect the robustness of decision‐making. Outliers may be caused by several factors such as study selection criteria, low study quality, small‐study effects, and so on. Although outlier detection is well‐studied in the statistical community, limited attention has been paid to meta‐analysis. The conventional outlier detection method in meta‐analysis is based on a leave‐one‐study‐out procedure. However, when calculating a potentially outlying study's deviation, other outliers could substantially impact its …

We provide comparisons between methods to assess replicability (Appendix A, referenced in Section 1), proofs of theoretical results (Appendix B, referenced in Section 3.4), details of simulation designs (Appendix C, referenced in Section 4.1), additional numerical results (Appendix D, referenced in Section 4.2), and additional case studies (Appendix E, referenced in Section 5).

Medicare beneficiaries’ healthcare spending varies across geographical regions, influenced by availability of medical resources and institutional efficiency. We aimed to evaluate whether social vulnerability influences healthcare costs among Medicare beneficiaries. Multivariable regression analyses were conducted to determine whether the social vulnerability index (SVI), released by the CDC, was associated with average submitted covered charges, total payment amounts, or total covered days upon hospital discharge among Medicare beneficiaries. We used information from discharged Medicare beneficiaries from hospitals participating in the Inpatient Prospective Payment System. Covariate adjustment included demographic information consisting of age groups, race/ethnicity, and Hierarchical Condition Category risk score. The regressions were performed with weights proportioned to the number of …

ObjectiveSeveral systematic reviews and meta-analyses have summarized the evidence on the efficacy and safety of various outpatient cervical ripening methods. However, the method with the highest efficacy and safety profile has not been determined conclusively. We performed a systematic review and network meta-analysis of published randomized controlled trials to assess the efficacy and safety of cervical ripening methods currently employed in the outpatient setting.Data SourcesWith the assistance of an experienced medical librarian, we performed a systematic search of the literature using MEDLINE, Embase, Scopus, Web of Science, Cochrane Library, and ClinicalTrials.gov. We systematically searched electronic databases from inception to January 14, 2020.Study Eligibility CriteriaWe considered randomized controlled trials comparing a variety of methods for outpatient cervical ripening.MethodsWe …

Objectives COVID-19, since its emergence in December 2019, has globally impacted research. Over 360 000 COVID-19-related manuscripts have been published on PubMed and preprint servers like medRxiv and bioRxiv, with preprints comprising about 15% of all manuscripts. Yet, the role and impact of preprints on COVID-19 research and evidence synthesis remain uncertain. Materials and Methods We propose a novel data-driven method for assigning weights to individual preprints in systematic reviews and meta-analyses. This weight termed the “confidence score” is obtained using the survival cure model, also known as the survival mixture model, which takes into account the time elapsed between posting and publication of a preprint, as well as metadata such as the number of first 2-week citations, sample size, and study type. Results Using 146 …

This supplementary file includes an algorithm to improve efficiency when within-study covariance matrices are reported (Appendix A); the detailed form of the variance estimator of the proposed method (Appendix B); the estimation algorithm and the variance estimator of the model assuming equal between-study variances across treatment comparisons (Appendix C); the modified formulation of the model to relax the consistency assumption (Appendix D); an example to show the limitation of the radar plot (Appendix E), which motivated our proposed origami plot; additional results for the labor induction NMA (Appendix F); regularity conditions and the sketch proof of Theorem 1 (Appendix G); additional lemmas with proofs about the convergence properties of the proposed algorithm (Appendix H); additional simulation results (Appendix I); and some simple calculations (Appendix J).

BackgroundStudies included in a meta-analysis are often heterogeneous. The traditional random-effects models assume their true effects to follow a normal distribution, while it is unclear if this critical assumption is practical. Violations of this between-study normality assumption could lead to problematic meta-analytical conclusions. We aimed to empirically examine if this assumption is valid in published meta-analyses.MethodsIn this cross-sectional study, we collected meta-analyses available in the Cochrane Library with at least 10 studies and with between-study variance estimates > 0. For each extracted meta-analysis, we performed the Shapiro–Wilk (SW) test to quantitatively assess the between-study normality assumption. For binary outcomes, we assessed between-study normality for odds ratios (ORs), relative risks (RRs), and risk differences (RDs). Subgroup analyses based on sample sizes and event …

Trading off benefits and harms requires knowledge of the absolute risk reduction or risk difference, making risk difference a critical measure for decision making. The confidence interval of risk difference is the basis for imprecision judgments made by guideline developers. Estimating risk difference is not straightforward, however, and the available methods have various limitations. Four methods are discussed in this article. The major limitation of the first method, pooling of risk differences generated from multiple studies in a meta-analysis, is the inconsistency of risk difference across baseline risks. The major limitation of the second method, transforming a pooled relative effect (such as a risk ratio or odds ratio) into a risk difference, is that its confidence interval does not incorporate uncertainty in the baseline risk. This confidence interval widens in a linear fashion as the baseline risk increases, making risk difference …

IntroductionThis updated guidance from the Grading of Recommendations Assessment, Development, and Evaluation addresses rating up certainty of evidence due to a dose-response gradient (DRG) observed in synthesis of intervention and exposure studies.Study Design and SettingThis guidance was developed using iterative discussions and consensus in multiple meetings and was presented to attendees of the Grading of Recommendations Assessment, Development, and Evaluation Working Group meeting for feedback in November 2022 and for final approval in May 2023.ResultsThe guidance consists of two steps. The first is to determine whether the DRG is credible. We describe five items for assessing credibility: a) is DRG identified using a proper analytical approach; b) is confounding the cause of the DRG; c) is there serious concern about ecological bias; d) is the DRG consistent across studies; and e …

Risk difference is a frequently-used effect measure for binary outcomes. In a meta-analysis, commonly-used methods to synthesize risk differences include: (1) the two-step methods that estimate study-specific risk differences first, then followed by the univariate common-effect model, fixed-effects model, or random-effects models; and (2) the one-step methods using bivariate random-effects models to estimate the summary risk difference from study-specific risks. These methods are expected to have similar performance when the number of studies is large and the event rate is not rare. However, studies with zero events are common in meta-analyses, and bias may occur with the conventional two-step methods from excluding zero-event studies or using an artificial continuity correction to zero events. In contrast, zero-event studies can be included and modeled by bivariate random-effects models in a single step. This …

ObjectiveStudies with statistically significant results are frequently more likely to be published than those with non-significant results. This phenomenon leads to publication bias or small-study effects and can seriously affect the validity of the conclusion from systematic reviews and meta-analyses. Small-study effects typically appear in a specific direction, depending on whether the outcome of interest is beneficial or harmful, but this direction is rarely taken into account in conventional methods.MethodsWe propose to use directional tests to assess potential small-study effects. The tests are built on a one-sided testing framework based on the existing Egger's regression test. We performed simulation studies to compare the proposed one-sided regression tests, conventional two-sided regression tests, as well as two other competitive methods (Begg's rank test and the trim-and-fill method). Their performance was …