Holger Schünemann, MD, PhD, MSc, FRCPC

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RRs) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. This paper addresses considerations for rating the certainty of evidence (COE) in RRs. We recommend the full implementation of GRADE (Grading of Recommendations, Assessment, Development and Evaluation) for Cochrane RRs if time and resources allow.If time or other resources do not permit the full implementation of GRADE, the following recommendations can be considered: (1) limit rating COE to the main intervention and comparator and limit the number of outcomes to critical benefits and harms; (2) if a literature review or a Delphi approach to rate the importance of outcomes is not feasible, rely on informal judgements of knowledge …

Background Allergic rhinitis (AR) impacts patients' physical and emotional well‐being. Assessing patients' values and preferences (V&P) related to AR is an essential part of patient‐centered care and of the guideline development process. We aimed to systematically summarize the information about patients' V&P on AR and its symptoms and impact on daily life. Methods We conducted systematic review in a MEDLINE, Embase, PsychInfo, and CINAHL databases. We included studies which quantitatively assessed patients' V&P for specific outcomes in AR by assessing utilities, applying discrete choice approaches, or rating and ranking outcomes. We grouped outcomes as AR symptoms, functional status, and care‐related patient experience. Study selection and data extraction were supported by the Laser AI tool. We rated the certainty of evidence (CoE) using the GRADE approach. Results Thirty‐six studies (41 …

ObjectivesIncorporating health equity considerations into guideline development often requires information beyond that gathered through traditional evidence synthesis methodology. This article outlines an operationalization plan for the Grading of Recommendations Assessment, Development, and Evaluation (GRADE)-equity criterion to gather and assess evidence from primary studies within systematic reviews, enhancing guideline recommendations to promote equity. We demonstrate its use in a clinical guideline on medical cannabis for chronic pain.Study Design and SettingWe reviewed GRADE guidance and resources recommended by team members regarding the use of evidence for equity considerations, drafted an operationalization plan, and iteratively refined it through team discussion and feedback and piloted it on a medicinal cannabis guideline.ResultsWe propose a seven-step approach: 1) identify …

Introduction Soil-transmitted helminths (STH) are parasitic worms that infect nearly a quarter of the world's population, particularly those living in communities without access to adequate water, sanitation, and housing. Emerging evidence suggests that it may be possible to interrupt transmission of STH by deworming individuals of all ages via community-wide MDA (cMDA), as opposed to only treating children and other focal populations. Transitioning from a policy of STH control to STH elimination in targeted areas would require a fundamental shift in STH policy and programming. This policy change would require updated guidance to support countries as they adapt their current approaches for STH surveillance, supply chain management, community mobilization, and core programmatic activities in pursuit of STH elimination. There is an opportunity to engage with key stakeholders, such as program implementers and implementation partners, to understand what evidence they need to confidently adopt a new policy guideline and to deliver guideline adherent management at scale. Methods We aimed to engage with STH stakeholders to develop a Target Policy Profile (TPoP), a single document that describes optimal characteristics and evidence requirements that STH stakeholders prioritized in future potential STH transmission interruption efforts. Steps in TPoP development included a scoping review and key informant interviews (KIIs), which were used to design a two-stage Delphi technique to identify and verify TPoP components. Results The scoping review resulted in 25 articles, and 8 experts participated in KII's. Twenty respondents …

ROBVALU: A tool for assessing risk of bias in studies about peoples’ values, utilities, or the importance of health outcomes - Bangor University Privacy & Cookies Contact Us My Bangor English Cymraeg Research Portal Directory of expertise Researchers Research outputs Impact Research projects Research activities Research Data Media coverage Accolades Equipment Research Portal Research outputs ROBVALU: A tool for assessing risk of bias in studies about ... ROBVALU: A tool for assessing risk of bias in studies about peoples’ values, utilities, or the importance of health outcomes Research output: Contribution to journal › Article › peer-review Overview Cite Electronic versions Documents BMJ_ROBVALU_Manuscript_clean Accepted author manuscript, 0.98 MB, PDF document Embargo ends: 31/12/99 Samer G. Karam McMaster University, Hamilton Yuan Zang McMaster University, Hamilton Hector Pardo-…

BackgroundThere is insufficient systematised evidence on the effectiveness of individual intranasal medications in allergic rhinitis (AR).ObjectiveTo perform a systematic review to compare the efficacy of individual intranasal corticosteroids and antihistamines against placebo in improving the nasal and ocular symptoms and the rhinoconjunctivitis-related quality-of-life of patients with perennial or seasonal AR.MethodsWe searched four electronic bibliographic databases and three clinical trials databases for randomised controlled trials (i) assessing adult patients with seasonal or perennial AR and (ii) comparing the use of intranasal corticosteroids or antihistamines versus placebo. Assessed outcomes included the Total Nasal Symptom Score (TNSS), the Total Ocular Symptom Score (TOSS) and the Rhinoconjunctivitis Quality-of-Life Questionnaire (RQLQ). We performed random-effects meta-analyses of mean …

Essential medicine lists (EMLs) are important medicine prioritization tools used by the World Health Organization (WHO) EML and over 130 countries. The criteria used by WHO’s Expert Committee on the Selection and Use of Essential Medicines has parallels to the GRADE Evidence-to-Decision (EtD) frameworks. In this study, we explored the EtD frameworks and a visual abstract as adjunctive tools to strengthen the integrate evidence and improve the transparency of decisions of EML applications. We conducted user-experience testing interviews of key EML stakeholders using Morville’s honeycomb model. Interviews explored multifaceted dimensions (e.g., usability) on two EML applications for the 2021 WHO EML–long-acting insulin analogues for diabetes and immune checkpoint inhibitors for lung cancer. Using a pre-determined coding framework and thematic analysis we iteratively improved both the EtD framework and the visual abstract. We coded the transcripts of 17 interviews with 13 respondents in 103 locations of the interview texts across all dimensions of the user-experience honeycomb. Respondents felt the EtD framework and visual abstract presented complementary useful and findable adjuncts to the traditional EML application. They felt this would increase transparency and efficiency in evidence assessed by EML committees. As EtD frameworks are also used in health practice guidelines, including those by the WHO, respondents articulated that the adoption of the EtD by EML applications represents a tangible mechanism to align EMLs and guidelines, decrease duplication of work and improve coordination. Improvements …

Health guidelines impact clinical, public health and policy practice, but there is no regulation for their development, often leading to variability in quality and trustworthiness. The International Guideline Training and Certification Programme (INGUIDE), established by faculty at McMaster University in partnership and under the auspice of the Guidelines International Network (GIN), addresses this shortcoming by offering structured, evidence‐based training and certification for those involved in guidelines (inguide.org). This commentary describes INGUIDE's background, purpose, structure and significance after approximately 3 years of operation. INGUIDE's mission is to enhance the quality of health guidelines globally. It provides comprehensive training to ensure the systematic development of guidelines based on the best evidence and adherence to international quality standards, as reflected in its ISO:9001 …

BackgroundPrognostic models incorporate multiple prognostic factors to estimate the likelihood of future events for individual patients based on their prognostic factor values. Evaluating these models crucially involves conducting studies to assess their predictive performance, like discrimination. Systematic reviews and meta-analyses of these evaluation studies play an essential role in selecting models for clinical practice.MethodsIn this paper, we outline three thresholds to determine the target for certainty rating in the discrimination of prognostic models, as observed across a body of validation studies.Results and ConclusionWe propose three thresholds when rating the certainty of evidence about a prognostic model's discrimination. The first threshold amounts to rating certainty in the model's ability to classify better than random chance. The other two approaches involve setting thresholds informed by other …

ObjectivesTo make informed decisions, the general population should have access to accessible and understandable health recommendations. To compare understanding, accessibility, usability, satisfaction, intention to implement, and preference of adults provided with a digital “Plain Language Recommendation” (PLR) format vs. the original “Standard Language Version” (SLV).Study Design and SettingAn allocation-concealed, blinded, controlled superiority trial and a qualitative study to understand participant preferences. An international on-line survey. 488 adults with some English proficiency. 67.8% of participants identified as female, 62.3% were from the Americas, 70.1% identified as white, 32.2% had a bachelor's degree as their highest completed education, and 42% said they were very comfortable reading health information. In collaboration with patient partners, advisors, and the Cochrane Consumer …

ObjectivesThis article describes considerations for addressing intransitivity when assessing the certainty of the evidence from network meta-analysis (NMA) using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. Intransitivity is induced by effect modification, that is, when the magnitude of the effect between an intervention and outcome differs depending on the level of another factor.Study Design and SettingTo develop this GRADE concept paper, the lead authors conducted iterative discussions, computer simulations, and presentations to the GRADE project group and at GRADE working group meetings. The GRADE Working Group formally approved the article in July 2022.ResultsNMA authors can have a higher or a lower threshold to rate down the certainty of the evidence due to intransitivity, which depends on the extent of their concerns regarding the …

BackgroundObservational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies.ObjectiveTo develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome.Methods and resultsROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that …

ObjectivesTo provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy.Study Design and SettingA project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance.ResultsBefore rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion …

ObjectivesGuidelines and essential medicine lists (EMLs) bear similarities and differences in the process that lead to decisions. Access to essential medicines is central to achieve universal health coverage. The World Health Organization (WHO) EML has guided prioritization of essential medicines globally for nearly 50 years, and national EMLs (NEMLs) exist in over 130 countries. Guideline and EML decisions, at WHO or national levels, are not always coordinated and aligned. We sought to explore challenges, and potential solutions, for decision-making to support trustworthy medicine selection for EMLs from a Grading of Recommendations, Assessment, Development and Evaluations (GRADE) Working Group perspective. We primarily focus on the WHO EML; however, our findings may be applicable to NEML decisions as well.

Cow's milk allergy is rare in exclusively breastfed infants. To support the continuation of breastfeeding an infant after diagnosis with a cow's milk allergy, it is critical to examine the evidence for and against any form of cow's milk elimination diet for lactating mothers. In this narrative review, we highlight the lack of high-quality evidence, hence subsequent controversy, regarding whether the minuscule quantities of cow's milk proteins detectable in human milk cause infant cow's milk allergy symptoms. Current clinical practice recommendations advise a 2–4 week trial of maternal cow's milk dietary elimination for: a) IgE-mediated cow's milk allergy only if the infant is symptomatic on breastfeeding alone; b) non-IgE-mediated associated symptoms only if the history and examination strongly suggest cow's milk allergy; and c) infants with moderate to severe eczema/atopic dermatitis, unresponsive to topical steroids and …

This editorial aims to inform the scientific community on the initiative set forth by the European Academy of Allergy and Clinical Immunology (EAACI)-Respiratory Effectiveness Group (REG) on March 15, 2023, in Lisbon.

IntroductionThis updated guidance from the Grading of Recommendations Assessment, Development, and Evaluation addresses rating up certainty of evidence due to a dose-response gradient (DRG) observed in synthesis of intervention and exposure studies.Study Design and SettingThis guidance was developed using iterative discussions and consensus in multiple meetings and was presented to attendees of the Grading of Recommendations Assessment, Development, and Evaluation Working Group meeting for feedback in November 2022 and for final approval in May 2023.ResultsThe guidance consists of two steps. The first is to determine whether the DRG is credible. We describe five items for assessing credibility: a) is DRG identified using a proper analytical approach; b) is confounding the cause of the DRG; c) is there serious concern about ecological bias; d) is the DRG consistent across studies; and e …

Nonrandomized studies (NRS) on allergen immunotherapy (AIT) particularly lend themselves to evaluate outcomes that are insufficiently addressed in randomized controlled studies (RCTs). However, NRS are prone to several sources of bias, which limit their validity. We aimed at comparing AIT effects between RCTs and NRS and evaluate the reasons for discrepancies in study results. In this analysis, we compared NRS on AIT (including subcutaneous and sublingual immunotherapy, SCIT and SLIT, respectively) with SLIT and SCIT RCTs from published meta‐analyses, assessing the Risk of Bias (RoB) for each study and the certainty of evidence from NRS and RCTs using the GRADE approach. We found: (1) very serious RoB in the 7 NRS included in the meta‐analysis showing a large difference between AIT and controls (standardized mean difference [SMD] for symptom score [SS], −1.77; 95% CI, −2.30, −1.24 …

Introduction Data from mHealth apps can provide valuable information on rhinitis control and treatment patterns. However, in MASK‐air®, these data have only been analyzed cross‐sectionally, without considering the changes of symptoms over time. We analyzed data from MASK‐air® longitudinally, clustering weeks according to reported rhinitis symptoms. Methods We analyzed MASK‐air® data, assessing the weeks for which patients had answered a rhinitis daily questionnaire on all 7 days. We firstly used k‐means clustering algorithms for longitudinal data to define clusters of weeks according to the trajectories of reported daily rhinitis symptoms. Clustering was applied separately for weeks when medication was reported or not. We compared obtained clusters on symptoms and rhinitis medication patterns. We then used the latent class mixture model to assess the robustness of results. Results We analyzed …

Background Venous thromboembolism (VTE) is the third most common vascular disease. Medical inpatients, long-term care residents, persons with minor injuries, and long-distance travelers are at increased risk. Objective These evidence-based guidelines from the American Society of Hematology (ASH) intend to support patients, clinicians, and others in decisions about preventing VTE in these groups. Methods ASH formed a multidisciplinary guideline panel balanced to minimize potential bias from conflicts of interest. The McMaster University GRADE Centre supported the guideline-development process, including updating or performing systematic evidence reviews. The panel prioritized clinical questions and outcomes according to their importance for clinicians and adult patients. The Grading of Recommendations Assessment, Development and …