Harry Hochheiser

Background Coronavirus Disease 2019 (COVID-19) continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear. Here, we present projections of COVID-19 hospitalizations and deaths in the United States for the next 2 years under 2 plausible assumptions about immune escape (20% per year and 50% per year) and 3 possible CDC recommendations for the use of annually reformulated vaccines (no recommendation, vaccination for those aged 65 years and over, vaccination for all eligible age groups based on FDA approval). Methods and findings The COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023 and April 15, 2025 under 6 scenarios representing the intersection of considered levels of immune escape and vaccination. Annually reformulated vaccines are assumed to be 65% effective against symptomatic infection with strains circulating on June 15 of each year and to become available on September 1. Age- and state-specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster. State and national projections from 8 modeling teams were ensembled to produce projections for each scenario and expected reductions in disease outcomes due to vaccination over the projection period. From April 15, 2023 to April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November to January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million …

Between December 2020 and April 2023, the COVID-19 Scenario Modeling Hub (SMH) generated operational multi-month projections of COVID-19 burden in the US to guide pandemic planning and decision-making in the context of high uncertainty. This effort was born out of an attempt to coordinate, synthesize and effectively use the unprecedented amount of predictive modeling that emerged throughout the COVID-19 pandemic. Here we describe the history of this massive collective research effort, the process of convening and maintaining an open modeling hub active over multiple years, and attempt to provide a blueprint for future efforts. We detail the process of generating 17 rounds of scenarios and projections at different stages of the COVID-19 pandemic, and disseminating results to the public health community and lay public. We also highlight how SMH was expanded to generate influenza projections …

Transfer learning has become a pivotal technique in machine learning, renowned for its effectiveness in various real-world applications. However, a significant challenge arises when applying this approach to sequential epidemiological data, often characterized by a scarcity of labeled information. To address this challenge, we introduce Predictive Volume-Adaptive Weighting (PVAW), a novel online multi-source transfer learning method. PVAW innovatively implements a dynamic weighting mechanism within an ensemble model, allowing for the automatic adjustment of weights based on the relevance and contribution of each source and target model. We demonstrate the effectiveness of PVAW through its application in analyzing Respiratory Syncytial Virus (RSV) data, collected over multiple seasons at the University of Pittsburgh Medical Center. Our method showcases significant improvements in model performance over existing baselines, highlighting the potential of online transfer learning in handling complex, sequential data. This study not only underscores the adaptability and sophistication of transfer learning in healthcare but also sets a new direction for future research in creating advanced predictive models.

Methods: This is a retrospective cohort study using the Pediatric Health Information Systems database. Centers with> 500 admissions for children with SLT, both initial and subsequent encounters, from 1/1/15-12/31/22 were included. Patient demographics, ABX days, survival, and predicted mortality were queried. Non-parametric continuous data were compared with the Wilcoxon rank sum test, categorical data with the Chi-squared test, and correlation was assessed with linear regression. All statistical analyses were performed with R.Results: There were 222,193 PICU encounters across 9 hospitals, of which 6,032/222,193 (2.7%) had prior SLT. Most patients with SLT were male (3,295 [54.6%]), significantly older (10 [3, 6] vs 4 [1, 12] years), and more likely to have cardiovascular, gastrointestinal, oncologic, metabolic, or urologic comorbidities compared to children without SLT (all P< 0.001). SLT recipients had …

RationaleThe American Academy of Neurology (AAN) recommends annual sexual and reproductive health (SRH) counseling for all people with epilepsy of gestational capacity (PWEGC). Child neurologists report discussing SRH concerns infrequently with adolescents. Limited research exists regarding documentation of such counseling.MethodsWe retrospectively studied clinical notes using natural language processing to investigate child neurologists’ documentation of SRH counseling for adolescent and young adult PWEGC. We segmented notes into sentences and evaluated for references to menstruation, sexual activity, contraception, folic acid, teratogens, and pregnancy. We developed training sets in a labeling application and used machine learning to identify additional counseling instances. We repeated this iteratively until we identified no new relevant sentences. We validated results using external …

PURPOSEManual extraction of case details from patient records for cancer surveillance is a resource-intensive task. Natural Language Processing (NLP) techniques have been proposed for automating the identification of key details in clinical notes. Our goal was to develop NLP application programming interfaces (APIs) for integration into cancer registry data abstraction tools in a computer-assisted abstraction setting.METHODSWe used cancer registry manual abstraction processes to guide the design of DeepPhe-CR, a web-based NLP service API. The coding of key variables was performed through NLP methods validated using established workflows. A container-based implementation of the NLP methods and the supporting infrastructure was developed. Existing registry data abstraction software was modified to include results from DeepPhe-CR. An initial usability study with data registrars provided early …

Artificial intelligence (AI) and machine learning (ML) facilitate the creation of revolutionary medical techniques. Unfortunately, biases in current AI and ML approaches are perpetuating minority health inequity. One of the strategies to solve this problem is training a diverse workforce. For this reason, we created the course “Artificial Intelligence and Machine Learning applied to Health Disparities Research (AIML + HDR)” which applied general Data Science (DS) approaches to health disparities research with an emphasis on Hispanic populations. Some technical topics covered included the Jupyter Notebook Framework, coding with R and Python to manipulate data, and ML libraries to create predictive models. Some health disparities topics covered included Electronic Health Records, Social Determinants of Health, and Bias in Data. As a result, the course was taught to 34 selected Hispanic participants and evaluated by a survey on a Likert scale (0–4). The surveys showed high satisfaction (more than 80% of participants agreed) regarding the course organization, activities, and covered topics. The students strongly agreed that the activities were relevant to the course and promoted their learning (3.71 ± 0.21). The students strongly agreed that the course was helpful for their professional development (3.76 ± 0.18). The open question was quantitatively analyzed and showed that seventy-five percent of the comments received from the participants confirmed their great satisfaction.

Methods: Using the MIMIC dataset, we collected structured clinical data on 4,630 ICU admissions for both upper and lower gastrointestinal hemorrhage from 2008-2019. Observations were discretized to 1-hour time intervals, and K-mean clustering was performed for patient state representation. A Gastrointestinal Bleeding Reinforcement Learning agent, GaBRieL, was trained to recommend blood product administration using offline Q-learning and evaluated on a 20% held-out validation set. At a given timestep, blood product administration was treated as a binary decision for each of packed red blood cells (PRBC), fresh frozen plasma (FFP), and platelets (Plts).Results: When compared with clinicians acting on unseen validation data, GaBRieL recommended transfusion of PRBC at higher hemoglobin levels [8.91 vs 8.06, p<. 0001], FFP at lower INR levels [1.61 vs 2.23, p<. 0001], and Plts at higher platelet levels …

Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single …

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the US COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of …

BackgroundTranscranial magnetic stimulation (TMS) is a painless non-invasive method that allows focal activation or deactivation of a human brain region in order to assess effects on other brain regions. As such, it has a unique role in elucidating brain connectivity during behavior and at rest. Information regarding brain connectivity derived from TMS experiments has been published in hundreds of papers but is not accessible in aggregate.ObjectiveOur objective was to identify, extract, and represent TMS-connectivity data in a graph database. This approach uses nodes connected by edges to capture the directed nature of interregional communication in the brain while also being flexible enough to contain other information about the connections, such as the source of information and details about the experiments that produced them.MethodsData related to interregional brain connectivity is first extracted from full-text publications, with creation of a table-like structure that list data of multiple types, principally the source and target brain regions, sign (excitatory/inhibitory) and latency. While machine-reading methods were explored, so far human experts have had to extract and verify data. These data are used to populate a neo4j graph database. A graphical user interface coupled with a query system allows users to search for networks and display information about connections between any two brain regions of interest.ResultsExperiments involving two TMS stimulating coils, in which one is over a putative source region and the other is over another region with a measurable effect in the body (such as the primary motor cortex) are the most …

BackgroundThe COVID-19 Scenario Modeling Hub convened nine modeling teams to project the impact of expanding SARS-CoV-2 vaccination to children aged 5–11 years on COVID-19 burden and resilience against variant strains.MethodsTeams contributed state- and national-level weekly projections of cases, hospitalizations, and deaths in the United States from September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of 1) vaccination (or not) of children aged 5–11 years (starting November 1, 2021), and 2) emergence (or not) of a variant more transmissible than the Delta variant (emerging November 15, 2021). Individual team projections were linearly pooled. The effect of childhood vaccination on overall and age-specific outcomes was estimated using meta-analyses.FindingsAssuming that a new variant would not emerge, all-age COVID-19 outcomes were projected to decrease …

BackgroundInfectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications.MethodsFour reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 30th, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss’ kappa …

Software is vital for the advancement of biology and medicine. Through analysis of usage and impact metrics of software, developers can help determine user and community engagement. These metrics can be used to justify additional funding, encourage additional use, and identify unanticipated use cases. Such analyses can help define improvement areas and assist with managing project resources. However, there are challenges associated with assessing usage and impact, many of which vary widely depending on the type of software being evaluated. These challenges involve issues of distorted, exaggerated, understated, or misleading metrics, as well as ethical and security concerns. More attention to the nuances, challenges, and considerations involved in capturing impact across the diverse spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small …

It would be highly desirable to have a tool that detects the outbreak of a new influenza-like illness, such as COVID-19, accurately and early. This paper describes the ILI Tracker algorithm that first models the daily occurrence of a set of known influenza-like illnesses in a hospital emergency department using findings extracted from patient-care reports using natural language processing. We include results based on modeling the diseases influenza, respiratory syncytial virus, human metapneumovirus, and parainfluenza for five emergency departments in Allegheny County Pennsylvania from June 1, 2010 through May 31, 2015. We then show how the algorithm can be extended to detect the presence of an unmodeled disease which may represent a novel disease outbreak. We also include results for detecting an outbreak of an unmodeled disease during the mentioned time period, which in retrospect was very likely an outbreak of Enterovirus D68.

BackgroundPrevious studies have shown that patients with CKD are less likely than those without CKD to receive invasive care to treat acute coronary syndrome (ACS). However, few studies have accounted for whether such care was clinically indicated or assessed whether nonuse of such care was associated with adverse health outcomes.MethodsWe conducted a retrospective cohort study of US veterans who were hospitalized at Veterans Affairs Medical Centers from January 2013 through December 2017 and received a discharge diagnosis of ACS. We used multivariable logistic regression to investigate the association of CKD with use of invasive care (coronary angiography, with or without revascularization; coronary artery bypass graft surgery; or both) deemed clinically indicated based on Global Registry of Acute Coronary Events 2.0 risk scores that denoted a 6-month predicted all-cause mortality≥ 5 …

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario …

MethodsWe attempted to predict the need for operation and bowel resection in small bowel obstruction from both institutional data and the MIMIC dataset, using a variety of standard machine learning models as well as neural networking. Under a Markovian model of sequential decision-making, we have also attempted to build reinforcement learning models for patients admitted with planned non-operative management to identify early failure.ResultsIn both datasets, our best AUROC for predicting need for both operation and bowel resection is around 0.6–0.65. Performance varied little across models, with little predictive benefit to any model above that of standard logistic regression. In attempting to implement RL, we have faced several challenges, including poor data collection in non-critical care environments, difficulty assessing timing of surgical decision-making, and nontrivial design of a reward structure …

PurposeReal-world evidence for radiation therapy (RT) is limited because it is often documented only in the clinical narrative. We developed a natural language processing system for automated extraction of detailed RT events from text to support clinical phenotyping.Methods and MaterialsA multi-institutional data set of 96 clinician notes, 129 North American Association of Central Cancer Registries cancer abstracts, and 270 RT prescriptions from HemOnc.org was used and divided into train, development, and test sets. Documents were annotated for RT events and associated properties: dose, fraction frequency, fraction number, date, treatment site, and boost. Named entity recognition models for properties were developed by fine-tuning BioClinicalBERT and RoBERTa transformer models. A multiclass RoBERTa-based relation extraction model was developed to link each dose mention with each property in the …

Objectives This study aimed to enable clinical researchers without expertise in natural language processing (NLP) to extract and analyze information about sexual and reproductive health (SRH), or other sensitive health topics, from large sets of clinical notes. Methods (1) We retrieved text from the electronic health record as individual notes. (2) We segmented notes into sentences using one of scispaCy's NLP toolkits. (3) We exported sentences to the labeling application Watchful and annotated subsets of these as relevant or irrelevant to various SRH categories by applying a combination of regular expressions and manual annotation. (4) The labeled sentences served as training data to create machine learning models for classifying text; specifically, we used spaCy's default text classification ensemble, comprising a bag-of-words model and a neural network with attention. (5) We applied each model to …