Deborah Estrin

From ELIZA to Alexa, Conversational Agents (CAs) have been deliberately designed to elicit or project empathy. Although empathy can help technology better serve human needs, it can also be deceptive and potentially exploitative. In this work, we characterize empathy in interactions with CAs, highlighting the importance of distinguishing evocations of empathy between two humans from ones between a human and a CA. To this end, we systematically prompt CAs backed by large language models (LLMs) to display empathy while conversing with, or about, 65 distinct human identities, and also compare how different LLMs display or model empathy. We find that CAs make value judgments about certain identities, and can be encouraging of identities related to harmful ideologies (eg, Nazism and xenophobia). Moreover, a computational approach to understanding empathy reveals that despite their ability to display empathy, CAs do poorly when interpreting and exploring a user’s experience, contrasting with their human counterparts.

Background: Digital health-tracking tools are changing mental health care by giving patients the ability to collect passively measured patient-generated health data (PGHD; ie, data collected from connected devices with little to no patient effort). Although there are existing clinical guidelines for how mental health clinicians should use more traditional, active forms of PGHD for clinical decision-making, there is less clarity on how passive PGHD can be used.Objective: We conducted a qualitative study to understand mental health clinicians’ perceptions and concerns regarding the use of technology-enabled, passively collected PGHD for clinical decision-making. Our interviews sought to understand participants’ current experiences with and visions for using passive PGHD.Methods: Mental health clinicians providing outpatient services were recruited to participate in semistructured interviews. Interview recordings were deidentified, transcribed, and qualitatively coded to identify overarching themes.Results: Overall, 12 mental health clinicians (n= 11, 92% psychiatrists and n= 1, 8% clinical psychologist) were interviewed. We identified 4 overarching themes. First, passive PGHD are patient driven—we found that current passive PGHD use was patient driven, not clinician driven; participating clinicians only considered passive PGHD for clinical decision-making when patients brought passive data to clinical encounters. The second theme was active versus passive data as subjective versus objective data—participants viewed the contrast between active and passive PGHD as a contrast between interpretive data on patients’ mental health and objective …

We focus on the stories of five older adults who became voice assistant users through our study, and with whom we speculated about future interfaces through two design probes, one for health data reporting and one for positive reminiscing. We delivered a voice-first ambient interface (VFAI) to each participant, and closely observed participants’ journeys through periodic themed interviews (16 hours, 21 minutes of transcribed recordings), usage log reviews (4,657 entries), and phone and text support. Participants’ lived experiences impacted their perceptions and interactions with their VFAI, fueling rich insights about how to design for diverse needs. For example, while one participant saw increased potential in the VFAI after interacting with the design probe for health data reporting, another was skeptical of using it to communicate with her doctor. We contribute an in-depth exploration of VFAIs to support aging in …

Background Over the last decade, augmented reality (AR) has emerged in health care as a tool for visualizing data and enhancing simulation learning. AR, which has largely been explored for communication and collaboration in nonhealth contexts, could play a role in shaping future remote medical services and training. This review summarized existing studies implementing AR in real-time telemedicine and telementoring to create a foundation for health care providers and technology developers to understand future opportunities in remote care and education. Objective This review described devices and platforms that use AR for real-time telemedicine and telementoring, the tasks for which AR was implemented, and the ways in which these implementations were evaluated to identify gaps in research that provide opportunities for further study. Methods We searched PubMed, Scopus, Embase, and MEDLINE to identify English-language studies published between January 1, 2012, and October 18, 2022, implementing AR technology in a real-time interaction related to telemedicine or telementoring. The search terms were “augmented reality” OR “AR” AND “remote” OR “telemedicine” OR “telehealth” OR “telementoring.” Systematic reviews, meta-analyses, and discussion-based articles were excluded from analysis. Results A total of 39 articles met the inclusion criteria and were categorized into themes of patient evaluation, medical intervention, and education. In total, 20 devices and platforms using AR were identified, with common features being the ability for remote …

Background Augmented reality (AR) and virtual reality (VR) have increasingly appeared in the medical literature in the past decade, with AR recently being studied for its potential role in remote health care delivery and communication. Recent literature describes AR’s implementation in real-time telemedicine contexts across multiple specialties and settings, with remote emergency services in particular using AR to enhance disaster support and simulation education. Despite the introduction of AR in the medical literature and its potential to shape the future of remote medical services, studies have yet to investigate the perspectives of telemedicine providers regarding this novel technology. Objective This study aimed to understand the applications and challenges of AR in telemedicine anticipated by emergency medicine providers with a range of experiences in using telemedicine and AR or VR technology. Methods Across 10 academic medical institutions, 21 emergency medicine providers with variable exposures to telemedicine and AR or VR technology were recruited for semistructured interviews via snowball sampling. The interview questions focused on various potential uses of AR, anticipated obstacles that prevent its implementation in the telemedicine area, and how providers and patients might respond to its introduction. We included video demonstrations of a prototype using AR during the interviews to elicit more informed and complete insights regarding AR’s potential in remote health care. Interviews were transcribed and analyzed via thematic coding. Results …

Digital biomarkers of mental health, created using data extracted from everyday technologies including smartphones, wearable devices, social media and computer interactions, have the opportunity to revolutionise mental health diagnosis and treatment by providing near-continuous unobtrusive and remote measures of behaviours associated with mental health symptoms. Machine learning models process data traces from these technologies to identify digital biomarkers. In this editorial, we caution clinicians against using digital biomarkers in practice until models are assessed for equitable predictions (‘model equity’) across demographically diverse patients at scale, behaviours over time, and data types extracted from different devices and platforms. We posit that it will be difficult for any individual clinic or large-scale study to assess and ensure model equity and alternatively call for the creation of a repository of …

Digital health advances have transformed many clinical areas including psychiatric and cardiovascular care. However, digital health innovation is relatively nascent in cancer care, which represents the fastest growing area of health-care spending. Opportunities for digital health innovation in oncology include patient-facing technologies that improve patient experience, safety, and patient-clinician interactions; clinician-facing technologies that improve their ability to diagnose pathology and predict adverse events; and quality of care and research infrastructure to improve clinical workflows, documentation, decision support, and clinical trial monitoring. The COVID-19 pandemic and associated shifts of care to the home and community dramatically accelerated the integration of digital health technologies into virtually every aspect of oncology care. However, the pandemic has also exposed potential flaws in the …

Older adults around the world lack access to a wide range of potentially life-changing digital applications, services, and information that could be provided by voice assistants (such as Amazon’s Alexa, Google’s Assistant, or Apple’s Siri). However, older adults’ needs are underrepresented in the design of voice assistants. Because of this, we are missing opportunities for digital inclusion, and increasing risks of excluding older adults as these devices permeate public settings. In this work, we video record older adults (n=26) interacting with a multi-modal voice assistants while waiting in line at food pantries, and use Interaction Analysis to draw insights from these recordings. We find that by being agnostic to body language, audio-prosodic features, and other contextual factors, voice assistants fail to capture and react to some important aspects of interactions. We discuss design (e.g, interpreting users’ posture as a …