Saibal Mukhopadhyay

Recurrent Spiking Neural Networks (RSNNs) have emerged as a computationally efficient and brain-inspired learning model. The design of sparse RSNNs with fewer neurons and synapses helps reduce the computational complexity of RSNNs. Traditionally, sparse SNNs are obtained by first training a dense and complex SNN for a target task, and, then, pruning neurons with low activity (activity-based pruning) while maintaining task performance. In contrast, this paper presents a task-agnostic methodology for designing sparse RSNNs by pruning a large randomly initialized model. We introduce a novel Lyapunov Noise Pruning (LNP) algorithm that uses graph sparsification methods and utilizes Lyapunov exponents to design a stable sparse RSNN from a randomly initialized RSNN. We show that the LNP can leverage diversity in neuronal timescales to design a sparse Heterogeneous RSNN (HRSNN). Further, we show that the same sparse HRSNN model can be trained for different tasks, such as image classification and temporal prediction. We experimentally show that, in spite of being task-agnostic, LNP increases computational efficiency (fewer neurons and synapses) and prediction performance of RSNNs compared to traditional activity-based pruning of trained dense models.

This paper presents the first systematic study of the evaluation of Deep Neural Networks (DNNs) for discrete dynamical systems under stochastic assumptions, with a focus on wildfire prediction. We develop a framework to study the impact of stochasticity on two classes of evaluation metrics: classification-based metrics, which assess fidelity to observed ground truth (GT), and proper scoring rules, which test fidelity-to-statistic. Our findings reveal that evaluating for fidelity-to-statistic is a reliable alternative in highly stochastic scenarios. We extend our analysis to real-world wildfire data, highlighting limitations in traditional wildfire prediction evaluation methods, and suggest interpretable stochasticity-compatible alternatives.

This study introduces RT-HMD, a Hardware-based Malware Detector (HMD) for mobile devices, that refines malware representation in segmented time-series through a Multiple Instance Learning (MIL) approach. We address the mislabeling issue in real-time HMDs, where benign segments in malware time-series incorrectly inherit malware labels, leading to increased false positives. Utilizing the proposed Malicious Discriminative Score within the MIL framework, RT-HMD effectively identifies localized malware behaviors, thereby improving the predictive accuracy. Empirical analysis, using a hardware telemetry dataset collected from a mobile platform across 723 benign and 1033 malware samples, shows a 5% precision boost while maintaining recall, outperforming baselines affected by mislabeled benign segments.

In this article, we introduce a processing-in-memory (PIM)-based satisfiability (SAT) solver called Processing-in-memory-based SAT solver using a Recurrent Stochastic neural network (PRESTO), a mixed-signal circuit-based PIM (MSC-PIM) architecture combined with a digital finite state machine (FSM) for solving SAT problems. The presented design leverages a stochastic neural network with unsupervised learning. PRESTO’s architecture supports fully connected -SAT clauses with mixed- problems, highlighting its versatility in handling a wide range of SAT challenges. A test chip is fabricated in 65-nm CMOS technology with a core size of 0.4 mm and demonstrates an operating frequency range of 100–500 MHz and a peak power of 35.4 mW. The measurement results show that PRESTO achieves a 74.0% accuracy for three-SAT problems with 30 variables and 126 clauses.

Locally interacting dynamical systems, such as epidemic spread, rumor propagation through crowd, and forest fire, exhibit complex global dynamics originated from local, relatively simple, and often stochastic interactions between dynamic elements. Their temporal evolution is often driven by transitions between a finite number of discrete states. Despite significant advancements in predictive modeling through deep learning, such interactions among many elements have rarely explored as a specific domain for predictive modeling. We present Attentive Recurrent Neural Cellular Automata (AR-NCA), to effectively discover unknown local state transition rules by associating the temporal information between neighboring cells in a permutation-invariant manner. AR-NCA exhibits the superior generalizability across various system configurations (i.e., spatial distribution of states), data efficiency and robustness in extremely data-limited scenarios even in the presence of stochastic interactions, and scalability through spatial dimension-independent prediction.

Learning accurate, data-driven predictive models for multiple interacting agents following unknown dynamics is crucial in many real-world physical and social systems. In many scenarios, dynamics prediction must be performed under incomplete observations, i.e., only a subset of agents are known and observable from a larger topological system while the behaviors of the unobserved agents and their interactions with the observed agents are not known. When only incomplete observations of a dynamical system are available, so that some states remain hidden, it is generally not possible to learn a closed-form model in these variables using either analytic or data-driven techniques. In this work, we propose STEMFold, a spatiotemporal attention-based generative model, to learn a stochastic manifold to predict the underlying unmeasured dynamics of the multi-agent system from observations of only visible agents. Our analytical results motivate STEMFold design using a spatiotemporal graph with time anchors to effectively map the observations of visible agents to a stochastic manifold with no prior information about interaction graph topology. We empirically evaluated our method on two simulations and two real-world datasets, where it outperformed existing networks in predicting complex multiagent interactions, even with many unobserved agents.

Compute-In-Memory (CIM) has gained prominence as a promising hardware architecture for Machine Learning Accelerators (MLA) within the landscape of Intelligent Sensors (IS). The acceleration of Deep Neural Networks (DNNs) by MLAs highlights the need for improved energy efficiency. In recent years, CIM-aware DNN model compression techniques, such as low-precision quantization, have been extensively investigated to enhance the energy efficiency of TinyML models for edge devices. However, existing approaches primarily focus on post-training compression of pre-trained models and overlook the energy consumption during compression-aware training. In this paper, we propose a Hamming weight-based quantization framework, named HamQ, to enhance the energy efficiency of analog CIM. A key contribution of this work is in a novel regularizer to reduce Hamming weight of quantized model weights …

We propose a mixed-signal Compute-In-Memory (CIM) based beamforming accelerator (BeamCIM) that uses linear embedding to transform high-dimensional analog inputs to lower dimension digital features and perform digital beamforming using those features. We show that BeamCIM reduces both digitization (number of ADCs) and computation requirements while maintaining beamforming quality. Simulations in 28nm CMOS for a 256 input to 16 output beam system shows that proposed system achieves 19 × reduction in area and 14 × reduction in power compared to a baseline true time delay based digital implementation of a beamformer.

Spiking Neural Networks (SNNs) have become an essential paradigm in neuroscience and artificial intelligence, providing brain-inspired computation. Recent advances in literature have studied the network representations of deep neural networks. However, there has been little work that studies representations learned by SNNs, especially using unsupervised local learning methods like spike-timing dependent plasticity (STDP). Recent work by \cite{barannikov2021representation} has introduced a novel method to compare topological mappings of learned representations called Representation Topology Divergence (RTD). Though useful, this method is engineered particularly for feedforward deep neural networks and cannot be used for recurrent networks like Recurrent SNNs (RSNNs). This paper introduces a novel methodology to use RTD to measure the difference between distributed representations of RSNN models with different learning methods. We propose a novel reformulation of RSNNs using feedforward autoencoder networks with skip connections to help us compute the RTD for recurrent networks. Thus, we investigate the learning capabilities of RSNN trained using STDP and the role of heterogeneity in the synaptic dynamics in learning such representations. We demonstrate that heterogeneous STDP in RSNNs yield distinct representations than their homogeneous and surrogate gradient-based supervised learning counterparts. Our results provide insights into the potential of heterogeneous SNN models, aiding the development of more efficient and biologically plausible hybrid artificial intelligence systems.

This paper presents a novel denoising algorithm with the untrained deep neural network (DNN) for Radio Frequency (RF) signal modulation recognition considering the mobile and Internet of Things (IoT) applications. In denoising with the untrained DNN, which is called deep image prior (DIP), a DNN is used as a deep generative network to generate the less noisy signal from the pure noise. Considering denoiser for RF signal, which can’t apply the denoising algorithm selectively, we propose a new DIP algorithm called hybrid DIP to improve the data reproducing capability when the noise level is low. The experimental results show that the proposed method can increase the classification accuracy of the received RF signals, especially with low signal-to-noise ratio (SNR), with minimal impact on the signal with high SNR.

This paper presents aging simulation studies on an 8T-SRAM based Analog Computing-In-Memory (ACIM) circuit in the 65nm CMOS technology. Our methodology integrates device-level aging models and circuit simulations in SPICE. We assess the impact of Hot Carrier Injection (HCI) and Negative Bias Temperature Instability (NBTI) reliability degradation mechanisms on the accuracy of ACIM. The results reveal that aging effects amplify computational errors, with HCI in NMOS devices within the 2T path demonstrating a more pronounced aging-induced degradation compared to NBTI.

Automatic Modulation Classification (AMC) is a technique used in wireless communication systems to identify the modulation type of received signals at the receiver. Improving spectrum utilization efficiency is essential for AMC and is carried out by fundamental signal processing methods within the physical layer. Convolutional Neural Networks (CNN) based deep-learning models have recently been employed in AMC systems, demonstrating superior performance. However, the large size of CNN models, floating-point weights, and activations make deploying such systems with limited hardware resources quite complex. In this paper, we propose a hybrid Radio Frequency-based Machine Learning (RFML) model that combines Short-time Fourier Transform and Convolutional Neural Network (STFT-CNN) for AMC. Simulations on RadioML2016.10a demonstrate an average recognition accuracy of 79% for a Signal …

In the “Beyond Moore’s Law” era, with increasing edge intelligence, domain-specific computing embracing unconventional approaches will become increasingly prevalent. At the same time, the adoption of a wide variety of nanotechnologies will offer benefits in energy cost, computational speed, reduced footprint, cyber-resilience and processing prowess. The time is ripe to lay out a roadmap for unconventional computing with nanotechnologies to guide future research and this collection aims to fulfill that need. The authors provide a comprehensive roadmap for neuromorphic computing with electron spins, memristive devices, two-dimensional nanomaterials, nanomagnets and assorted dynamical systems. They also address other paradigms such as Ising machines, Bayesian inference engines, probabilistic computing with p-bits, processing in memory, quantum memories and algorithms, computing with skyrmions …

Neuromorphic event-based cameras can unlock the true potential of bio-plausible sensing systems that mimic our human perception. However, efficient spatiotemporal processing algorithms must enable their low-power, low-latency, real-world application. In this talk, we highlight our recent efforts in this direction. Specifically, we talk about how brain-inspired algorithms such as spiking neural networks (SNNs) can approximate spatiotemporal sequences efficiently without requiring complex recurrent structures. Next, we discuss their event-driven formulation for training and inference that can achieve realtime throughput on existing commercial hardware. We also show how a brain-inspired recurrent SNN can be modeled to perform on event-camera data. Finally, we will talk about the potential application of associative memory structures to efficiently build representation for event-based perception.

One of the primary challenges of deploying deep neural networks (DNNs) is ensuring their reliable performance in unpredictable edge environments, which are often disrupted by a variety of uncertainties and variations. Estimating uncertainty is crucial in order to understand the reliability of task predictions and prevent system failures. However, quantifying uncertainty stemming from non-ideal properties of processing hardware has not yet been thoroughly studied. To address this, we present Cross-Layer Uncertainty Estimator (CLUE), which quantifies task uncertainty originating from both sensing/processing hardware variations and DNN algorithm uncertainty. Our experimental results demonstrate that CLUE provides uncertainty with up to 80.4% less calibration error and only 12% of energy overheads compared to using task DNN solely. Furthermore, CLUE is able to detect unreliable tasks that stem from …

Hardware-based Malware Detectors (HMDs) have shown promise in detecting malicious workloads. However, the current HMDs focus solely on the CPU core of a System-on-Chip (SoC) and, therefore, do not exploit the full potential of the hardware telemetry. In this paper, we propose XMD, an HMD that uses an expansive set of telemetry channels extracted from the different subsystems of SoC. XMD exploits the thread-level profiling power of the CPU-core telemetry, and the global profiling power of non-core telemetry channels, to achieve significantly better detection performance than currently used Hardware Performance Counter (HPC) based detectors. We leverage the concept of manifold hypothesis to analytically prove that adding non-core telemetry channels improves the separability of the benign and malware classes, resulting in performance gains. We train and evaluate XMD using hardware telemetries …

Analog computing-in-memory (ACIM) is a promising technology that performs computation on the bitlines to alleviate memory bottleneck, but the power consumption of peripheral circuits limits its power efficiency (TOPS/Watt). This work shows that a cryogenically cooled ACIM (Cool-CIM) can leverage improved device characteristics, such as higher I ON /I OFF ratio, and higher transconductance, to lower the operating voltage and power of peripheral circuits. Circuit simulations with empirically calibrated models of 14nm FinFET at cryogenic temperature show 54% reduction in EDP and 80% increase in TOPS/W for voltage optimized cryogenic ACIM compared to 300K operation. The study establishes the feasibility of in-sensor processing of analog signals in cryogenic infrared sensors.

We present an unsupervised deep learning model for 3D object classification. Conventional Hebbian learning, a well-known unsupervised model, suffers from loss of local features leading to reduced performance for tasks with complex geometric objects. We present a deep network with a novel Neuron Activity Aware (NeAW) Hebbian learning rule that dynamically switches the neurons to be governed by Hebbian learning or anti-Hebbian learning, depending on its activity. We analytically show that NeAW Hebbian learning relieves the bias in neuron activity, allowing more neurons to attend to the representation of the 3D objects. Empirical results show that the NeAW Hebbian learning outperforms other variants of Hebbian learning and shows higher accuracy over fully supervised models when training data is limited.

Large multi-agent systems such as real-time strategy games are often driven by collective behavior of agents. For example, in StarCraft II, human players group spatially near agents into a team and control the team to defeat opponents. In this light, clustering the agents in the game has been used for various purposes such as the efficient control of the agents in multi-agent reinforcement learning and game analytic tools for the game users. However, despite the useful information provided by clustering, learning the dynamics of multi-agent systems at a cluster level has been rarely studied yet. In this paper, we present a hybrid AI model that couples unsupervised and self-supervised learning to forecast evolution of the clusters in StarCraft II. We develop an unsupervised Hebbian learning method in a set-to-cluster module to efficiently create a variable number of the clusters with lower inference time complexity than K …

In this work, we present an efficient, reconfigurable solution to the memory- and compute-expensive problem of quantum state tomography implemented in an FPGA architecture. Based on the unique characteristics of the closed-form least squares solution for tomographically complete quantum measurement sets, we are able to encode the necessary infor-mation for any multi-qubit system into an algorithm-hardware co-design scheme and solve for the quantum state yielding the measurement results without any traditional multiplication or division blocks in FPGA. We demonstrate this method on a basic Pynq-Z2 FPGA with various approaches resulting in exponential speedup over both direct closed-form implementation and opti-mizer methods with negligible accuracy loss. This architecture is easily reconfigurable to any number of qubits and enables rapid reconstruction of quantum states from measurement data …