Wei Wang

Despite the demonstrated empirical efficacy of prompt tuning to adapt a pretrained language model for a new task, the theoretical underpinnings of the difference between" tuning parameters before the input" against" the tuning of model weights" are limited. We thus take one of the first steps to understand the role of soft-prompt tuning for transformer-based architectures. By considering a general purpose architecture, we analyze prompt tuning from the lens of both: universal approximation and limitations with finite-depth fixed-weight pretrained transformers for continuous-valued functions. Our universality result guarantees the existence of a strong transformer with a prompt to approximate any sequence-to-sequence function in the set of Lipschitz functions. The limitations of prompt tuning for limited-depth transformers are first proved by constructing a set of datasets, that cannot be memorized by a prompt of any length for a given single encoder layer. We also provide a lower bound on the required number of tunable prompt parameters and compare the result with the number of parameters required for a low-rank update (based on LoRA) for a single-layer setting. We finally extend our analysis to multi-layer settings by providing sufficient conditions under which the transformer can at best learn datasets from invertible functions only. Our theoretical claims are also corroborated by empirical results.

BACKGROUNDMany studies have explored the relationship between depression and metabolic syndrome (MetS), especially in older people. China has entered an aging society. However, there are still few studies on the elderly in Chinese communities.AIMTo investigate the incidence and risk factors of depression in MetS patients in mainland China and to construct a predictive model.METHODSData from four waves of the China Health and Retirement Longitudinal Study were selected, and middle-aged and elderly patients with MetS (n= 2533) were included based on the first wave. According to the center for epidemiological survey-depression scale (CESD), participants with MetS were divided into depression (n= 938) and non-depression groups (n= 1595), and factors related to depression were screened out. Subsequently, the 2-, 4-, and 7-year follow-up data were analyzed, and a prediction model for …

The defense of Use-After-Free (UAF) exploits generally could be guaranteed via static or dynamic analysis, however, both of which are restricted to intrinsic deficiency. The static analysis has limitations in loop handling, optimization of memory representation and constructing a satisfactory test input to cover all execution paths. While the lack of maintenance of pointer information in dynamic analysis may lead to defects that cannot accurately identify the relationship between pointers and memory. In order to successfully exploit a UAF vulnerability, attackers need to reference freed memory. However, main existing schemes barely defend all types of UAF exploits because of the incomplete check of pointers. To solve this problem, we propose UAF-GUARD to defend against the UAF exploits via fine-grained memory permission management. Specially, we design two key data structures to enable the fine-grained …

ChatGPT, an artificial intelligence (AI)-powered chatbot developed by OpenAI, is creating a buzz across all occupational sectors. Its name comes from its basis in the Generative Pretrained Transformer (GPT) language model. ChatGPT’s most promising feature is its ability to offer human-like responses to text input using deep learning techniques at a level far superior to any other AI model. Its rapid integration in various industries signals the public’s burgeoning reliance on AI technology. Thus, it is essential to critically evaluate ChatGPT’s potential impacts on academic clinical and translational medicine research.

Anchor link prediction exacerbates the risk of privacy leakage via the de-anonymization of social network data. Embedding-based methods for anchor link prediction are limited by the excessive similarity of the associated nodes in a latent feature space and the variation between latent feature spaces caused by the semantics of different networks. In this article, we propose a novel method which reduces the impact of semantic discrepancies between different networks in the latent feature space. The proposed method consists of two phases. First, graph embedding focuses on the network structural roles of nodes and increases the distinction between the associated nodes in the embedding space. Second, a federated adversarial learning framework which performs graph embedding on each social network and an adversarial learning model on the server according to the observable anchor links is used to associate …

Recently, an exciting experimental conclusion in Li et al. (Knowl Inf Syst 62(2):611–637, ) about measures of uncertainty for knowledge bases has attracted great research interest for many scholars. However, these efforts lack solid theoretical interpretations for the experimental conclusion. The main limitation of their research is that the final experimental conclusions are only derived from experiments on three datasets, which makes it still unknown whether the conclusion is universal. In our work, we first review the mathematical theories, definitions, and tools for measuring the uncertainty of knowledge bases. Then, we provide a series of rigorous theoretical proofs to reveal the reasons for the superiority of using the knowledge amount of knowledge structure to measure the uncertainty of the knowledge bases. Combining with experiment results, we verify that knowledge amount has much better performance for …

This paper investigates the attitude tracking control problem of rigid spacecraft with inertia parameter identification. Based on the relative attitude and angular velocity error dynamics, a basic adaptive backstepping based attitude tracking control scheme is firstly designed such that asymptotic attitude tracking can be achieved. However, the parameter identification error cannot decay to zero if the persistent excitation (PE) condition is not satisfied. To solve this issue, a filtering‐based concurrent learning adaptive backstepping control scheme is then proposed, by incorporating torque filtering technique with concurrent learning technique. A more mild rank condition, which consists of some collectable historical data, is provided to guarantee the convergence of parameter identification error. In addition, a valid data collection algorithm is given. It should be mentioned that a distinctive feature of the proposed filtering …

We present a deep-learning-based platform, MIND-S, for protein post-translational modification (PTM) predictions. MIND-S employs a multi-head attention and graph neural network and assembles a 15-fold ensemble model in a multi-label strategy to enable simultaneous prediction of multiple PTMs with high performance and computation efficiency. MIND-S also features an interpretation module, which provides the relevance of each amino acid for making the predictions and is validated with known motifs. The interpretation module also captures PTM patterns without any supervision. Furthermore, MIND-S enables examination of mutation effects on PTMs. We document a workflow, its applications to 26 types of PTMs of two datasets consisting of ∼50,000 proteins, and an example of MIND-S identifying a PTM-interrupting SNP with validation from biological data. We also include use case analyses of targeted …