Predicting Features of Human Mental Disorders through Methylation Profile and Machine Learning Models

Xiong, Zihan; Zhang, Xuanhua; Zhang, Mengxuan; Cao, Bosen

doi:10.1109/mlbdbi51377.2020.00019

Cited by 7 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This block consists of two sub-layers: a multi-head self-attention mechanism and a point-wise fully connected feed-forward network 23 . Layer normalization was applied before each sub-layer to stabilize the training process 27 . Additionally, dropout and residual addition were implemented after each sub-layer to reduce the risk of overfitting and mitigate the vanishing gradient problem.…”

Section: Methodsmentioning

confidence: 99%

“…These features are then input into the Transformer block, which captures complex dependencies between them by modeling positional relationships through positional encoding and attention mechanism. The Pre-LN configuration stabilizes the training process, eliminating the need for a learning rate warm-up stage required by the original Post-Layer Normalization (Post-LN) setting 23,27 . Pre-trained with vast amounts of data, this model can function as a foundational model that can be adapted to various new diseases and tissues through fine-tuning.…”

Section: Mainmentioning

confidence: 99%

See 1 more Smart Citation

T-cell receptor sequences correlate with and predict gene expression levels in T cells

Wang,

2023

Preprint

View full text Add to dashboard Cite

T cells exhibit high heterogeneity in both their gene expression profiles and antigen specificities. We analyzed fifteen single-cell immune profiling datasets to systematically investigate the association between T-cell receptor (TCR) sequences and the gene expression profiles of T cells. Our findings reveal that T cells sharing identical or similar TCR sequences tend to have highly similar gene expression profiles. Additionally, we developed a foundational model that leverages TCR information to predict gene expression levels in T cells.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Mainmentioning

confidence: 99%

T-cell receptor sequences correlate with and predict gene expression levels in T cells

Wang,

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Fig. 1-b illustrates the design of LucaOne, which utilizes the Transformer-Encoder ( 21 ) archi-tecture with the following enhancements: The vocabulary of LucaOne comprises 39 tokens, including both nucleotide and amino acid symbols(refer to Supplementary B ); The model employs Pre-Layer Normalization over Post-Layer Normalization, facilitating the training of deeper networks ( 45 ); Rotary Position Embedding(RoPE ( 46 )) is implemented instead of absolute positional encoding, enabling the model to handle sequences longer than those seen during training; It incorporates mixed training of nucleic acid and protein sequences by introducing token-type embeddings, assigning 0 for nucleotides and 1 for amino acids; Besides the pre-training masking tasks for nucleic acid and protein sequences, eight supervised pre-training tasks have been implemented based on selected annotation informa-tion(refer to Supplementary D ). …”

Section: Model Architecturementioning

confidence: 99%

“…The model employs Pre-Layer Normalization over Post-Layer Normalization, facilitating the training of deeper networks ( 45 );…”

Section: Model Architecturementioning

confidence: 99%

LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language

He,

Fang,

Shan

et al. 2024

Preprint

View full text Add to dashboard Cite

In recent years, significant advancements have been observed in the domain of NLP with the introduction of pre-trained foundational models, paving the way for utilizing similar AI technologies to interpret the language of biology. In this research, we introduce "LucaOne", a novel pre-trained foundational model designed to integratively learn from the genetic and proteomic languages, encapsulating data from 169,861 species encompassing DNA, RNA, and proteins. This work illuminates the potential for creating a biological language model aimed at universal bioinformatics application. Remarkably, through few-shot learning, this model efficiently learns the central dogma of molecular biology and demonstrably outperforms competing models. Furthermore, in tasks requiring inputs of DNA, RNA, proteins, or a combination thereof, LucaOne exceeds the state-of-the-art performance using a streamlined downstream architecture, thereby providing empirical evidence and innovative perspectives on the potential of foundational models to comprehend complex biological systems.

show abstract

“…Classification of cases and controls for certain diseases is also performed using DNA methylation data. Examples of machine learning applications using epigenetic data include classification of coronary heart disease, neurodevelopmental syndromes, schizophrenia, Alzheimer's disease, psychiatric disorders and others [33][34][35][36][37][38][39].…”

Section: Introduction 1backgroundmentioning

confidence: 99%

Disease classification for whole blood DNA methylation: meta-analysis, missing values imputation, and XAI

Kalyakulina

Yusipov

Bacalini

et al. 2022

Preprint

View full text Add to dashboard Cite

Background: DNA methylation has a significant effect on gene expression and can be associated with various diseases. Meta-analysis of available DNA methylation datasets requires development of a specific pipeline for joint data processing. Results: We propose a comprehensive approach of combined DNA methylation datasets to classify controls and patients. The solution includes data harmonization, construction of machine learning classification models, dimensionality reduction of models, imputation of missing values, and explanation of model predictions by explainable artificial intelligence (XAI) algorithms. We show that harmonization can improve classification accuracy by up to 20% when preprocessing methods of the training and test datasets are different. The best accuracy results were obtained with tree ensembles, reaching above 95% for Parkinson's disease. Dimensionality reduction can substantially decrease the number of features, without detriment to the classification accuracy. The best imputation methods achieve almost the same classification accuracy for data with missing values as for the original data. Explainable artificial intelligence approaches have allowed us to explain model predictions from both populational and individual perspectives. Conclusions: We propose a methodologically valid and comprehensive approach to the classification of healthy individuals and patients with various diseases based on whole blood DNA methylation data using Parkinson's disease and schizophrenia as examples. The proposed algorithm works better for the former pathology, characterized by a complex set of symptoms. It allows to solve data harmonization problems for meta-analysis of many different datasets, impute missing values, and build classification models of small dimensionality.

show abstract

Predicting Features of Human Mental Disorders through Methylation Profile and Machine Learning Models

Cited by 7 publications

References 6 publications

T-cell receptor sequences correlate with and predict gene expression levels in T cells

T-cell receptor sequences correlate with and predict gene expression levels in T cells

LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language

Disease classification for whole blood DNA methylation: meta-analysis, missing values imputation, and XAI

Contact Info

Product

Resources

About