Abstract
Intrinsically disordered proteins (IDPs) differ from folded proteins in that they are dynamic, lack a stable three-dimensional conformation, and have low sequence similarity between similar proteins. The conformational heterogeneity of IDPs - while beneficial for their diverse functions - limits the use of traditional experimental tools to determine their conformation. The experimental difficulty, along with low sequence similarity, results in data scarcity, and makes it difficult to classify/detect IDPs that are similar or dissimilar, a task relevant to understand biology and evolution. We address this challenge using Multi-task ProtBERT (MT-ProtBERT), a multi-task extension of ProtBERT tailored for low-data regimes. MT-ProtBERT integrates Dynamic Window Masking, a Multi-Scale 1D Convolutional classifier (MS-Conv1D), and auxiliary objectives that jointly optimize masked language modeling and biochemistry-informed tasks. We evaluate this framework on two tasks under limited data: (i) phosphorylation site prediction (S/T/Y) in short sequences and small datasets, and (ii) protein compaction prediction on two small datasets (684 and 530 sequences), including sequences comparable in length to typical disordered regions. MT-ProtBERT consistently outperforms PARROT, an RNN-based IDP-specific model, across all tasks. These results demonstrate that combining self-supervised and biochemistry-informed tasks, and multi-scale learning enables robust modeling of unstructured proteins under data scarcity.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Sun, J., Ghosh, K., Houston, L., & Mahoor, M. H. (2026). MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data. https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data
MLA 9
Sun, Jian, et al. "MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data." https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data.
Chicago (author–date)
Sun, Jian, Kingshuk Ghosh, Lilianna Houston, and Mohammad H. Mahoor. 2026. "MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data." https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data.
Harvard
Sun, J., Ghosh, K., Houston, L. and Mahoor, M. H. (2026) 'MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data', Available at: https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data.
Vancouver
Sun J, Ghosh K, Houston L, Mahoor MH. MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data. https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data
IEEE
J. Sun, K. Ghosh, L. Houston, and M. H. Mahoor, "MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data," https://omanscience.com/en/articles/mt-protbert-multi-task-learning-protbert-for-intrinsically-disordered-proteins-classification-with-scarce-data.